On science...

Many moons ago I worked in a lab as part of my doctoral thesis. I was running experiments on what affected oxygenation of red blood cells (RBCs) in a non-equilibrium environment.

After all, we already new that pH, CO2, and a few other factors accelerated or decelerated the uptake of O2 by the RBCs... but all of these studies had been run under equilibrium.

Thanks to a new device, which I had co-developed, we could simulate the real environment in our lung capillaries much better.

Before running the experiments my mentors had walked me through the scientific method which is

"a method of procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses."

(Source: Oxford Languages)

This method has been the backbone of almost all human advancement since the 17th century. It helped define a huge number of laws in natural sciences and make progress in medicine and technology. There is no internet without the scientific method, no modern drugs, no mobile phones, no cars, no airplanes...

Sidenote: Empiricsim is a fundamental element of the scientific method. Agility is the application of empiricism, hence the scientific method, to product and organizational development.

Yet, even today the vast majority of people does not know how to apply the scientific method. They do not understand what "n" i.e. the number of subjects/runs is needed to consider something as evidence. They do not know how to setup an experiment. They do not know how to analyze the experiment...

Let me give you an example. Yesterday, I was at a conference. In total, the talks were good... not great, but good. One particular talk stood out for me. The presenter (I won't mention their name) was very charismatic and seemed to be a very nice and intelligent person. They could articulate themselves really well and had a great stage presence.

But... the case they presented was scientifically completely flawed. First, in order to assess whether people had a certain trait, they only asked people to do a self-assessment. We know that most people are not good at self-assessing. It's like asking people whether they are intelligent instead of running an IQ test.

Second, in order to understand how those people had achieved that trait, they asked the subjects to name what they believe was most important to achieve that trait. And as both - the people who self-assessed as strong and those that self-assed as weak - had mentioned the same thing, they did not consider that to be significantly relevant with regards to developing the trait.

Let me use a hypothetical case to demonstrate the flawed approach. Imagine we want to assess people's ability to drive... I know this can result in a lot of debate but hang in with me.

Instead of coming up with a test to do a decent assessment, we ask people to score themselves on a scale from 1 (significantly below average) to 5 (significantly above average) with regards to their driving skills.

We might get a nice bell curve and believe "as we got a bell curve, this seems to be scientifically valid."

In our next step, we decide we want to understand what makes great drivers great. And in order to do that, we ask the same subjects what they believe is key to becoming a great driver. And all of them - no matter how they scored - mention more or less the same things e.g. driving practice, having sufficient sleep before driving, and not driving under influence of any drugs.

As all subjects mention that, and not only the ones who self-assessed as 4 or 5, we believe these three practices can't be relevant for being a great driver.

This experiment design and analysis is flawed from start to finish... the results of a flawed experiment do not matter. And based on these results we should not come up with any model on how to create great drivers.

Yet, this was exactly what the speaker had done. They had come up with their own model - based on a completely flawed experiment - to achieve a certain trait. And worst of all, they guaranteed - multiple times - that their approach would result in the desired trait.

Especially in the Agile or Product Management community, these types of talks are very damaging to the overall intention of bringing more rigor and scientific discipline to product development.

So if you are a product coach, an agile coach, or an innovation coach - internally or externally - you should be aware of the scientific method. You should be aware of what it is, how it works, and why it is important.

The scientific method did not appear out of nowhere... it was the answer to deal with snake oil salesman and other myths e.g. religion. It had and has the intention to create real progress for humanity through an evidence based approach to wisdom and understanding of our environment and ourselves.

I know it is not easy to apply the scientific method with rigor as we have to move from opinions to evidence and that requires a lot of discipline. #FromNothingComesNothing