When you think about experimentation, what images come to mind? Perhaps you picture a clean white laboratory, with people in white aprons, masks, and gloves. Maybe you recall science classes filled with test tubes and various mixtures, or a homework assignment to grow a potato in a pot or crystals from salt. Indeed, these are classic examples of experimentation. Through these activities, we gained a better understanding of the world and learned more effectively. So, it’s natural to associate experimentation with students and scientists. But if you don’t belong to either group, you might wonder why experimentation should matter to you.
Here’s the surprise: students and scientists share a crucial commonality—learning by doing and navigating uncertainties. However, I’m not asking you to see yourself as a student at a school desk or as someone in an apron and protective goggles. Instead, I want to show you how the right approach to experimentation, tailored to your situation, can aid in decision-making and waste reduction. Let’s explore how experimentation fits into the context of product development!
The world is full of assumptions
Probably, this might come as another surprise to you. We make assumptions quite often in many different situations. I would even go as far as to say that making assumptions could be seen as part of our daily routine. Let’s take a look at the following statements.
- I assume that my colleague understood my request the way I intended.
- The team assumes that the chosen solution will solve their customer’s problem.
- The cost calculations for building a new house are based on the assumption that prices will not continue to rise.
Did you notice the assumptions? I hope they stand out, especially since the words related to ‘assuming’ were used explicitly. However, it might not be so obvious at first glance if you're not clear about what making an assumption involves. Let’s look at some other examples.
- We will increase user satisfaction if we deliver feature ABC.
- Our product has a low market share because users don’t like our User Interface.
- He didn’t manage to finish his task because he’s already overwhelmed with all his responsibilities.
Do you see the assumptions in the statements above? Not so sure? Let’s start with the definition of assumption before we move forward.
An assumption is an assertion or statement that is taken as true or supposed as a fact without proof or substantiating evidence [1].
We tend to make assumptions frequently, as it helps us navigate the complexities of daily life. While this can simplify our decision-making process and reduce the cognitive load on our brains, there’s a difficult truth we must confront: when making an assumption, we can be completely wrong (note: we can also be right, but without evidence it might be called a ‘guessing game’).
Assumptions in product development
Ready to go back to our examples? Let’s do it! Where are the potential assumptions? As you read the statements, look for words that describe future consequences or provide explanations, but... without clear evidence or sufficient data to support them.
- We will increase user satisfaction if we deliver feature ABC.
- Our product has a low market share because users don’t like our User Interface.
- He did not manage to finish his task because he’s already overwhelmed with all his responsibilities.
In the first example, we assume that users will be happy with feature ABC. But how do we know that? What led us to choose this particular feature? In the second example, we assume that users don’t like the UI and that this is related to our market share (did you notice that we actually have two assumptions here? One might be true, both might be true, or neither). In the last sentence, we assume someone is overwhelmed with responsibilities, and that’s why they couldn’t finish their task. This is another interesting example, because human beings are complex, and there could be many reasons why someone is unable to complete a task. Moreover, the outcomes might also be influenced by a combination of different factors.
The consequences of blindly accepting these assumptions can be costly and uncomfortable. We could build a feature nobody wants, leading not only to a lack of increased user satisfaction but also to wasted time and money. We might rebuild the entire User Interface only to find that it has little impact and that the previous version was good enough (and the market share remains low). We might even demotivate someone by taking away their favorite tasks, when they actually needed a completely different solution to their problem. This could result in losing a valuable expert who was instrumental in developing your best products.
These examples illustrate just a few scenarios of what can happen if we ignore the fact that we’ve made assumptions and treat them as truths. This opens the door to various types of waste, which can adversely affect the company. That’s why the ability to work with assumptions is an important skill to develop.
How to to deal with assumptions
So, is there a way to effectively deal with assumptions? Try the following strategies:
- Make Them Transparent: Clearly distinguish between what is fact and what is assumption in the statements you make.
- Question Them: Seek out data or evidence that supports or contradicts the assumption. Maybe the data is somewhere, and you can easily get it. Are there other possible perspectives or explanations? What are the potential consequences if you're wrong? How risky is the assumption?
- Validate Them: If your assumption needs validation (note: not every single assumption should be validated), find ways to test it. Gather data or evidence that either confirms or disproves the assumption. This could involve running a small test, conducting research, or seeking feedback from others. All these activities fall under the umbrella of experimentation.
Experimentation: the way of learning and verifying assumptions
We need a mechanism to effectively validate assumptions, especially those that carry significant risks. In other words, if we accept an assumption as true and it turns out to be wrong, it could lead to costs and undesired consequences. This is where experimentation comes in.
In the context of product development, experimentation can be seen as a way to verify our assumptions while limiting our investments in the learning process. In other words, it’s a (relatively) quick way to determine whether a particular path is worth pursuing, such as deciding if we should build feature ABC. Waiting until the full feature is built to verify the assumption is often too late. Instead, experimentation allows us to gather feedback—evidence and data—earlier in the process, closing the feedback loop. This feedback can either suggest continuing on the current path and potentially taking the next step, or stopping to gather more information or explore other options.
For example, instead of adopting a "King Julien" development style [2], we could look for ways to test our assumption about user satisfaction increase related to the feature ABC. It might be helpful to ask ourselves a few questions, for instance:
- What is the quickest, simplest, or most effective way to learn about our assumption?
- How can we learn as much as possible without fully building feature ABC?
- How strong of a signal do we need? What is the required level of evidence?
We might choose to conduct interviews with a group of users or observe them to identify moments of dissatisfaction that align with the feature idea. Does the feature address at least one user’s pain point? We could also prototype the feature and see how users interact with it to gather feedback. If our assumption is wrong, experimentation helps reduce potential waste and may reveal new paths and ideas that we hadn’t considered before. The world of experimentation offers techniques with different levels of evidence that require varying levels of investment. You can choose the approach that works best for your current situation.
Is it for me?
If you are facing complex challenges or developing products, experimentation can be a powerful tool! I believe that speed is a crucial aspect of our work—not just in terms of delivery (which, in my experience, is a major focus in many companies) but more importantly in terms of learning and making informed decisions based on evidence. The Scrum Guide mentions experimentation among examples of product-related activities:
"The Scrum Team is responsible for all product-related activities from stakeholder collaboration, verification, maintenance, operation, experimentation, research and development, and anything else that might be required." [3]
This serves as an important reminder for us. Experimentation can be considered an important citizen of our Product Backlogs. When we choose to experiment, we do not necessarily become scientists in a laboratory; rather, we apply scientific thinking to better navigate the complexity of product development.
[1] https://www.law.cornell.edu/wex/assumption
[2] “Hurry up, before we come to our senses!”; King Julien is a character from the Madagascar movie
[3] Scrum Guide, version 2020
Hello! I'm Joanna. I work as a trainer, consultant, mentor, and coach in Action Learning. I'm passionate about helping individuals, teams, and organizations tackle complex challenges. Enjoying my content? Want to learn more? You're warmly invited to get in touch with me: