Product Owners have a tough job. I was in a Sprint Review recently where the Scrum Team had some stakeholders talking about an idea they thought was excellent. So, like any studious Product Owner, they immediately typed up the idea and added it to their Product Backlog. A few Sprints later, they were working on it, and released it to production. I don’t believe that feature has been used to date.
I am convinced that not all feedback is created equal. There are many different types of feedback, each with their costs and benefits. A great Product Owner knows how to get the best feedback, filter through it, and then act on that feedback.
Many Product Owners come from a background where they are used to eliciting the needs (aka requirements) from other parties. This tendency can carry itself into the Product Owner role, where they are acting more as a proxy for stakeholders. This lends to being a backlog keeper and working with different stakeholders to prioritize requests. The Product Owner role is grossly undervalued if this is all they are doing.
In the Scrum.org Professional Scrum Product Owner class, and as Ron Eringa describes in his blog, a Product Owner should seek to be more of an entrepreneur, and not just be the backlog keeper, but be the vision keeper and promoter. This adds substantially more value to the organization.
One of the best ways the Product Owner can move towards being an entrepreneur is to elicit feedback from a variety of sources to understand their blind spots on what is needed within the product, and make decisions based on that information in an empirical fashion.
Not all feedback is created equal. So how can a Product Owner know what feedback is the most valid and when they have enough information to act? A helpful model I created is called the Feedback Validity Pyramid:
[Conlin Feedback Validity Pyramid]
- What users say, what users need, and what users say they need are three different things.
Explanation from the bottom up:
Uninformed Masses:
- Overview: Soliciting feedback from individuals who do not have a vested interest in the product.
- Benefits: They can bring fresh insights when getting feedback from them as they are not tied into the ecosystem of the product being created.
- Risks: Take their feedback with a heavy grain of salt as they may not be aware of the goals the product is trying to achieve.
Stakeholder Feedback:
- Overview: Soliciting feedback from people with a vested interest in the product.
- Benefits: Stakeholders can be a great source for feedback as they may have context and perspectives that you do not.
- Risks: They can often get caught up with accomplishing their own agendas within the product, which may not necessarily be the best thing for the product.
User Feedback:
- Overview: Soliciting feedback from people that are actual users and consumers of the product.
- Benefits: Receiving perspectives from people that will actually be using the product and how it could integrate with their jobs, pains, and gains they experience.
- Risks: What users say, what users need, and what users say they need are often three different things. Users are inherently bad at conceptualizing new ideas, and can only bring their past experiences when asking for feedback.
User Observation:
- Overview: Watching actual users of your product interact with your product in their natural environment.
- Benefits: Getting to watch how a user makes use of the product can be invaluable for little nuances you may not have considered.
- Risks: People act differently when being observed (the observer effect). Keep this in mind and beware of leading the user to confirm your own biases.
Market Evidence:
- Overview: Looking at data for how your actual product is being used by the market.
- Benefits: Actual data around things like how the product is being used, drop-off rates, conversions, etc. can truly paint the picture for how market reception is going.
- Risks: Confirmation bias can play a part here where data tells the story you want it to tell.
The general trend with these is that the further up the pyramid you go, the more valid the feedback is that you receive (keeping in mind the risks). However, the further up you go, the more expensive the feedback becomes. A Product Owner’s job is to maximize the value of the product. In order to gain new insights, they need to experiment with how they can get the most valid feedback for the best return on investment. If you’re working in a risky area, you’ll probably need to go closer to the top of the pyramid because it’s most likely worth the investment. Are there areas where you can get less valid feedback for a cheaper cost? Probably, but know the risks you have leaving unknowns based solely on feedback from stakeholders or users.
Saras Sarasvathy has a great quote on this, what she calls “effectual reasoning”: “Effectual reasoning may not necessarily increase the probability of success of new enterprises, but it reduces the costs of failure by enabling the failure to occur earlier and at lower levels of investment.”
Here’s an example. Sanjay Saini recounts a time when he was working on building out a mobile app for a warehouse to more quickly scan packages. They had several rounds of feedback from stakeholders before releasing to the warehouse that would use it. Only one problem: the people in the warehouse couldn’t use the app because they were required to wear gloves, making touch functions impossible! This Scrum Team thought they were doing great. They were getting feedback from stakeholders, but missed several really important things that could have been prevented if they moved further up the pyramid, even to the user feedback level.
Take aways:
Using this model, what kind of feedback are you getting today? Is it the most valid? Are you increasing your risk by not getting true market data? How can you release your product earlier? Stay tuned for a blog on how to decrease your investment cycle.
If you found this blog helpful, see below for ways to connect and learn more:
Check out our upcoming classes!
Want to chat? Reach out to info@unstuckagile.com so we can help you get Unstuck or Subscribe to our Newsletter.
Also, subscribe to our YouTube channel or connect with us on LinkedIn.
