TL; DR: Product Discovery Anti-Patterns
Scrum has proven to be an effective product delivery framework for all sorts of products. However, Scrum is equally well suited to build the wrong product efficiently as its Achilles heel has always been the product discovery part. What product discovery part, you may think now. And this is precisely the point: The Product Owner miraculously identifies what is the best way to proceed as a Scrum Team by managing the Product Backlog. How that is supposed to happen is nowhere described in the Scrum Guide. Consequently, when everyone is for themselves, product discovery anti-patterns emerge.
From sunk costs, HIPPO-ism, my-budget-my-features to self-fulfilling prophecies — learn more about the numerous product discovery anti-patterns that can manifest themselves when you try to fill Scrum’s product discovery void.
🗞 Shall I notify you about articles like this one? Awesome! You can sign up here for the ‘Food for Agile Thought’ newsletter and join 30,000-plus other subscribers.
🎓 Join Stefan in one of his upcoming Professional Scrum training classes!
Scrum’s Achilles Heel: Product Discovery
In the attempt to fill Scrum’s product discovery void, product delivery organizations regularly turn to other agile frameworks like lean UX, jobs-to-be-done, lean startup, design thinking, design sprint—just to name a few. The wave of agile transition projects, particularly in large, established organizations over recent years, has provided those frameworks with a tremendous tailwind. Meanwhile, some ideas have gained buzzword status in the process, causing the occasional collateral damage along the way. (Think of “MVP.”)
Generally, it is safe to assume that product discovery anti-patterns result from a broader set of issues by comparison to the anti-patterns of a particular framework such as Scrum. (Download for free: The Scrum Anti-Patterns Guide.)
The main contributing variables to various product discovery anti-patterns are:
- Existing organizational dysfunctions, for example, the organization is structured in functional silos,
- A substantial degree of ego issues among individual players—the what-is-in-for-me-syndrome—resulting in personal agendas being pursued,
- A complex, multi-layered reporting structure within organizations that filters as well as delays the flow of information, thus impeding communication and decision-making.
Additionally, particularly larger organizations struggle with the necessary transition at the core of the process of becoming an agile, learning organization: The move from allocating budgets to projects staffed with temporary teams group of FTEs to building and funding lasting, empowered product teams.
I do believe that a separation between product discovery on the one side (product management), and product delivery (Scrum teams) on the other side is no longer a viable approach. To prevail in today’s game of an accelerated innovation-based competition—software is eating the world—, every organization needs to acquire a holistic understanding of an agile product creation process:
A vision leads to a strategy that (probably) results in a portfolio of products (and services). Each of those products has a product roadmap that needs to be reflected in the product’s backlog, which ultimately provides the Sprint Backlog. This core agile product creation process is agnostic to the product delivery framework, be it Scrum, XP, or any other framework.
The organization needs to inspect and adapt every layer of this hierarchy continuously within appropriate time intervals. Apparently, there is a difference in the inspect & adapt cadence when product strategy and Sprint Backlog are compared to each other. (Example: The Secret Tesla Motors Master Plan (just between you and me) from August 2nd, 2006.)
Keeping this holistic product creation process in mind, you can often observe some of the following product discovery anti-patterns in practice.
Product Discovery Anti-Patterns: The Fallacy of Knowing Better
- ‘We know what to build’: The is no user research, nor any other interactions of the product delivery organization with customers. Several reasons are causing this phenomenon ranging from a founder or entrepreneur who pursues his or her product vision without engaging in customer discovery activities. Or the product delivery organization is solely briefed indirectly by key account managers. The sales department probably deems a direct contact of the Scrum Team members with customers too risky and hence prevents it from happening. What these patterns share is either a bias that is hurting the learning effort or a personal agenda. While the former can be overcome by education, the latter is more difficult to come by as the culprits typically reject the idea that selfish motives guide them. For becoming an effective product delivery organization, it is essential that the team directly communicate with customers regularly.
- Loving the solution: This is a variation of the ‘we know what to build’ syndrome. There are no ugly children, at least not from their parents’ perspective. This bias seems to apply to products and services, too; think of Ikea: people value what they created over other products. This is also why you avoid asking strangers for their opinion: They might not be sharing your conviction. The solution to this product discovery anti-pattern is simple: Fall in love with the problem, not your solution.
- No shared understanding: The product delivery organization does not make any attempts to create a shared understanding among the team members and the stakeholders on what to build for which reason. The best way to secure engaged and effective communication with internal and external stakeholders is to include them in the product creation process. People care for what they help create. The best way to do so, in my experience, is Jeff Patton’s user story mapping. It is not just a great way to turn ‘requirements’ into feasible user stories at a tactical level. User story mapping is also very well suited to understanding the product roadmap and release planning.
- Persona-driven self-fulfilling prophecy: You create the personas first based on what you believe to be the customers or users of our next [add an adjective with a positive connotation here] product or service will be. This is a potentially dangerous mental model as you may fall victim to a self-fulfilling prophecy. It is not unlikely that you unconsciously create user tests in a way that will verify your personas in the process. Instead, you should flip the process: first, you observe potential customers in their natural habitat to figure out a problem worth solving. Then you create a prototype addressing that previously identified problem. After the prototype, you create preliminary personas from the insight you gained during the related research. “Preliminary” needs to be stressed, too, as the persona design needs to be constantly calibrated with what you learn while continuously running user tests. There are no static personas. Read More: “Creating Personas.”
- Annual roadmaps: Product roadmaps are planned once a year for twelve months in advance. Contrary to popular belief in traditional command & control organizations, product roadmaps are subject to continuous planning. The agile product delivery process is centered around the idea to work solely on those ideas that provide the highest value to customers and the organization at any given time. The product roadmap needs to be adapted to the learnings from running product experiments regularly in an appropriate cadence to meet that standard.
- The incrementalism trap: You get so focused on improving incrementally that you miss the available innovative leap ahead. Remedy: Apply Elon Musk’s first principles thinking model and break down a situation into its elementary pieces to put it back together again in a way that serves your purpose better.
Join us on March 9, 2021, for a live virtual workshop to create actionable recommendations on how to deal with the outcome of a forensic Product Backlog analysis. (Participation is free.)
Product Discovery Anti-Patterns: Silos at Work
- Design as a silo: A UX/UI design team/department is organized as a functional silo outside the Scrum team. UX/UI competence is essential for a lot of cross-functional teams. If those competencies are only available by sourcing them from a different entity, the organization creates an artificial hand-over. The interruption will impede both the team’s speed and quality as crucial information and learnings will only be relayed but not experienced firsthand.
- User tests/user research as a silo: User tests need to be handed over to a functional silo. The researchers will be reporting back once the results are available. User tests are an integral and thus continuous part of the product discovery process. Hence, user research shall be run by the team itself by embedding someone with the necessary competencies into the team.
- Don’t waste Developers on user tests: The developers do not participate in user tests because they are deemed too valuable/expensive. They are supposed to code as it is easier to hire an additional UX designer by comparison. This is the wrong mental model: the earlier developers participate in identifying a problem worth solving, the better the ROI on their engagement will become as they will bring the technical expertise on the feasibility to the table. Building the wrong thing due to isolating the engineers is, for several reasons, incredibly expensive. It is a colossal waste of resources, and there will be significant costs of delay—the team could have been built something more valuable in the meantime. Lastly, it is incredibly frustrating to see your effort go to waste, no matter what you were paid to make it. (Read More: (1) Daniel Pink: Drive. (2) In Chapter 2—“The Meaning of Labor”—of his book “The Upside of Irrationality,” Dan Ariely describes an experiment dubbed the ‘Building of Bionicles’ (Paperback edition 2011, pages 66 to 77) where Harvard University students were paid to assemble Lego figures. The experiment aimed at a better understanding of people’s reactions to small reductions in the meaning of their work. His conclusion: “The translation of joy into a willingness to work seems to depend to a large degree on how much meaning we can attribute to our own labor.” (See above, page 74.)
- No Developers or designers working in customer support: Developers or designers do not work with customer support from time to time. Have you heard of “support-driven development” that everyone in a company should spend five percent of his or her time in customer support? This is not a new idea, and a lot of successful companies practice it to simplify the flow of information from a client’s problem to its solution. It proves to be an effective way to discover pain points and create products that clients love. Kayak.com co-founder Paul English: “Customers are a big source of my e-mails. Anytime anyone contacts us with a question, whether it’s by e-mail or telephone, they get a personal reply. The engineers and I handle customer support. When I tell people that, they look at me like I’m smoking crack. They say, “Why would you pay an engineer $150,000 to answer phones when you could pay someone in Arizona $8 an hour?” If you make the engineers answer e-mails and phone calls from the customers, the second or third time they get the same question, they’ll actually stop what they’re doing and fix the code. Then we don’t have those questions anymore.” Read More: Everyone on Support.
- A sole source of truth: The product delivery organization relies solely on either quantitative or qualitative feedback. You need to take both sources into account to come to an informed decision. Data does not provide any information ‘why’ things are happening. Data only addresses ‘what’ is happening. Simultaneously, focusing exclusively on qualitative feedback may make you vulnerable to bias and unfavorably mental models. The road to self-fulfilling prophecies is paved with good intentions.
Product Discovery Anti-Patterns: Cargo-Cult Discovery
- Would you use this feature? Your product discovery process equals asking prospective customers what problems they have. “It’s really hard to design products by focus groups. A lot of times, people don’t know what they want until you show it to them.” The famous Steve Jobs quote that everyone in product design and management has heard of before. The fact is that it is useless to ask prospective customers in advance what features they would appreciate in a new product. Whether you send questionnaires or ask them directly in an interview, they will usually fail to imagine both the situation and the new product in a way that will provide you with actionable insight. The problem is called “psychological interference” due to the high level of required abstract thinking. Read More: Four Lessons Learned from Making Customer Value Your Priority.
- Selling non-existing features: What features do you need us to provide to close the deal? Sales managers chase sales objectives by asking prospects for a feature wish-list and give those to the product delivery organization as requirements. Customers’ problem is that they usually lack the depth of knowledge required to provide useful answers to this question. They also lack the level of abstract thinking necessary to come up with a viable, usable, and feasible solution. As the saying goes: if the only tool you are familiar with is a hammer, every problem will look like a nail. Pursuing the sales process in such a way will lead the product into a feature comparison race to the bottom, probably inspired by bonuses and personal agendas. This is why product folks like to observe customers in their typical environment using a product to avoid misallocating resources on agenda-driven features. At a systems level, reconsidering individual monetary incentives for salespeople is helpful, too. In a learning organization, teams win, not individuals.
- What features do you need this year? To be fair, some product managers behave similarly to sales when they ask internal customers what features are required. A call to send in your requirements typically ends up with an epic, Christmas-like wish-list of “requirements.” If the Scrum team’s resources are basically free of charge, there is an incentive to grab as much of those as possible by extending the list — whether those requests are valuable to the organization or not. You might also observe that stakeholders define requirements that they do not expect to be fulfilled just to gain leverage in future “negotiations” with the product delivery organization. This submission process is usually motivated on the stakeholder side by optimization efforts leading to a local maximum. (Note: Internal stakeholders acting this way behave perfectly rationally under such conditions.) Instead, the product delivery organization needs to maximize the outcome of the whole organization. This maximization effort requires entering a competition of ideas across all internal stakeholders to channel resources to the most valuable ideas.
Product Discovery Anti-Patterns: Ideas, Hypotheses, and Feedback
- #NoTransparency: There is no transparency on how the product delivery organization chooses ideas for the hypotheses shortlist and how those ideas make it from that shortlist to the experiment backlog. Establishing the principle that internal stakeholders have to enter a competition of ideas to avoid creating local maxima at the organization’s expense is in itself challenging. The prerequisite for a successful transition to that principle is transparency on the one hand and a competition watchdog on the other hand. Otherwise, stakeholders might opt out of the process as their efforts appear futile, and rumors might arise: Are HiPPOs driving the selection process? Or will those who shout the loudest be privileged when ideas make it into the backlog of experiments? Do I have to pitch my ideas at 2 a.m. in a bar to the Product Owner to be heard? Make sure that everyone understands how and why decisions are made. Visualizing both the hypotheses funnel as well as the backlog of experiments has proven to be supporting.
- No feedback process: There is no process on how the product delivery organization deals with suggestions for new features or other ideas, and feedback is provided randomly at best. It’s a good idea to have a process that actively involves stakeholders and organization members in the product discovery process. People like to have a purpose and to be a part of something larger than themselves. Providing everyone in the organization an opportunity to contribute without regard to rank makes working as a Product Owner easier. Such a process doesn’t require fancy technology—a simple shared spreadsheet or form is enough to get it started. Initially, a template to suggest new product features could consist of questions that address the why, the how, the what, and the for whom. It could handle the tactical or strategic nature of the suggestion, a possible time-frame, or an estimate of the expected return on investment. Most importantly, designing a process to handle product ideas should be kept agile: the process should start with a simple solution and then be improved once initial feedback from stakeholders on the process has been analyzed.
- Ignoring sales: The sales organization is not effectively included in the hypotheses creation process. However, beware of salespeople’s tendency to demand new features once the quarter is nearing to meet sales targets to secure bonuses. This attempt opens the discussion of bonuses in general; a learning organization should not provide individual financial incentives. The possibility of a moral hazard is otherwise too large: Using the organization’s resources for free to meet personally beneficial objectives is all too human. Do you remember the origins of the financial crisis from 2008?
- Ignoring customer support: The customer support organization is not adequately included in the hypotheses creation process. To my experience, cooperating closely with the customer care team is most beneficial for any product delivery organization. Failing to include them is one of the worst product discovery anti-patterns.
- Hypotheses selection without engineers: The product management does not include developers early in the process of identifying potential hypotheses to test. “Idea is worth nothing; execution is.” Usually, a product delivery organization is flooded with ideas from everywhere: customers, sales, internal stakeholders, HiPPOs, friends, and family—you name it. The most important job of the product people is hence bringing order and transparency to the chaos by establishing a process that turns the long list of ideas and requirements into a shortlist. The purpose of this shortlist is to run experiments that verify or falsify shortlisted ideas as only valuable, usable, and feasible ideas can become a part of the product backlog. To do so, it has proven useful that a triumvirate from product management (business), design, and engineering decides which of the items on the hypotheses shortlist will make it into the backlog of experiments. Excluding the developers from that decision process flaws the whole process from the beginning.
Product Discovery Anti-Patterns: Egos at Work
- HiPPO-ism: The product discovery process is at least partly driven by the beliefs of individuals of the higher management caste. This can be best described with the famous quote from Jim Barksdale, the then CEO of Netscape: “If we have data, let’s look at data. If all we have are opinions, let’s go with mine.” Think about it: how does the pay-grade, and hence the individual’s position in the social hierarchy of the organization, qualify that person to have the ‘right’ idea? The Macintosh was a brain-child of Steve Jobs, and he deserved credit for that. He also almost killed it prematurely by refusing to add expansion slots to the design. I do believe that the Macintosh II made the iPhone possible.
- ‘My budget’ syndrome: Stakeholders do not pitch for development resources but claim that they allocate “their” budget on feature requests as they see fit. That process leads to the creation of local optima at a silo or departmental level. The effect can be observed mainly in organizations that tie additional benefits to individuals. Instead, resources need to be allocated in the spirit of optimization for the whole organization. Note: ‘pet projects’ also fall in this category.
- Sunk cost fallacy: “We have already invested so much. Let’s finish it.” No matter how much has already been spent, past expenditure does not justify continued work on a product that provides little or no value.
- Bonus in limbo: We are nearing the end of a quarter. Bonus-relevant KPIs (key performance indicators) are at risk of not being met. The responsible entity demands product changes or extensions in the hope that those will spur additional sales. This behavior is comparable with the ‘what features do you need to close the deal’ anti-pattern, but it demanded in a more pressing fashion, typically four weeks before the end of a bonus period.
- Financial incentives to innovate: The organization incentivizes new ideas and suggestions monetarily. (Contributing to the long list of ideas and thus the hypotheses short-list should be intrinsically motived. Any possible personal gain might inflate the number of suggestions without adding value.
Conclusion
A holistic product discovery & delivery process is at the heart of any learning organization employing Scrum. Filling the product discovery void of Scrum or any other product delivery framework of your choice becomes, therefore, is a necessity. And there are plenty of agile frameworks and practices to choose from for that purpose. Watching out for the product discovery anti-patterns listed above will save your organization significant resources in doing so.
What product discovery anti-patterns am I missing? Please share your experience with us in the comments.
✋ Do Not Miss Out: Join the 9,000-plus Strong ‘Hands-on Agile’ Slack Team
I invite you to join the “Hands-on Agile” Slack team and enjoy the benefits of a fast-growing, vibrant community of agile practitioners from around the world.
If you like to join now all you have to do now is provide your credentials via this Google form, and I will sign you up. By the way, it’s free.
Product Discovery Anti-Patterns — Related Posts
28 Product Backlog and Refinement Anti-Patterns
A Forensic Product Backlog Analysis — Making Your Scrum Work
Hiring: 71 Scrum Product Owner Interview Questions to Avoid Agile Imposters