We have now launched a new Scrum.org course – Professional Agile Leadership – Evidence-Based Management (EBM), and it is an exceptional occasion to share my experience with implementing the framework.
I have been actively working with the Evidence-Based Management (EBM) framework since 2015. Attempts have been incorporated in different environments, including large corporations, medium-sized companies, and start-ups. In this article, I am sharing my best knowledge, practice and… struggles.
If you are looking for an explanation of what the EBM is, please take a look first at the EBM Guide. You may also want to review some additional materials like my article and video devoted to this topic.
The reason to use EBM in complex environments is to make decisions based on facts. The implementation can be successful regardless of the industry or business domain. The most important thing is to rethink empiricism. This mindset is crucial to implement the EBM in the organization successfully.
However, the sentences above might seem quite understandable and straightforward, the application of the framework may cause some perturbations.
Challenges
Some common challenges might be met while implementing EBM. They may occur not only alone but in pairs or even scores of them in parallel.
Not understanding the power of the framework in general I have met this challenge quite often. It may happen especially in a large corporate environment where not all decision-makers, stakeholders, or other key employees were trained in the concept. Also, the situation might occur when there is a poor understanding of empiricism that is a driver of the framework. For example, in one large corporate environment, we could not create awareness of the framework for all decision-makers (lack of availability, lack of willingness). EBM was introduced to the entire organization, but only a few departments followed the framework and truly measured value.
The process of overcoming this challenge requires consequence, consistency and persistence. All necessary people should understand the reason (why) and know the expected benefits of the EBM implementation. By extension, consultants, Agile Coaches and Scrum Masters must invest their time in teaching key stakeholders, showing positive examples from other companies, working on the empirical mindset and help apply Key Value Areas (KVAs), measurable goals and right metrics.
Focusing on goals without measuring the organizational capabilities and market value Another pitfall might happen when the organization has a goal but does not measure the impact of the actions taken to reach this goal. Example: the medium size organization, the company set up a strategic goal (measurable and time-bound) as well as activities that were supposed to reach this goal. No one measured the progress, no one thought about inspection and adaptation, they just filled the plan with activities that had been scheduled for six months. The result - the goal was not reached and surprise occurred on that day.
A suggestion on how not to fall into this mode is to measure all four Key Value Areas (Current Value - CV, Unrealized Value - UV, Time-to-Market – T2M, Ability to Innovate – A2I) to understand the current situation with organizational capabilities and market value and make decisions accordingly.
Focusing only on one Key Value Area (KVA) (often Time-to-Market). This challenge is the most common one in my experience. Example: A large company struggled to apply EBM. There were some tensions between the IT department and the business. IT wanted to use EBM; however, the business was not supportive. Finally, IT implemented the framework and measured only Time-to-Market as other KVAs and measures were not available for them. Measuring only one KVA gives only a narrow perspective. It leads to a lack of understanding of the current value, unrealized value and organizational capabilities.
Based on this learning, some of these organizations attempted the second round of applying the EBM. This time with all interested parties.
Suggestion - Underlining that all KVAs matter to make better decisions is crucial. Having a strategic goal for IT and business helps overcome tensions to achieve greater value.
Focusing only on organizational goals (increasing revenue) without paying attention to customers; Also, low focus on value, outcomes in general There is another yet popular challenge I have been observing for years. Example: a large corporate organization has a strategic goal related to revenue. All Product Goals are reflected in the same manner. The majority of efforts that the company made did not get them closer to the goal. The organization did not connect the fact that fulfilling desired customer outcomes and satisfaction gaps might be strongly related to the revenue. Decision-makers were still stubborn and did not want to change the goal. Product Owners were helpless and not empowered.
Suggestion. Try to find people from top management that can be open enough to listen to some arguments. Without focusing on value and customers, having increased revenue is wishful thinking or a short-term strategy (i.e., as a result of the marketing campaign).
Being afraid of experimentation and no inspection & adaptation Applying the EBM framework, frequently measuring, having clear, measurable goals, indicates the need for experimentation. Around 70% of organizations are reluctant to run experiments (based on the own research, more about this topic in my article Why experiment?). Avoiding experiments and frequent inspection and adaptation, results in implementing unvalidated assumptions. Example: the company did the research on the new product line. They set up the meaningful goal, they applied the EBM concept with some measures. Initially, they measured the situation once per month. Then there were some changes in the top level. The new management team was reluctant to this process of frequent measuring and experimentation in addition. The goal remained the same. They were nowhere with reaching this goal in a year, the huge customer satisfaction gap existed, market share was shrinking.
Advice -in this situation, there is a need to reintroduce the concept, share previous success, explain what is meant by experimentation and why this is important. Share what will happen without frequent inspection and adaptation. Share about delivering outcomes and validation of hypotheses.
Lack of understanding Unrealized Value (UV) This KVA indicates if there are any opportunities to the market, any gaps in customer satisfaction, any desired outcomes that might be fulfilled. This area can be a source of organizational, strategic goals (if the EBM is measured on the company level) or a source of Product Goals (when the EBM is measured on the Product level). Very often, UV is misunderstood and therefore underestimated. There are many reasons why such a situation might happen. Usually, this is related to the lack of understanding customer needs, current expectations, market demand, etc. Not communicating with customers leads to false assumptions about their desired outcomes (we assume we know better), lack of experimentation and finally not delivering them value. Example: the organization started using the EBM. The Sales and Marketing department was reluctant to cooperate on measures related to the customers and did not agree to contact users directly. The products were measures in three KVAs regularly, but the strategic goal and UV were unknown.
Idea - Try to gain the interest of these departments that are accountable for customer relationships and have significant knowledge about customers. Share the importance of having these data. Also, top management involvement would be helpful.
Decision-makers are not involved There is difficulty and lots of challenges in implementing the framework when top management, key stakeholders, all important decision-makers are not involved (sometimes not even interested) in the decision-making process based on evidence. I frequently met this situation in the large organizations still working with a traditional, plan-based approach. It was challenging to gain their interest in empirical approaches and experiments while the company wanted a detailed plan for the next five years.
Suggestion - Try to meet decision-makers, explain, share examples, ideally from the same organization if there are any, show benefits and then involve. What would happen with the organization that follows detailed long-term plans instead of market changes?
In the second part of this article "Success of implementing EBM" you can explore some positive examples of implementing the EBM framework.