How to Measure Value with Evidence-Based Management
“How do we measure value?” is a common question that puzzles many Scrum Teams. There are an array of things they might measure, but how do they know which ones actually tell them if they're delivering value and moving in the right direction?
By measuring the value that a product delivers to customers and to the organization, the Evidence-Based Management (EBM) framework provides a way for teams and organizations to make better-informed decisions to better serve customer needs. As described in the EBM Guide, the framework focuses on achieving goals through validating experiments with evidence.
This activity is designed to help you understand how value is currently measured for your product. It’s also a good activity to improve your current approach to measuring value. It can be facilitated as a workshop for Scrum Teams working on a product or product portfolio together and may include internal stakeholders.
Activity Guidelines
1. What measures do you currently track?
(Suggested timebox: 3-4 minutes)
Have each participant individually answer the question, what do I measure currently in the product?
Attendees write down the metrics that they measure, listing them on individual stickies.
2. What business areas do those measures track?
(Suggested timebox: 5-7 minutes)
Once everyone has written down their measures, in small groups or pairs, they allocate them to the four Key Value Areas (KVAs) from the Evidence-Based Management framework.
The 4 Key Value Areas of EBM:
- Unrealized Value (UV): Measures that quantify the potential future value that could be realized if the organization met the needs of all potential customers or users
- Current Value (CV): Measures that quantify the value that the product delivers today
- Ability to Innovate (A2I): Measures that quantify the effectiveness of an organization in delivering new capabilities
- Time to Market (T2M): Measures that quantify how quickly the organization can deliver and learn from feedback they gather from experiments.
3. What areas do you currently focus on measuring?
(Suggested timebox: 10 minutes)
After the different measures have been allocated to the different KVAs, attendees can observe the results and discuss the following questions within their group or pair to help them visualize and understand what they currently measure:
- Are there many measures for any particular Key Value Area? Why?
- Are there few or no measures for any particular Key Value Area? Why?
4. Are your current measures helpful?
(Suggested timebox: 5-7 minutes)
Attendees should mark any measures they currently track that are NOT useful to them. Useful measures are ones that help them gauge their capability to deliver, learn, and pivot, in addition to understanding the state of their product and their progress toward goals.
This step is necessary to identify which measures are not critical and valuable to track.
5. What measures are you missing?
(Suggested timebox: 15+ minutes)
Attendees can now add measures that can help them better understand their capability to improve and deliver value. The EBM Guide provides Key Value Indicators (KVIs) that are suggested measures teams and organizations can use to measure for each of the four Key Value Areas.
Display the cards from the Key Value Indicators measures of each Key Value Area.* For in-person workshops or discussions, you can print and cut out these examples for use. In a virtual setting, consider posting them on a virtual board for easy access. Attendees may choose measures that are not specified in the KVIs.
As the small groups or pairs, go through the measures and discuss the following questions. These help you focus on customer outcomes and how to measure whether you are delivering on those outcomes.:
- For the Key Value Areas that you discovered were not sufficiently covered (in step 3), which measures would be useful to start measuring.
- Why? What would they tell you that you didn't know previously?
- What measures do you need to inform you about the value of your Increment and progress toward goals?
- How much Unrealized Value is there to pursue in satisfying your customer satisfaction gaps? (To determine your customer satisfaction gap, determine their current experience with your product versus their desired experience.)
6. How will we collect and use these measures?
(Suggested timebox: 10 minutes)
Capturing data is only helpful if it is actionable. Conclude this activity by asking attendees to decide on their steps. Have the small groups or pairs discuss the following:
- How would you collect the measures?
- How would you visualize the measures?
- How would they inform your current goals?
- How would these measures impact your decision-making?
You have created the initial version of measures as Key Value Indicators! Remember to use current data, revisit your measures frequently, and stop tracking unnecessary measures especially as your goals change.
*Courtesy of https://magdalenafirlit.com/key-value-metrics/
Resources:
