Scrum Masters, What Are You Measuring in 2025?

It is often said that, if you want to improve something then it has to be measured. Over the years as I worked with many different teams, one observation that stood out was often teams were not measuring what was supposed to be measured and what they were measuring was not worth improving, although everyone was focused on improving it. 

No points or cookies for guessing the measure I am talking about. Yes, it is one and only #Velocity. There were some other measures that were also tracked and measured but were not really helpful but none did more harm than #Velocity.

Watermelon/Cobra Effect

When the right measures are not defined, often the metrics tend to create a watermelon or cobra effect i.e. they lead to behaviours which are counterproductive. 

Here are some examples of the watermelon or cobra effect.

1. Velocity: In a company that I was working for, a SOW was established that defined that the Scrum Team will deliver 60 points worth of work every sprint. 
   Effect: The Scrum Team started filling up the backlog with unrelated work items just to hit the mark of 60 points of work. It was also observed that the estimates were at times inflated.
2. Code Coverage > 85% : One of the teams had this coding metric as part of their DoD. To ensure that DoD is met; unit tests were written which would do nothing but simply return true.
3. Stories Planned vs Stories Completed: Another typical vanity metric that was easily gamed. If the team is unable to deliver a particular story due to some blocker or dependency, it would simply add a new story to the backlog to avoid the variation in planned vs delivered.
4. Sprint Pass vs Fail: This one was an extreme metric. Organization also tied the team incentive(sprint bonus) with the Sprint success and failure. The teams never had a failed sprint. 🙂
5. Improvements: In another team, the guideline was set that there has to be at least 1 improvement every Sprint. And the list of improvements looked like-
   1. Improved the accuracy of estimates.
   2. Improved collaboration by creating a whatsapp group of team members.
   3. Improving the onsite/offshore connect by creating overlapping work-shifts.

As a Scrum Master, one of the key accountability to perform is to improve the overall effectiveness of the Scrum Team. If the Scrum Team has wrong metrics of measurement, often, there will be no improvement and the Scrum Team will end up doing mechanical Scrum with zero focus on being effective.   

So, Scrum Masters - Are you still measuring Velocity in 2025?

If the answer is yes, then at least use it for the right reasons i.e. understand and help others understand that velocity is not a productivity metric, at the very best it is a forecasting metric and it should be only used for that sole purpose only; nothing beyond it.

If you are not measuring it anymore, let me know in the comments, what are you measuring?

Also, what about other metrics that can be easily gamed or turn out to be vanity metrics with no real value in making the Scrum Team effective. How are those metrics dealt with? Do you still have them or moved on?

What you should measure in 2025.

The answer starts much before determining the metrics to measure. The first and foremost thing that we might have to determine is - WHAT is the GOAL we want to achieve through the measurement? And what is it that we WANT to MEASURE? Do we plan to measure the Activities, Output, Outcomes or Impact. Hint: more than activities or output start focusing on Outcomes and Impacts.

Outcome - The result or effect of action, situation or an event.

The first step is to define the OUTCOME clearly. The outcome that needs to be measured.

For example: Consider, we are a Social Media platform and after the recent release we want to see a surge in our active users. So, the Outcome we are after is “Improve the User Engagement”. 

The relevant metrics that might help to track the improved user engagement include:

1. Customer Usage Index - Are customers finding the new features or improvements useful?
2. Hit Ratio - How often the users are accessing the newly released functionality.
3. Active Time Spent on newly released features/functionality.

Impact - The force or influence on something.  

Similar to outcomes, the IMPACT that needs to be measured should be defined with utmost clarity. For example: For the same Social Media platform, where we improved the User Engagement, we might want to see the Impact in the form of “Revenue Growth”.

The relevant metrics that might help to track the increase in revenue may include:

1. Product Cost Ratio - Total Costs compared to Revenue
2. New Signups vs Renewals vs Unsubscribe Ratio - How many new users joined, or existing users renewed the subscription compared to users who left post the new release.

Conclusion: 

As Dr. John Little said, “We are in the measurement business and not in the forecasting business,”so it is imperative that to achieve desired results we will have to measure the right stuff and focus on improving it. Defining clear outcomes and impacts and establishing metrics to measure them will propel things in the right direction. Evidence Based Management could be a good starting point to help identify and define the metrics that suit your context.