The topic of Agile Metrics inevitably comes up in many situations and conversations. I have been hiring Scrum Masters lately. One of my screening questions read, “What standard metrics would you track if any and for what purpose?” I cannot tell you how many candidates mention Velocity, Burndown and Burnup charts. Very few can reasonably explaining the meaning and use for those.
So far, I hired 2 Scrum Masters whose answer to the question didn’t have any of those metrics. What these two have in common, they mentioned and could talk about Cycle Time. Mind you, that was not the only reason they got the job, but it gave them advantage over others. Rarely do you hear Scrum practitioners bringing up Cycle Time, Lead Time, Throughput, Work Item Age. These all firmly used to belong to the Kanban world. Somehow during the Holy Scrum Kanban decades of feud these metrics were banished from the Scrum land and forgotten by many.
Entering PSK
It changed last year when Scrum.org partnered with Daniel Vacanti and others and developed the Professional Scrum with Kanban (PSK) course. I was lucky attend Dan’s course in Miami earlier this week. We were joking that the course was too heavy on math and numbers. It wasn’t. However, what was refreshing is to hear Dan passionately talking about numbers. Those numbers that matter, the Agile Metrics. Dan is a CEO of ActionableAgile - the home of a great suite of tools for Kanban practitioners. Dan graciously granted fellow Professional Scrum Trainers permission to use the software for educational purposes.
In the remainder of this post I will focus on the key features of the software and highlight how Scrum (and Kanban) practitioners can benefit from using it in their daily (or maybe weekly) routine.
Revving Up Our Agile Metrics Engines
As you log in to ActionableAgile you see by a seemingly chaotic Cycle Time Scatterplot. The software uses example data to show its capabilities – nice touch. There is a reason the scatterplot looks chaotic. Because it is. For each date (on the X axis) it marks Cycle Time (Y axis) of all items that were completed that day. There is a nice and exhaustive set of controls for the plot. One can turn on and off Summary Statistics, color work items by various categories, show select percentiles and average (on this a bit later), display a selected percentile cycle time trend line. All these options turned on are shown in the below screenshot.
I am not going into each detail of all the possible analysis you can perform looking at this data. Trend line is quite useful if you know what to use it for.
Digging the Hole – It Is All About Predictability
The heart of this plot are the percentiles that are calculated automatically - these are the core of Agile Metrics. By default, the software calculates 50, 70, 85, and 95 percentiles. There is no option to calculate a user-entered percentile value, but I never needed that option anyway. Why percentiles are so important? It is because they enable you to have a meaningful conversation around answering the eternal question, “When will it be done?”
For example, the data loaded by the software shows that 85th percentile stands at 16 days. What it means, that with everything else equal, a usual right-sized worked item entering your system has a probability of 85% of being completed in less than 16 days. On the flip side, it also means that out of 100 items entering your system about 15 will take more than 16 days complete. Don’t you think these numbers are crucial for your Product Owner and stakeholders? I know I do.
Show Me the Money
There’s another interesting way to look at the data. Think of a cost to the organization to get a work item done. Let’s assume our teams are fairly stable and we know the full burdened cost of operating it. What it actually means, is that you can put a dollar number with a probability of getting an item under that cost. But be very careful, this is a very fertile ground for abusing the numbers. We all know how team velocity was used and abused despite many warnings from Agile practitioners.
There's a very good reason PSK makes its service level expectations (SLE) a team-level indicator and warns to think twice before sharing it with the outside world. That goes well against the grain of Kanban Method where these indicators are for the world to see. Moreover they are the basis for the conversations in Kanban meetings. But that story is for a different blog post.
If you have the right focus for the conversation though, it becomes very matter of fact for a lot of people. Think about it this way. According to the chart it takes almost 50% more of time (and so costs 50% more) to get to the 95th completion percentile. It takes 16 days to get to the 85th, and whopping 7 days more to add another 10 on top of it.
With those numbers in hand you can devise a strategy to attack waste. As we know from a very thorough Kanban Maturity Model, different levels of Kanban maturity address different types of wastes. It all starts with Muri, or “busy-ness”. As we mature and bring in more practices, we start addressing Mura, or “unevenness”. And only at higher levels, around the fit-for-purpose one, do we get to address those wastes that usually are mentioned by Lean practitioners. Those reside under the Muda category.
The problem with a lot of Kanban implementations (incorrect ones, of course) is that people are trying to tackle Muda wastes first, while letting people thrash it their busy daily life routines and not stabilizing the system first and bringing enough predictability to it. But I digress.
Scrum framework at its core has mechanisms that deal with both Muri and Mura. Enough to say that Mura is taken care of by the mantra of “short term stability vs. long-term flexibility.” Scrum Sprint cadence provides a good enough solution for eradicating Mura waste.
Going back to the discussion with the stakeholders, we can take a look at the Muda factors that consume 7 days to get to another 10% more of predictability and devise a plan how to attack those.
In the second part of this review I will talk about 2 other hugely important and impactful charts – Cumulative Flow Diagram and Aging Work in Progress. The third part will be devoted to the Aging Work in Progress Chart and I will finish this series with a look at the forecasting using Monte Carlo simulation.