Development KPIs and Scrum?
The management of our company is asking us to provide KPIs to evaluate the effectiveness of our Scrum team.
We provide the number of planned PBIs, the number of PBIs realized, the number of story points planned, the number of story points realized and the velocity.
I am not at all convinced of the value of these KPIs in order to evaluate our team. I believe more than value added to the product, customer satisfaction and return on investment are more important points.
In your opinion, which KPI should be used to measure the effectiveness of a team.
P.S. Sorry for my english!
Bonjour Alain,
Two points to keep in mind:
1. A PBI can often be broken down into multiple smaller PBIs.
2. Story points, usually, are relative estimates provided by the team. What 1 story point represents changes from one team to another.
KPIs often act as incentives. I imagine you can see how placing an incentive on the number of PBIs or story points realized might affect the two processes above ;)
You're on the right track, I think, with suggesting that they use metrics based on the product's value. It's what I believe is most in the spirit of agile. Here are, however, the obstacles you may face:
1. It's sometimes harder to measure these KPIs than the former (number of PBIs, story points, etc.)
2. These KPIs measure the combined strength of the entire Scrum team, including the product owner. Management might be unhappy if they can't easily differentiate between the development team being good/bad at building software and the product owner being good/bad at prioritizing towards value delivery. Some people (myself included) believe it's good to evaluate the team as a whole, but some prefer the ability to praise/blame at a finer level of granularity.
Do you know what the KPIs are to be used for?
My organisation is doing the same.
I find it quite amusing that we have KPI 'demands' from top management such as, Story Point Completion, Backlog Preparation and Planning %, Velocity, and Test Coverage, but never has the Product Value or ROI been discussed openly with even the Product Owners (???).
What does this tell me? That a lot of the traditional customs are still deeply routed in management and our organisation's Agile adoption is far from the end.
What does the Product Owner think would make the best Key Performance Indicators for assessing value delivery?
This is actually an interesting topic and would be considering the following:
1. Sprint goal achieved
2. Bugs report/defect
3. Scope changes
4. Story points committed v completed
5. Priorities delivered considering sprint injections in compliance with client service level agreement
6. Rate of resolving dependencies
7. Retrospective (adapting to action items)
Any other opinion?
There is a pattern for measurements: Goal/Question/Metric. It seems like your leaders are starting with metric or question. I could be wrong of course.
Try to figure out what the very goal is without thinking of metrics. Then ask what questions do I need to answer to see I'm reaching the goal. Finally look into what you need to measure to answer the question.
Starting with metrics isn't a good idea.
Reading your question, it seems that the company may be going for that which is most easily measured, instead of going for that which, although being a little harder to measure, actually answers the question of how much value the Scrum Team is delivering.
Story Points, number of PBIs and unplanned work do not provide an answer for that... It's like trying to see if you're driving to the right destination by looking at the gas gauge... I'd take a look at the EBM framework here to get some ideas on how to structure a more solid data driven practice.
Also, Ian hit the nail in the head... If you are the Product Owner for that team, the final decision on how to measure effectiveness is ultimately yours. However, if you are not, it's still worth it to have a conversation with the Product Owner and maybe present this as a suggestion to consider :)
When I am met with this situation my go to statement is to ask management what problem they are trying to solve. Ask them if there has been any problems reported to them by the stakeholders related to the teams' ability to consistently deliver value. If not, then why do they need metrics and especially metrics that would be used to compare one team to another?