SCRUM Team Performance Measurement

I don't believe that these metrics are very meaningful.

1. Agile Maturity Metrics (To what extent team at individual level thinks that they known, they follow Agile concepts. There is big list for this - TDD, Vertical slicing, Definition of Done, Definition of Ready, Reviews etc etc.) - We do this quarterly.

Why does it matter what the team thinks they know? How do you account for people thinking that they know more than they do? Or people underestimating their knowledge, skills, and abilities? Both are very real problems. This doesn't seem like a useful or usable metric, and one that is very easily gamed.

2. Story Point Velocity - This tells how team is doing sprint to sprint. But I feel this doesn't tell whole story, what if we team spent more time on Bugs, SPIKES, other Non-Story tasks.. How do we measure this.

Story Points have very limited use. You cannot use them across teams as there is no consistent value. You also can't compare across large periods of time as teams may renormalize their Story Point values. The best use of Story Points is on a single team in a relatively short window as one factor in determining how much work to bring into a Sprint. I'm strongly opposed to Story Point estimates being brought outside of the context of a single team for any purpose.

3. Quality - We have given weightage to bugs based on their criticalness.

Who determines how critical a bug is? Is criticality the same as urgency or importance? Again, a vague and easily gamed metric that doesn't seem like it adds a lot of value.

Instead of these metrics, focus on the outcomes that the teams are enabling. How frequently do you deploy to either public facing test or production environments? How quickly do you respond to feedback? Are your users and customers satisfied with the product and support services provided by the organization? These are more important than trying to assess individual teams.

Maybe you can have a look at "Evidence Based Management" ?

https://www.scrum.org/resources/evidence-based-management

and the new https://www.scrum.org/open-assessments/agile-measurement-open

I along with my manager have been looking for ways to measure our team's performance.

Isn’t that a matter, and a measurement, best left to the teams themselves?

Isn’t that a matter, and a measurement, best left to the teams themselves?

 Hi Ian I read most of your articles and I like them. 

As I said in original question since last 18 months we are following Scrum. I serve two responsibilities Scrum Master and Development Team member. I have experienced how Scrum has changed the way we earlier worked, not drastically but I see lot of positives.

As it happens with every new thing, first few days/months everyone is so excited and over the time that excitement starts to decline. I dont want that to happen with our team.

In retrospectives these days frequently I hear "we are following process properly; There is nothing to improve now; We have matured enough;". 

I dont know how to approach this mentality. Please note my teammates saying such things out of ignorance not arrogance.

So I thought lets look for some genuine measurement metrics, put them before teammates and keep improving. Because I believe no matter how good you are at something there is always scope for improvement.

I hope I have not digressed. 

Thanks for your replies Olivier and Thomas. 

"There is nothing to improve now" by ignorance.

I remember my first Scrum experiment. It was exactly the same. Nothing to say in Retrospective because we where so accustomed to our dysfunctionments that we didn't see them.

Is it possible to be helped by an external coach ?

What about asking them to describe with lots of details how is working the very best teams in the world and what could be the very first step to copy them ?

@Oliver's suggestion of reading on Evidence Based Management is good one. It helped me get a much better understanding of how to discuss this kind of topic with our management.

A few things in your responses stand out for me.

I serve two responsibilities Scrum Master and Development Team member. 

From my experience, when someone tries to fill 2 roles neither of them get the attention needed.  And when I have a Scrum Master/Developer scenario it usually ends up with the person doing more development.  In that case the activities that a Scrum Master does to help the team remove impediments and improve their team dynamics are often sacrificed to writing code. Is it possible for you to step away from development and focus entirely on Scrum Master duties for a couple of months so that you have more time to use for the situation you are trying to solve?

In retrospectives these days frequently I hear "we are following process properly; There is nothing to improve now; We have matured enough;". 

"we are following process properly" shows a misunderstanding of Scrum. Scrum does not advocate any process. It does provide a framework within which to improve.  So if they are following a process it would be a process that the organization developed.  If they are following it properly, maybe that process should be inspected to see if it is the source of the problem.  If you are trying to introduce metrics like you described, then that process is against Scrum and agile in general.

Story Point Velocity - This tells how team is doing sprint to sprint. But I feel this doesn't tell whole story, what if we team spent more time on Bugs, SPIKES, other Non-Story tasks.. How do we measure this.

(I've mentioned this in other threads but it is my perspective and it isn't popular among the Scrum Masters where I work.  However I have coached it many times to teams and all of them have found it to be useful.) Why is there a difference between bugs and stories?  Don't both of them provide value to stakeholders/users and benefits the product as well? So I advocate that everything in the Product Backlog is a story.  Then all the PBIs can be handled in exactly the same way.  As for "SPIKES, other Non-Story tasks", Spikes should be time-boxed instead of estimated.  Not even sure what a "Non-Story task" would be. Can you give an example of that?

So over to you guys how do you monitor/measure your Scrum team's performance? How do you know you are improving?

I don't monitor/measure the team's performance. I let the team do that for themselves. I do track some data points for myself.  I track things like points taken into sprint, points finished in the sprint, points not finished in the sprint, number of items that are in the sprint backlog, number of sprint backlog items finished, number of sprint backlog items not finished, work in process on a daily basis, number of items added to the sprint backlog during the sprint, number of items removed from the sprint backlog during the sprint.  I calculate cycle time, lead time, throughput as best I can since Jira (our tool) does not make that easy. I have found it easier to do random samples for cycle time and lead time.  But throughput can be done using stories taken/done/not done. I do not share these with anyone outside of the team because the data is only relevant to that team and I never use any of that data to compare one team to another. I let the team interpret the data as they see fit. I make some of my own assumptions from it and will share my interpretations with team as part of my Scrum Master responsibilities. 

How do I know they are improving? By their abilities to deliver value and stakeholder satisfaction. Can they consistently deliver increments of value? Are they getting faster at it? Are the stakeholders happy with the cadence, quality and value that is being delivered?