How to measure productivity of my Scrum Team?

In Scrum productivity is measured in terms of actual delivery, not story points. If an increment if value is produced in accordance with a Sprint Goal which satisfies the Product Owner, then success is being achieved.

Story points are for helping teams forecast how much work can be taken on in a sprint. The tracking of story point burndown is not to measure productivity, but rather to show how much work remains in a backlog.

Hi Ashurai,

Why is measuring team productivity is important to you? What problems are you trying to solve?

Hello Ashurai

Scrum is not about delivering more but delivering ONLY features which customer wants. I would recommend to use different metric to measure your team productivity -

1. Thumbs up from stakeholders to the new increment
2. Unit test coverage
3. Continuous Integration status
4. Team Happiness Index
5. How long does To-Do to Done take?
6. Customer Reported Defects
7. Feature Usage Index

Cheers
Sanjay

The last word on page 12 of the scrum guide is probably the most important one but is seldom followed. That word is "Value".


"The Product Backlog lists all features, functions, requirements, enhancements,and fixes that constitute the changes to be made to the product in future releases. Product Backlog items have the attributes of a description, order, estimate and VALUE"

I have almost never seen "VALUE" in a product backlog item. Story points are good indication from a project management perspective but from a business perspective it is the Value produced by each sprint that is probably the right indicator of productivity ...

Ashurai,

As Sanjay pointed out, although productivity is important, Scrum is not about delivering more.

I recommenced reading a little article, The Deadly Disease of the Focus Factor.

It might help you think about productivity in a way that better serves your team.
https://www.scrum.org/About/All-Articles/articleType/ArticleView/articl…

Best,
Daniel

Very good point Sandeep. Problem with defining value is that you have no idea of the value until your customers are using it. You can guess that feature A has twice that value as feature B, but until you have built, shipped and gathered feedback you're really guessing.

Also, the relative value is also kind defined by the order of items in the backlog. If two items are estimated the same, the one with higher value should normally be ordered first.

The "value" attribute of a Product Backlog Item is perhaps best seen in terms of risk reduction. Using MoSCoW priority as the attribute is one way to express value and it can certainly help to inform a Sprint Plan.

Trying to anticipate value is indeed very difficult...as Fredrik says you really have to ship something first. That's why it's important to keep batch sizes small and to release frequently. In an agile way of working a product should be its own market research tool.

Have come across the need to measure team's productivity primarily wrt code quality delivered by them when working with different teams (belonging to different vendors). Based on our experience, we found that measuring code quality attributes such as defects density, code complexity, code coverage etc vis-a-vis velocity throws some interesting light and patterns which could be helpful in measuring productivity to some extent. Thus, went ahead and put up a software quality metrics tool for everyone to try and see if it could be helpful. Do check http://agilesqm.appspot.com.

I have one question... as you do relative sizing, why is a similar story 3 points less.
By using story points you should compare stories and if they are similar they get the same points. Than you also may can see an effect of good or bad practices through your velocity. By sizing them differently you can not see this effect

BR,

Posted By Philipp Eisbacher on 23 May 2014 02:32 AM
I have one question... as you do relative sizing, why is a similar story 3 points less.
By using story points you should compare stories and if they are similar they get the same points. Than you also may can see an effect of good or bad practices through your velocity. By sizing them differently you can not see this effect

BR,

Size is an ambiguous term, and depending on the project can incorporate different things, such as number of units, complexity, technical risk, etc. I'm guessing it's possible that the relative size of a similar story diminishes over time because there may already be some reusable base code for it (less complex), or the technical risk drops due to familiarity with the Story goal.

Understanding that tracking story points is a direct reflection on how much work remains in a product backlog and that any type of story point metric shouldn't be used as a measuring stick outside the scrum team, suppose you have a development manager that attends plannings and reviews and points out that the majority of teams appear to have consistency in the their delivery of story points points per sprint but there are some teams that consistently "appear" to deliver lower story point totals.  They are looking at it from an "apples to apples" comparison. 

I've seen it noted many times here in many threads that normalization/calibration should and should not be done.  If normalization had been done here you would expect similar story point totals completed among the teams, no?  The obvious point being made by the product development manager is their concern over the comparison on story point totals completed per sprint seems to be less with some teams.  Should the explanation be that the PO and SM don't directly impact these story point totals and that this is not a cause for concern because story points don't drive performance?  As noted velocity is not a measurement of performance and the end result is measurement of value to the client.

I just want to make sure the correct explanation is provided.  Thanks ahead of time for your help.

If normalization had been done here you would expect similar story point totals completed among the teams, no?

No, because there are all sorts of other variables. They may or may not be observing the same Definition of Done, they may have different team sizes or vary in the type of commitment they are able to make, they could be operating under different conditions of uncertainty, they may have different skills or levels of cross-skilling, they may face different impediments and dependencies within the organization and so on.

A case for normalization can be made when teams are working on the same product and thus draw work from the same Product Backlog. However this is a matter for them rather than external stakeholders, and the above considerations will still apply. Parties outside those teams ought to be concerned about value release in the form of integrated increments.

Hi everyone,

This is an interesting topic. According to your explanation, I can understand that Story Point just shows the remaining work not productivity of a Sprint. The productivity is value which delivers to the customer. How can we measure this value? Because my team would like to have a specific index to know this Sprint we increase the productivity against the previous Sprint.

Thank you so much for your feedback.

Best Regards,

This is a helpful article on Agile and value-based metrics:



https://platinumedge.com/blog/be-careful-what-you-measure-10-metrics-agile-teams

And this is an excellent article on why you should never use story points in any metric:

https://age-of-product.com/faking-agile-metrics/

I'm on the other side of this discussion, a little bit. Scrum as a framework is also about improving your product, team, and work environment. Sometimes there are productivity impacts the team is consuming, so it would be good to find a measurement to capture what today looks like and then again in a couple weeks AFTER SOME CHANGE has been implemented.

I am ONLY talking about a measurement to help prove the change you're experimenting with is working well enough or if further change is needed. I am NOT suggesting this measurement validate the output of a team sprint over sprint. Scrum is about identifying process improvements rather than trying to squeeze more and more output from the team. It is more about the responsibility of exposing process failure points in current working processes and fixing them.

Hi John and Timothy,

Thank you so much for your feedback. It is useful for me. Highly appreciated.

Best Regards,

I feel there is contradiction with "measuring the customer feedback only" approach of measuring success vs "monitoring progress towards sprint goal".

How would be PO or anyone able to track the progress towards sprint goal without tracking and monitoring the "productivity" and make estimates?

May I know how would you quantify "Projected Capacity of the Development Team" and "Past Performance of the Development team" without even measuring productivity? 

Please help.

Measuring productivity can be done in many ways. It really depends on what is considered productive.  I have never seen anyone say that you shouldn't measure productivity.  All I have ever seen are individual objections to how that measurement is done.  I am one of the people that advocate using many methods in order to determine if the teams are being successful at providing the value expected from them while allowing them to manage those expectations. 

May I know how would you quantify "Projected Capacity of the Development Team" 

I would not quantify project capacity of the Development Team.  I would allow the Development Team to quantify that in a way that helps them to forecast the amount of work that they feel they can accomplish. That measurement is only beneficial to the Development Team.  If the Product Owner needs to understand the projected capacity, they should be asking the Development Team for the answer.  Remember that the Scrum Team is responsible for the delivery of value so they all play a part.  All roles in the team will depend on the others for the information that is needed.

and "Past Performance of the Development team" 

Past performance is based on the ability of the Development Team to accomplish the goals of the Sprints that they undertook.  It isn't how many hours they have worked or the number of stories they complete.  While both of those data elements can be data points none of them alone provide the answer. 

Forecasts, estimates, throughput, cycle time are all useful data points in helping to measure and understand productivity.  But the measurement of productivity will vary from team to team based on the way each team functions.  You will find a lot of "best practices" but I argue what you are actually finding are a lot of good ideas.  The best practice is the one that the individual team finds most useful for them.