code & test quality metrics in scrum

It is up to the development organization to create a Definition of Done. What is seen as quality is listed here.

Using metrics is A possibility of creating transparancy regarding the actual state of quality and could be input for discussion on both the DoD and how this impact future development. I fully agree with you that the most important measurement is the happiness of the customer and whether or not the product fulfills it's intent, BUT the customers view on quality on the back-end of the product is usually very limited. 



To give you a more concrete answer; are metrics regarding quality and test coverage 100% needed? No, but I do think you will miss out on a lot of insights and learning opportunities if you don't measure anything regarding these two factors. Not to mentioned the limitation on transparency.

If these metrics are intended to improve the code/test standards and technology then i would agree to have those. If they are intended to review the person behind coding and testing then i would not support them.

We started using Evidence based management with some KPIs or KVMs as production incidents , build failures , merge conflicts , release issues etc. We started to measure it in order to optimize our quality of code , coding standards , test best practices , code reviews , build release pipelines etc. What we do here is to make the outcome more transparent to allow the opportunity to apply corrective measures. Don't you think Scrum also focuses on continuous improvement ?

Code quality metrics can be very important as a means and measurement for improvement and reducing / preventing technical debt. So I like them very much. But they should never be used for the blame game. And given that any metric, stat or outcome can be manipulated or tricked, it should only be used for mere improvement

I do not believe the necessity of measurements, because we can give any numbers that we want. Moreover the most important measurement is to happiness of the customer. I really want to hear anyone's feedback.

What are your thoughts about flow metrics, such as those described in the Kanban Guide for Scrum Teams?

I support the use of measurements and metrics, especially when they are easy to obtain. They can help orient discussions around successes and highlight areas for improvement. However, it can be difficult to fall into the trap of driving work from the metrics.

I can think of two examples off the top of my head - test coverage and code complexity. Test coverage does not tell you how effective your tests are at checking the behavior of the system, just that lines are hit. You still need discipline to write good tests, but it can definitely highlight areas that aren't executed under test at all. Likewise, code complexity can help find areas of the code that may be confusing or difficult to work with, but they don't say if it's accidental or necessary complexity.

My advice is to come up with questions that need to be answered and then figure out how to get data that can help the team answer those questions.

I believe there is value in metrics, but as others have said, we don't want to use metrics as a weapon against the team (pointing blame). People can also consciously or unconsciously game the system, which moves their focus from creating a quality increment/product to making sure the metric report looks good. For this reason, tracking the same metrics forever, loses value over time. A better approach is to pick a metric and track it over a short period of time like 3-6 sprints. Analyze the data that the metrics provide, have a discussion with the team and use the information to make improvements. Use the findings to Inspect and Adapt, then pick another metric and do the same thing. You can always come back to a metric over time. Moving your focus from one metric to another gives you different insight into how you can improve. Approaching it with a focus on a new metric regularly, like a retro with Inspection and Adaption mentality, will provide a new perspective to improving and get away from using the same metric forever, which in my experience tends to become a negative motivator. 

First let me say that over 2/3 of my 33 years of software development have been in Quality Assurance. And I am always vocal on the use of metrics. I'm not a big fan of metrics of any kind if used as a measurement of quality. If they are used as a point of data that drives analysis and conversation, then they are good.  Everyone knows that data can be interpreted to mean anything so it has to be used wisely.

I have personally experienced a service code that had 100% code coverage but also had the highest defect rate of the approximately 30 services in uses by an application.  I have also seen the 100% test coverage code base that was constantly failing in production use. And one of my most recent experiences as QA on a set of code was that the code coverage was around 35% and test coverage was 60% but that set of code was the most stable in all of the application code base.  Why? Because the parts covered were the most likely paths to be encountered and thus provided us knowledge that in 80% of all uses, the code would work.  We actually used those results to review the code to determine if the portions not covered were even needed and reduced the complexity of the code by eliminating portions.

My personal opinion and one that I always try to point people towards is around customer satisfaction with the product. Most of these fall into the Product Owner's domain to deal with but not all. That can be measured in a lot of ways but the methods I have found most productive are:

• How many new features are being requested and how many of them are delivered (PO)
  • This shows the desire to use the application. If no one asks for new things, chances are it isn't really being used.  Or it has all of the features needed in for the purposes it is being used and has become a cash cow.
• How many compliments/complaints/recommendations it receives across the internet. (PO)
  • If people are talking about it then it is being found as useful or needed.  We have all heard the old saying "any news is good news".  If your product is being discussed, it means people are using it or want to use it.  Take the information from this and plan future work. 
• How often are you fixing problems in a set of code. (Dev)
  • Shows that what ever metrics you have in place are not being interpreted correctly.
• Do you feel confident pushing a set of changes to production without any further validation other than what is covered in your metrics? (Organization)
  • Shows that you are measuring the correct things for your team but don't get complacent because all of that can change at any time.

You can not improve what you can not measure. The idea of measurement is to help you identify possible areas of improvement. 

@Temitope Adeshina

I kind of disagree on your idea that you cannot improve what you cannot measure. Even if you have a high performing team that always deliver items on time, does not mean they cannot improve. Improvement is not relative to measurement IMO.

Thomas captured much of what I was thinking about saying. Measure, but be careful about how you use what you measure. There is a difference between an indicator and a metric. 

In terms of quality, there are behaviors worth measuring like code coverage for new code only, or that every story has at least one automated test case. Things of that sort but by no means is there a standard.

The DoD is an excellent starting place. Have the team identify for themselves what quality is and what measurement can THEY use to validate the way they prefer to work is connected with customer usability.

If they choose anything that isn't supported or needs some change in place so that they can achieve this preferred process, then the SM needs to help remove those impediments.

Sherwin, in my opinion it is inherent that some form of measurement is required under any improvement initiative.   Otherwise, how can you know whether your change resulted in an improvement?



You need to identify a baseline measurement for what you are attempting to improve, and then you need to be able to measure it post-change to see if it resulted in an improvement, yes?

You can't make better what you don't measure