Quality metric in SCRUM

That's right, there is just one Product Backlog. There is no separate defects backlog, because once work has been accepted as meeting the Definition of Done it cannot be subsequently considered to be undone.

Any defects should be triaged as new requirements by the Product Owner, and the DoD should be revised to satisfy this change in scope.

Arguably, a more useful metric is to track waste incurred. Some defects (or newly discovered requirements if you prefer) are very serious and must be addressed immediately in-sprint. As such they pre-empt anything on the Product Backlog or which has been forecast for delivery in the current Sprint Backlog. Unplanned work is a form of waste, and it is important to capture metrics for this, because it can divert team effort from achieving Sprint Goals.

> the DoD should be revised to satisfy this change in scope. 

I should point out that this is just one of the inspect & adapt opportunities. If releases occur quarterly as you suggest then this should be challenged, as more frequent releases would lower batch sizes, and potentially improve the team's response to defect discovery.

Hi Ian,

thanks for the explanation, still I'm not clear on the metrics of software quality in agile eco system

What you are saying to achieve highest quality during pre-production. in pre-production we have a goal or commitment of zero bugs.

The quality of the software that is measured only when the code is in production. some of the metrics would be bugs surfaced per year or something like that and customer satisfaction.

If someone wants to know how good the quality of product which was delivered, what is the metric to look for ?

all the inspect and adapt is happening during pre-production.
Is tracking metrics, inspect and adapt during post production a characteristic of agile?

> If someone wants to know how good the
> quality of product which was delivered, what
> is the metric to look for ? 

There are many to consider: ROI, customer churn rate, market share etc. The important thing is that metrics should be actionable - vanity metrics are to be avoided.

> all the inspect and adapt is happening during pre-production. 
> Is tracking metrics, inspect and adapt during
> post production a characteristic of agile? 

Yes it is, at least in agility at scale. The Scrum Guide doesn't have much to say about this but other documented approaches do, such as Lean Startup and Agility Path.

I believe there is no direct measure for quality. If the PO chose the wrong PBIs, but the dev team implemented them in a high quality, you will still have a low ROI etc.
Defect rate is slightly better, but the number alone doesn't say much.
You can complement this with metrics like test coverage, static code analysis etc.
You can also measure, how often an increment is really shippable, meaning there are no bugs left which prevent it from being shipped.
But in the end what counts is what the customers perceive as quality.

I agree with Ludwig. The metric that counts for measuring quality is customer perception about the product. For example, you can create less-user friendly product with zero defect. And that doesn't mean that the product is high-quality.

> The metric that counts for measuring quality
> is customer perception about the product. 

We have to be careful here, because customer perception might not be an actionable metric. It can be a vanity measure depending upon how questions are framed, and even then the value of the data can be questionable. For example Steve Jobs was adamant that it isn't the customers' job to know what they want, and Eric Ries has taken a similar position.

I think you are right, Ian, however Steve Jobs' quote is missing context here. As far as I know, he said the customer doesn't know what he wants before he sees it (and knows that's not what he wanted). He quoted Henry Ford: "If I had asked people, they would have told me they needed a faster horse". This effect is the reason for Apple not doing market research, and it's also the reason for building shippable increments in Scrum.
So, I understand "customer perception" as perceiving the product (increment), and that is important for success, however it is difficult to measure.

> I understand "customer perception" as
> perceiving the product (increment), and that is
> important for success, however it is difficult to measure.

It's certainly very difficult to measure, or even to define satisfactorily. However, we may have a scrummish word for what is needed. In Welsh rugby there's a concept known as "hwyl", which is what happens when the team and supporting crowd get together, find their flow and common cause, and pull in the same direction.

All impossible to measure of course. Yet the most successful Scrum Teams I've seen all exhibit hwyl, even if they aren't familiar with the word, and their customers are in on the fun.

Great discussion. Very helpful insight.  Thanks to all the participants.