Backlog refinement and technical debt.

In Scrum the expectation is that a Definition of Done should be sufficiently robust for debt not to accrue. Furthermore, a team should allow for refactoring (and indeed any other tasks) when deciding how much work they can induct into a Sprint Backlog without compromising quality. Apart from that there's no prescription for how technical debt should be paid off once you've got it.

That said, it seems to me that your definition of technical debt is rather too broad to be useful. Technical debt is essentially the incurral of risk to product quality. It isn't "anything that the development team would like to do to facilitate their job, but doesn't specifically add value to to product". For example, the team might want a new IDE or plugin, but that isn't technical debt because it does not put the quality of the product at risk. It might reduce velocity but that's a separate concern.

What some teams do is to maintain a technical debt register. You can think of it as a RAID log which is under their own purview. It details the technical risks to the quality of product implementation in terms of probability, impact, and resolution. It may be possible to recommend mitigation by a certain sprint if the team have sight of a sufficiently well ordered Product Backlog. There may also be assumptions and dependencies in the register, and some risks can in fact be pressing issues.

A technical debt register can help inform the decision of how the debt should be reduced. Some items may need to be exposed to the Product Owner for consideration and some may not. Some may be handled along with the implementation of certain PBI's, whereas for others that might not be possible.

The use of a register like this certainly isn't part of Scrum, but nonetheless it can help a team get a grip on technical debt and on how the associated risks should be mitigated.

BTW, although I've compared a technical debt register to a RAID Log I'm not suggesting it should take the usual documentary form of one. In my experience it's better to use an information radiator. I coach people to use a separate Kanban board for this purpose. The states are To Do, In Progress, Done, and Escalate. Differently colored Post-Its represent the Risks, Assumptions, Issues, and Dependencies. Items are Done when they are either mitigated or accepted.

This also works for conventional RAID Logs at project or program level. Escalation from a Technical Debt Register would imply promotion to such a level. Managers are therefore encouraged to take an active interest in these registers, as well as impediments on a Scrum board, and to query any unescalated problems that appear to have stalled.

Like any other debt, a large accumulation of Technical Debt (bad code) can impact future Maintainability of the product. If team has made a conscious choice of adding bad code, then the items should be noted as soon as a decision to make a quick and dirty fix was made as opposed to doing it more elegantly and/or correctly. With the passage of time, teams forget about the mess they created. If a large amount of debt has accumulated, then team can propose doing a Technical Debt centric sprint as well after educating the Product owner and stakeholders about the importance doing so. Eventually, the Product Owner would need to make a conscious decision whether the economics justifies the cost of the technical debt.

Options I see:
# Special Technical Debt Sprint, once in a while. Items could be pick out of a Technical Debt Backlog.
# Refactor as part of User Story in current Sprint. This will help keep the focus on adding value and actual tecnical work will be decomposed as a Task Item and not be visible directly as PBI.

Interesting links:
http://martinfowler.com/bliki/TechnicalDebtQuadrant.html
http://www.industriallogic.com/blog/as-a-developer-is-not-a-user-story/

--Bhuvan.

> Special Technical Debt Sprint, once in a while.

Providing "special sprints" to clean up technical debt isn't really an option in Scrum. Instead, if technical debt is known to be building up, the Definition of Done should be revisited. It's essential to find out why the debt is being incurred and how this can be avoided.

Technical debt sprints, also referred to as hardening sprints, are essentially an antipattern. We had a thread on this topic last year: https://www.scrum.org/Forums/aft/307

Agreed, a holistic approach that includes Technical Debt within the DoD of the PBI’s picked up seems to be a more elegant and apt.

Hi Ian,

I think this depends a lot of whether you are working on a greenfield project or an old legacy system. I would say that any system that has been around for a long time has a lot of technical debt. Not because it was necessarily added deliberately but because the people who built it simply did not know any better at the time the system was originally built. Without discussing the definition of technical debt, I think that you will find yourself in situations where you need to clean up/refactor/rewrite stuff which does not add any direct value but will keep you from slowing down even further. Just make sure that you add something of business value every sprint.

That's right, a great deal of technical debt can be accumulated. In some cases it can exceed the value of the project itself....even by multiple times. In such cases the most pragmatic approach can indeed be to whittle down the debt gradually, sprint by sprint, while conspiring with the Product Owner to still deliver something of business value in order to pacify stakeholders.

Clearly it's not the ideal option, because very often it can indeed become a horse-trading conspiracy, in which the scale and ownership of the debt problem is not admitted to by the stakeholders concerned. They might be persuaded to turn a blind eye to the repayment of debt as long as they get *something* of immediate business value in return, but that isn't great for openness and transparency.

A better option is to fund a project exclusively for the elimination of debt, in which the PO is a suitable technical authority. Of course that means recognizing the scale of the issue, which is too politically uncomfortable for many organizations.

"...which is too politically uncomfortable for many organizations"

Incidentally, this is one of the reasons I value Enterprise Lean Startup for achieving agility at scale. One of the underlying principles is deciding whether to "pivot or persevere". For example, if technical debt is building, it may be advisable to "pivot" to a new architecture or technology stack. A new MVP would then be developed in order to test that hypothesis.

This approach can be implemented in Scrum...but as you can see, it requires unusually entrepreneurial product owners.