You may have seen news about a study claiming that agile projects fail 268% more often than non-agile projects. Agile has its drawbacks, but this number seemed suspect. When I dug in, I found that the study’s methodology is questionable. It seems designed to arrive at a pre-determined conclusion. Yet despite its flaws, it points to a truth that Scrum Teams would do well to heed. Spending enough time to refine and plan their work each Sprint is crucial.
About the Study
The research was conducted with six hundred developers (250 in the UK and 350 in the US) over four days. The researcher provided one group of developers detailed specifications to work from. The other group was only given a high-level outcome to achieve. The lack of requirements served as a proxy for agile development practices. The "agile" developers struggled to deliver. The "non-agile" developers were more successful. This result supposedly proves that agile doesn't work.
The main problem: what the study defines as "agile" isn’t. The Agile Manifesto does not forbid creating specifications or planning before beginning work. I don't know of any agile framework that says developers should begin work with nothing more than a problem statement. This study doesn’t prove that agile fails more often. It proves that teams that don't plan before they start fail more often. For anyone who has worked on a successful Scrum Team, that's a no-brainer! Sprints that include refined work are more likely to succeed. Those that consist of a grab-bag of unrefined items tend to fail. Professional Scrum succeeds when teams make time for refinement and planning.
Product Backlog Refinement
Developers achieve their Sprint Goals more often when the Sprint Backlog contains work they’ve refined in advance. Yet pressure to limit meetings in favor of "real work" lead too many teams to curtail refinement. The perception that refinement isn’t "real work" creates two main anti-patterns.
Anti-pattern: Refinement done by others, not Developers
All or most requirements definition is done without input from Developers. Business stakeholders collaborate with analysts to decompose work into bite-sized chunks. That work is handed off to the team.
Why it’s a problem:
Developers are robbed of the opportunity to understand the work and its value. Without their input, requirements may be unrealistic. And even detailed specifications can be incomplete.
How to fix it:
Recognize that refining requirements is part of the work Developers must do. Involve Developers at every stage of refinement. At strategic levels, an entire team might not be necessary. But at least someone from the team should take part. As the work breaks down into smaller, more tactical work items, the whole team contributes.
Remember that part of the work of development is to understand the problem. Developers do more than peck at keyboards. They must understand stakeholders and engage in design thinking.
Anti-pattern: Refinement is too short
Teams set aside limited amounts of time for refinement. Refinement meetings may be ad-hoc or recurring, but less than a couple of hours per Sprint, total.
Why it’s a problem:
There’s a common illusion that group decision-making is straightforward. Each person contributes an idea, the group evaluates them, and selects the best one. For clear tasks, that might work. For complex problems, it’s messier. Teams need time to express divergent ideas and explore them in robust discussion. As they establish shared understanding, they can converge on a decision. Short-cutting refinement discussion strangles its value.
How to fix it:
As before, fixing it starts with recognizing that understanding the work is part of the work. Teams should give themselves space for robust refinement.
How much time should you allow? Earlier versions of the Scrum Guide suggested limiting it to up to ten percent of the team’s capacity. Even though that guidance was removed in 2020, it's still a useful rule-of-thumb. It may sound like a lot, but spending that time creates better understanding about the upcoming work. Better understanding leads to better performance.
It also depends on the team’s experience and cohesion. Teams that have long, stable membership may need less time than teams with new members. Likewise, if the team has deep product domain knowledge, refinement might be easier. One team I was on had been together on the same product for so long that they could refine as part of Sprint Planning. But when in doubt, err on the side of too much and dial it back if you discover you don’t need as much time.
Sprint Planning
Good Sprint Planning also mitigates the risk of failure. As with refinement, teams often sabotage their success by curtailing Sprint Planning. Like refinement, one reason is the perception that planning isn’t "real work." But Sprint Planning should be a collaborative, working meeting where teams think critically about the work they need to do.
Anti-pattern: Sprint Planning is Too Short
I see it time and again: the team allots only one hour for Sprint Planning. Sometimes, they stop even sooner, after selecting a handful of items into the Sprint.
Why it’s a problem:
Like refinement discussions, good Sprint Planning requires time for robust debate. Without enough time to consider divergent ideas, Sprint Planning becomes a mechanical exercise. Teams pull some number of Product Backlog Items into the Sprint up to the team’s perception of its capacity. There’s no time to discuss a flexible Sprint Goal, and there's no time to plan the work.
How to fix it:
Sprint Planning’s time box is eight hours for a one-month Sprint. You read that right: a full day of planning for the maximum Sprint duration. While it’s typically less time for shorter Sprints, don’t make it too much shorter! You’re unlikely to plan a two-week Sprint effectively in less than half a day. Give your team time to think carefully about the goal they’re committing to. Let them identify the work it will take to achieve that goal, and plan to get started on that work.
Anti-pattern: Selecting PBIs before setting a Sprint Goal
This anti-pattern often grows from the previous one. To save time, teams first select Product Backlog Items into the Sprint. Then they tie themselves in knots trying to write a coherent Sprint Goal to describe them. Sometimes the Sprint Goal becomes, "Do all the things."
Why it’s a problem:
If the Sprint Goal is a checklist of PBIs, there’s little or no flexibility to adapt to new information or the challenges that inevitably arise. The same is true when the Sprint Goal is wrapped around a set of work items. Without that flexibility, Scrum Teams struggle to deliver.
How to fix it:
Start Sprint Planning by discussing a Sprint Goal. The Product Owner suggests a direction. Then the whole team collaborates on articulating a goal. It should be a concrete step toward realizing the Product Goal. It should be outcome-oriented rather than a list of items to complete. It should explain why the Sprint is important.
With a Sprint Goal in mind, the team can select Product Backlog Items that will contribute to achieving it. If they realize the goal requires too large a scope of work, they can revise the Sprint Goal. The cycle repeats until they arrive at a good, achievable goal and reasonable scope. That scope is what we forecast we’ll do to achieve the Sprint Goal.
Anti-pattern: Not Creating an Actionable Plan
Even when teams create Sprint Goals and select appropriate scope, they’ll often skip the final stage of Sprint Planning. They end the event without breaking down the forecast into an actionable plan. Having the "why" and "what," they lack the "how."
Why it’s a problem:
Stopping at forecasting scope eliminates the conversation about to collaborate to complete it. That conversation can expose gaps in understanding. This part of Sprint Planning also provides a check on selecting too many items into the Sprint. Just as the previous stage can highlight too ambitious a Sprint Goal, this stage can reveal that the forecast is too large. If it is, the team can re-evaluate the scope.
How to fix it:
Take the time to discuss how the team will work on the selected items. What tasks are necessary? Who will do them? What technology do we need? Do we need input from someone outside the team? How will we get it? These are only a few of the questions worth asking.
You don’t need to plan every item in the Sprint Backlog, or plan activity for every day of the Sprint. A few days will do. New information will come to light as we work, and we can adapt the plan each day in the Daily Scrum. Taking the time to plan gives the team insight into the work they need to do. That insight helps when the plan must change.
Conclusion
Do agile projects fail more often than non-agile ones? I doubt it. Plenty of studies show that big-design-up-front processes fail to deliver value more often than not. Scrum's iterative, incremental approach mitigates risk and diminishes cost. It also allows teams to adapt to changes more fluidly.
Agility is not an excuse not to plan. Nor is it an excuse to start working without understanding the requirements. Scrum provides formal and informal practices to help with planning. Shortcuts that omit those practices will sabotage your ability to deliver.