Skip to main content

Evidence-based Estimation

May 13, 2015

An analogy I can think of is... I want my dart to hit the dart board, and not necessarily the bull's eye.... as it calls for a lot of details which apparently is missing during estimation. However, if my dart doesn’t hit anywhere on the dart board... it's almost like shooting in the dark; a very disappointing estimation scenario.
dartboard
A vast chunk of new teams struggle in arriving at a correct story / feature estimate generating considerable waste; with most them failing due to a huge variation (with estimated Vs actual)! Very few succeed in this journey by coming closer to the actual in the initial cycles itself.

Project managers in traditional development focus on detailed scope, so that the cost, estimate, and time are accurate and frozen before kick starting a project. How far this is successful and precise is debatable! In agile projects, during product backlog refinement the development team members arrives at a high-level estimate for each story in the product backlog normally in story points (Fibonacci series) which aids in finer estimation during sprint planning. This helps the team to get started, rather than wait for all the project details to get finalized.

Story point is an arbitrary measurement of a feature’s size relative to other features and not the time needed to complete a feature.

Whale-Species-Chart-copy

For example, by looking at the picture of the whales you cannot determine the exact age or weight of each whale but can compare the size of each with respect to each other. This is a very handy approach as detailed information to estimate the effort may not be available so early. Story points are used to calculate how many user stories a team can complete in a sprint which is termed velocity.

The story point size assumptions are interpreted using an estimation scale, the most common ones include numeric sizing (1 through 10), t-shirt sizes (XS, S, M, L, XL, XXL, XXXL), Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, 34,...), etc.

A team to get started with story point estimating must be clear, synchronous, and agree upon the below aspects (I call it the ‘Five subtle rules’ as they are hidden and work at the back of our mind.).

Common reference – Each team member estimating should have a same reference. For example, the size L to measure a story Evidence Based Estimationpoint size should be the same criterion for everyone in the team. Any conflict with respect to the reference index results in huge variation and wrong estimation. Agreeing upon a common reference is very critical.

Collective wisdom – Estimation is a team activity and hence all should participate in this. Having one person do this will be a huge risk as the margin of error will be high! However, a collective decision helps in arriving at a common estimate which in all probability will be accurate.

In addition, during relative estimation the team should also take into account the effort, complexity, and uncertainty for every story before arriving at a number.

Effort - How much effort a story would take to complete, relative to the reference story.

Complexity - How complex is the story with respect to the reference story.

Uncertainty – How much risk/unknowns a story holds relative to the reference story

Even with this common agreement, it may still take a few iterations for the team before arriving at a common estimation value. With estimation being definitely hard, how do we get the best estimates of story size?

Why not take a closer look at “Evidence-based estimation”, be empirical, learn by experience, become proficient with every calibration, and adopt during the initial project estimation phase to understand its true benefits!

Here is the approach a team should take to proceed with "Evidence-based Estimation":

Let’s consider a scrum team of 7 members commence a 2-week sprint of 10 working days. With 6 hours as the effort per day, the capacity would approximately be 7 * 10 * 6 = 420 hours.


  1. During the 1st sprint planning meeting, the team begins from scratch with no story points assigned to stories in the product backlog.

  2. A story is picked from the product backlog followed by task and time breakdown for each. This step continues till the estimated capacity adds to about 400 hours/5 stories (considering the average team capacity is 420 hours).

  3. Through the sprint, the team works towards completing all the planned stories. Through the journey they identify the uncertainty, complexity and effort required to finish the story.

  4. Just before the retrospective meeting, the team assembles to story point the completed stories (definitely now with some experience/evidence).

  5. Among the 5 stories, the team picks the one with medium effort, complexity, and uncertainty and assigns a story point (keeping the 5 subtle rules in mind). For example, when we take an estimation scale of 1, 2, 3, 5, 8, 13; this story is assigned a story point of 3 and it becomes the common reference story. A closer look at this exercise clearly indicates that the story point number is emerging out of working experience/evidence and not by mere guess work!

  6. The team continues story point estimation for all the stories in Sprint 1 (per Step 5) and assigns a value lower, higher, or the same when compared to the reference story point.

  7. As the team proceeds from one story to another, the comparative reference points become varied apart from being evident which helps them with better estimation. At this point if the team feels a need to refine the previous estimates, they can. The idea is to get better estimates with experience that are realistic.





  8.  
  9.  
  10.  
  11.  


The teams may decide to use this estimation during the initial few sprints till they are confident of the process. With experience, the team becomes better with lean approach of estimation and are equipped at deriving an almost accurate story point during product backlog refinement, thereby aiding in better agility and transparency.


What did you think about this post?