How to estimate spike stories

Isn’t the spike you describe helping to bring work to “ready”, and part of Product Backlog refinement?

Ian

That's correct Ian...spike story is not delivering any business value in the current sprint , however it will help business delivery in the coming sprint.

Given that user stories are estimated on what is known at that time, complexity, relative stories in the past sprints. how is the analysis/POC/spike story estimated when there is no information on how complex this solution will be, we have not done similar POC in the past.

What factors are to be considered for extimation.

Should a Spike even be estimated? Isn't the purpose of a Spike to find all that is needed to eventually estimate the actual story for which the Spike was created?

Should a Spike even be estimated?

This is my point exactly. If team is spending time doing the analysis (could be a day or 5), that will have an impact on the team velocity (understandably will decrease from past sprints).

- Do we accept the decrease in velocity and notify (to management)

Or

- Make up a number to maintain the velocity (even though we are not delivering any business value as of now, however will help business delivery in the coming sprint(s))

The Scrum Guide states that Product Backlog Refinement activities "usually consumes no more than 10% of the capacity of the Development Team". Personally, I encourage teams to use this as a baseline when planning their Sprints. Depending on the length of the Sprint, it works out to about 0.5 to 2 days per person or about 4 to 16 hours per person, but there's nothing that says that everyone on the Development Team needs to be performing refinement activities or how to allocate that ~10% of capacity.

I consider "Spikes" to be refinement of existing work in the Product Backlog - either decomposing it into developable, testable, deliverable work or adding additional details to existing work such that the end result is one or more orderable, estimatable Product Backlog Items. Therefore, all of the things I mentioned above apply to Spikes.

I'd also point out one of the principles of Lean Software Development - "Decide as Late as Possible". When you have open questions or things to learn, delay the decision making process as long as possible, right up until the last responsible moment. If you make decisions too early, you need to rely on assumptions that may not actually be true when you go to do the work. That could make the decision wrong and the effort spent to fully understand the problem, evaluate and propose options, and choose one wasteful. By letting time pass and other decisions to be made first, you naturally change the set of options available to you and can make better informed decisions.

My coaching on Spikes is that aren't pointed. Instead I encourage that a time-box be set.  At any point during that time-box the team can decide that they done enough.  If the time-box is fully used then they revisit the purpose and decide how to move forward. 

I advocate that Spikes are activities undertaken to vet the technology that the team considers viable to implement the solution. As such it is part of refinement.  As @Thomas pointed out per the Scrum Guide it should take no more than 10% of the development team's time. Where I differ from @Thomas is that I don't interpret that to be 10% of capacity for a single Sprint.  If the team can deliver a potentially releasable increment during a Sprint and do 6 days of refinement (inclusive of Spikes and actual discussions on the PBIs) then I see no problem with them spending that time. It provides value in the long run. 

Where I differ from @Thomas is that I don't interpret that to be 10% of capacity for a single Sprint.  If the team can deliver a potentially releasable increment during a Sprint and do 6 days of refinement (inclusive of Spikes and actual discussions on the PBIs) then I see no problem with them spending that time. It provides value in the long run.

I don't interpret the 10% to be a maximum cap, either. I see it as a rule-of-thumb. If you're not spending something close to that amount of time on refinement and understanding upcoming work, you're probably not spending enough time on that type of work to be prepared for upcoming Sprint Planning. Of course, the specific time may vary per Sprint and you may choose to spend much, much more time on refinement than the suggested 10%.

I encourage my teams to estimate any kind of work (spikes, release support, supporting other teams etc.) they do in sprint if it helps in better planning and it is worth to do it. This may help to avoid overcommitting. At the end estimates are to help in planning and it is not a measure of the value that is being delivered. 

If you are using story point estimation. You can estimate in a similar way how you estimate user stories. Team will estimate based on what they know.

Thanks @Thomas for clarifying. I've began to think that I was a bit alone on that one.  In fact, I am alone at my current employer.

I encourage my teams to estimate any kind of work (spikes, release support, supporting other teams etc.) they do in sprint if it helps in better planning and it is worth to do it. This may help to avoid overcommitting. At the end estimates are to help in planning and it is not a measure of the value that is being delivered. 

If you are using story point estimation. You can estimate in a similar way how you estimate user stories. Team will estimate based on what they know.

If Spikes are time-boxed to a certain number of days or hours, don't you already have an estimate?  Why add a second layer of estimates with points?

In my opinion estimating the spikes may have below benefits.

1. It will go thru refinement process and help all the team members to get onboarded with the intent of the SPIKE. I see team members doing more critical thinking when they have to estimate. It also brings more transparency.  
2. It will help not to overcommit the work in sprint.  

Most of the times it can be guesstimate but done by team who know their work.  

Shouldn't Spikes be created as a result of refinement? And if so, shouldn't the entire team already know the intent and purpose of the Spike?  Spikes are for researching something to provide additional information and clarity to a Product Backlog Item which will be used for further refinement of said PBI. 

Time-box the Spike.  The Development Team should be able to determine if they are over/under committing even if they don't use story points as the measure. If you are relying entirely on story points to determine your commitment, you could see problems as you start to work on a new problem space because the work for that is not something for which they are familiar and your points will probably not be very representative of the actual work. 

Should a Spike even be estimated?

This is my point exactly. If team is spending time doing the analysis (could be a day or 5), that will have an impact on the team velocity (understandably will decrease from past sprints).

>>> A story marked as Spike indicates that it needs to be refined. Do you think estimating a spike will help in refining or just refining it to a level that it can be estimated will be helpful ? what is your current capacity spent on backlog refinement ? Would it help to increase it a bit to refine the spikes?

- Do we accept the decrease in velocity and notify (to management)

Or

- Make up a number to maintain the velocity (even though we are not delivering any business value as of now, however will help business delivery in the coming sprint(s))

>> How accurate you think your team's estimations are ?  Do you think velocity numbers in your organization are used for team's performance measurement ?

@SRIKANTH RANGA ROUTHU, In my opinion, Spikes should not be suffixed with the word "story" or "stories". Let's reflect on this... What is a story, specifically what is a User Story? What was the intended purpose of a User Story? We've grown accustomed to saying story for everything and I'll admit I was in the same place before.

My understanding is that a Spike is a research item which is time-boxed. The result of the Spike should enable a User Story to be more refined and to be in a Ready state. The use of electronic tools like JIRA perhaps forces the notion that anything and everything is a story (again we're omitting user from this).

Let's take an example. All I need to do is a simple singular task i.e. Upload data to the database. Isn't our normal approach in using these electronic tools to end up creating a story and writing something like "As a Developer, I want to upload data to the database, so that.....". Does this really make sense? Don't you think it adds unnecessary complexity and process? If we had the ability to just add this as a task, would you still call it a story?

Technically, estimating (in story points) a Spike makes no sense. Your mindset towards a Spike should be more on the lines of "Hey Team, I may need 10 hours to research how to do a particular task, Let's add this item to our Sprint backlog. Hopefully, this would allow me to get the answer to help us work on the Story #1234"

This is my point exactly. If team is spending time doing the analysis (could be a day or 5), that will have an impact on the team velocity (understandably will decrease from past sprints).

What is the real purpose of velocity? What would happen if velocity decreases?

Thank you all for your valuable feedback on this topic.

From the above, it makes sense to time-box the spike/analysis to help the team get to a position where enough information is learned to estimate the PBI and be taken up in the next sprint.

Or as stated by the Scrum Guide, spike/analysis could be conducted as Product Backlog Refinement activities (~10% of dev team)

@Steve Vb

Even though the company I work for has moved to "Agile" a while ago, some things are still micro managed by the management unfortunately.

ex:

- Every item that a team works in sprint (story, spike, prod support, etc....) has to be in Rally (ALM) and estimated

- Decrease in team's velocity should be explained in details 

These things will only change with proper education/knowledge and change in mindset I guess

These things will only change with proper education/knowledge and change in mindset I guess

>> I second your thought , think about conducting a training for management on Agile. May be they don't have clear understanding yet. 

Decrease in team's velocity should be explained in details

>> I see velocity as the assumed forecast made for a PBI which is not the actual efforts the team might need to make the PBI as Done. So, what benefits your management will get from the answers for the assumed efforts ( a non real value). But this again will be solved only if they are properly onboarded on Agile.

- Do we accept the decrease in velocity and notify (to management)

Or

- Make up a number to maintain the velocity

Is the goal to learn something through the time-boxed execution of a spike, or to artificially "game" the team's velocity metric to appease management?

Spike stories should not be estimated, if you estimate then spike will become user story which impacts on Burn-down & Velocity. ha you can decide defined time on spike to understand the story deep (Hours or 1 or 2 days) 

I agree with @Shiva. Spikes should not be estimated with story points, however some Agile management tools allow you to estimate and track hours on stories or tasks. This would be a better place to track the effort and time spent by the team on spikes. Velocity is a proxy to help you understand how much value the team delivers to the customer.

A quick note. Velocity is not an indication of value delivered or productivity. Story points or number of Product Backlog items completed only provide some data points for Sprint Planning and Release Planning. And if there isn't a 'Done' Increment being delivered, velocity is zero. Therefore if your Spike is being used to gather more information and not to deliver the Increment this Sprint, what is the point of estimating it as it should not be part of velocity.

Here is how the Scrum Glossary defines it:

Velocity: an optional, but often used, indication of the amount of Product Backlog turned into an Increment of product during a Sprint by a Scrum Team, tracked by the Development Team for use within the Scrum Team.

And here is a nice article from Gunther Verheyen: https://medium.com/@ullizee/velocity-in-scrum-actually-754cb58e50f6

If we are adding anything to the Product Backlog, does it not become a Product Backlog Item? As a Product Backlog Item, should it not qualify for the PBI attributes of description, order, estimate and value?

The Scrum Guide tells us this about the Product backlog…

“The Product Backlog lists all features, functions, requirements, enhancements, and fixes that constitute the changes to be made to the product in future releases. Product Backlog items have the attributes of a description, order, estimate, and value.”

It does not describe the types of product backlog items, but it does say all “features, functions, requirements, enhancements and fixes”.

As a Product Owner I would want to see an estimate of effort for PBIs in my Product Backlog (points if we are using them). I can use this for decision making and prioritization.

The question for me becomes: Should “spikes” be included as Product Backlog Items in the first place?

To borrow from Ian, if it is just part of bringing work to the “ready” part of backlog, couldn’t it just be part of the refinement process for the associated Product Backlog Item instead of its own PBI?

If the work warrants its own PBI, then I go back to my first statements supporting estimates.

Hi,

Spikes should not be estimated with user story points but with time and should be a time-box, this is a form of refinement. If we have points then we should also create tasks for normal refinement meetings in the sprint and give points as well.. right? Does not make sense. 

 

However, the questions we should ask is why sometimes teams wants to estimate with points?

Some wants to show the output and not the outcome for reporting to the company. 



To clarify, the output is all work done for the sprint but they do not understand that what should be shown as part of velocity is the outcome. The outcome is the working software delivered which is the major measurement of progress and give the most agile value. This is an agile principle.

 

I do not think that in most cases a stakeholder would give value the results of a spike in a sprint review which only helps to give more clarity to a future not done user story, right?  

Also are we celebrating 2 times? When doing the spike refining, in one sprint and executing the issue in a user story in another sprint? 

 

Another issue that I see is that the velocity chart will become over inflated, specially if spikes are over used. It will look very high and nice as if the delivery is high but it will not show the decrease in working software delivery because of using more time in spikes. So more difficult to have a transparent indicator to reflect in a Retro. 

 

Also sizing spikes in points is not possible. How can we do relative sizing and compare user story points with spikes? Is like comparing apples and oranges.. I would think that the points assigned to the spike are fake. 

 

Also my greatest concern is that the inflation of velocity based on fake numbers will lead to over committing in the future. 

 

In my team there is a tendency of some members to show output, and not outcome, so they want to size spikes, I think because of motivation to show off that they did the work. So I am working on explaining the risks of doing this sizing because of inflation and future planning and that spikes are not the the main measurement of progress but working software is. 



QUESTION

Anyway, if we do not size the spikes in “practical” terms, when planning, we would plan based on our velocity free of spikes. But if we plan a series of spikes how do you calculate, mathematically speaking, the decrease in velocity in points that will cause the execution of the time boxed spike?

 

Do you do some mathematical conversion from hours to points so we do a decrease in planned points as we would do when we have a member in the team which will take vacación that sprint? 

As already mentioned, spikes should be time-boxed.  This is usually no more than a day, which helps the Developers when selecting other PBIs during sprint planning, as they will know that they already have a days capacity allotted for the spike, as this will obviously affect their forecast. 

From The Professional Product Owner book: 

Every once in a while, usually in early Sprints, a Scrum Team will encounter a Product Backlog item that they cannot break down any further. This is typically because the team does not yet know enough about the technology or domain.

A spike is a research Product Backlog item whose goal is to learn more about what is needed to complete the requested functionality.

Spikes are typically very simple proof of concepts that are likely to be thrown away after they have served their purpose. The ultimate goal of a spike is to reduce risk through experimentation, allowing us to make better product decisions sooner.

Caution

The danger with spikes is that they may, like anything, become abused. Every Sprint ends up with a spike story, and before long teams start planning whole spike Sprints. As spikes do not produce any immediate value to the business, this is a slippery slope. Use them, but do not abuse them.

• Good reason for a spike: Download and install a candidate third-party API to see if it meets our performance needs. This would allow the Scrum Team to decide whether to attempt the feature or not.
• Poorer reason for a spike: Research a feature so that the Developers can feel more confident with their estimates. This is likely more about fear of failure than experimentation and does not necessarily help the Scrum Team make decisions.

A preferred approach is to both research and implement the Product Backlog item in the same Sprint. If both cannot be completed within the Sprint, then the whole Product Backlog item goes back on the Product Backlog, and the Developers complete it in a future Sprint. The Scrum Team is no worse off than if they had pre-emptively separated the story into different Sprints.

It is worth repeating that this focus on completing valuable quality increments every single Sprint is essential to the agility of Scrum Teams and the business as a whole. Each Sprint results in an Increment, which from the stakeholders’ point of view is “Done.”

I agree Thomas Owens