When using story points to estimate should we reestimate things when not finished?

I believe, that hint to that question is in the Scrum Guide:

(...) All incomplete Product Backlog Items are re-estimated and put back on the Product Backlog. The work done on them depreciates quickly and must be frequently re-estimated.

Story Points and Velocity is not something considered as part of Scrum and it hugely depends on the context and I would say that you should ask the team, try both approaches with them and choose this one that suits all of you better.



My personal opinion is that you should re-estimate, which means that estimate can go down, up, or stay the same - but should reflect the current situation, gathered knowledge and experience. I am not bothered about "lost story points", IMHO it does not matter.



Effort does not translate directly to time that you need to spend. 3SP PBI can take more time than 13SP one, not because it was more complex and vague, it may be simply just a time-consuming activity, but not complex at all.

Also, Velocity is something that I rather treat as a loosely defined metric. It is like an oil temperature or a fuel tank gauge in a car, based on them you assess if what you see is good enough to ride.

When using story points to estimate should we reestimate things when not finished?

If a team does not re-estimate, would there be a clear view of how much work is thought to remain?

Estimates purpose is to understand the work involved to address the problem statement.  Story Points are just a method of doing it.  

Let's take the technology out of this and pretend we are building a house.  The plan on Monday was to have all of the walls framed out with lumber by Friday at 5:00 pm so our estimate for doing so is 7 days. On Friday at 5:00 pm there are still 3 walls on the interior of the house that are not completed because it rained on Wednesday and put them behind. Those walls still need to be done and the next task of putting up sheet rock can not be done until all wall frames are in place. So going into next week would the estimate of work to put up the remaining walls be 7 days? No, that doesn't make any sense. 

Not consider that the scenario above was done using a timebox of 1 week to complete specific portions of work. You still want to finish the walls and start the sheet rock work and could possibly get it all done because there are plenty of walls that can be sheet rocked while the remaining 3 walls are framed.  

Estimate should be used to determine if the team feels the work is possible. Not something that they should judge their success/failure upon.  Let the team decide what works best for them and stop trying to make the estimates mean more than they should.  

Going to leave this here as my conclusion.  This is the 10th Principle listed with the Manifesto for Agile Software Development.

Simplicity--the art of maximizing the amount of work not done--is essential.

If you don't re-estimate partially done stories, your velocity will not be a true reflection of how much work you can actually get "done". Let me try to show what I mean with an (likely unrealistic) example.

In Sprint 1, I forecast 5 stories that are 8 points each, for a total of 40 points. 

At the end of sprint 1, none of the stories are "done", but I've roughly completed about 75% of the work on every story (about 6 out of the 8 points). I roll over all the stories and don't add any new work. I complete all stories in Sprint 2.

If I don't re-estimate, I'm claiming I've completed 40 points worth of work in Sprint 2. The reality is I've really only completed 10 points (the remaining 2 pts for each of the 5 stories)

With that being said, Velocity should be used as a guideline for planning. Both short and long term. Ultimately during Sprint Planning, it's up to the team to forecast how much work they think they can get "done" during the sprint, regardless of the story points associated with the work and their current Velocity. 

As per me , story point is one of the way to make a forecast of work during planning. When team is planning or refining the product backlog , whatever they know at that moment about a feature/story they estimate according to that. But this estimate can be different when the actual work begin, when more is known.

So, when the product backlog item is marked as not done let say in sprint 1 and cannot be split into Done and Undone items. Then what is the benefit of re-estimating this item ? Of course this item can be more clear and can be finished early or with less efforts. But its more important to use this experience in future estimations than re estimating this incomplete story which still needs some % of work to move it to Done. Situation seems to be similar to 90% syndrome link. What is the difference if team deliver this previously estimated 8 points story or later re estimated 2 points story? Does this re estimation increases transparency or accumulate some amount of wastage ? If it increases transparency then why not also re-estimate daily after work in a sprint is started on these stories ? If re-estimation adds wastage than value, then after the sprint ends we need to do it. 

I also think analogies or examples related to days and time somehow does not fit when compared with development work (which is complex). 

At the end its just my perspective. Choose a way which suits best for your team. 

IMO Harshal, you missed the key point of the estimation process. The number that the team produces, in the end, is meaningless, the process itself has the value. Story Points, planning poker or whatever is there only as a tool to facilitate discussion and encourage the people involved in sharing their view.

Our world is full of assumptions, but if you do not iterate on them as you go and gather more learnings, what is the point of making them in the first place?

If you actually did a PBI during the sprint, then re-estimating it afterward is pointless and only generates waste, just leave the original estimate (if you have one), but if you fail to finish PBI during the sprint, you should assess it in some point of time, i.e during the Sprint Review. Re-estimation will simply help you with this assessment. As you wrote, the item can be more clear now, but this doesn't imply that estimation can't go up.

What I am arguing here - just once in a while stop, look back on the bigger picture, assess things that you did or not, based on new learnings. If you estimate, then the number in the end just represents your shared understanding of the current state of an item. If you just leave it as it is, by embracing approach to not re-estimate, you unknowingly embrace people to not communicate - because why? We talk, assess and feel that this 3SP is truly 21SP one, but you know, we do not re-estimate here. IMHO it is against transparency.

What I am arguing here - just once in a while stop, look back on the bigger picture, assess things that you did or not, based on new learnings. If you estimate, then the number in the end just represents your shared understanding of the current state of an item. If you just leave it as it is, by embracing approach to not re-estimate, you unknowingly embrace people to not communicate - because why? We talk, assess and feel that this 3SP is truly 21SP one, but you know, we do not re-estimate here. IMHO it is against transparency.

Let say team regularly inspect the work needed to make an item Done and also the progress towards the goal. If an item is not done then team finds out the remaining work or the more work which is needed. They discuss, take learning and have shared understanding of what requires to complete it. But to achieve this do the team need to re-estimate ? Is it the estimates which is driving the above or vice versa ? Is there is a lack of transparency ?

I agree that SP is an arbitrary unit of its own which determines team common understanding and which invokes the discussions on the size of an item. Which is indeed the bigger picture than the number itself. But once agreed and at the end if not completed is there a need to again agree on its size ? or just inspect whats more needed to make it done. Of course take learning for the next iteration for such PBI and make better estimations for future. For me re estimating makes more sense when we can split the items into Done and undone where we re estimate for this undone part. But when cant split this PBI then this whole item with whatever amount of previous work done still has to be passed through the DOD. For me it would be productive to just act upon this item than discussing the size of it. ;-) 

Question - If we re-estimate a story which is deemed to be spilled over to the new sprint (important story, needed to be closed as per business requirements), how do we ascertain the work that was done in the previous sprint?

e.g. - A story worth 5 SP was picked up in a sprint, but could not be completed in during the time box. Now, whilst planning, we re-estimate the story for the remaining work to 2 points, close the sprint, and all undone stories spill over to the next sprint.

Where is the work accounted for the 3 story points, as that will not show up in the sprint burndown/synopsis for the sprint I just closed? Hence, effectively, those 3 points are lost.

Where is the work accounted for the 3 story points, as that will not show up in the sprint burndown/synopsis for the sprint I just closed? Hence, effectively, those 3 points are lost.

The work is accounted for by the state of the product at the end of the Sprint. It is accounted for in the Increment.  As for the points being lost, by the end of the Sprint the points do not matter.  Points are for estimating they should not equate to actual effort. They shouldn't be measured as success or failure.  

I'm going to leave this here for future.   This is a blog post by Rod Jefferies, the man said to have invented Story Points.  It is a good read and in my opinion pertinent to this topic. 

https://ronjeffries.com/articles/019-01ff/story-points/Index.html

why not also re-estimate daily after work in a sprint is started on these stories ? If re-estimation adds wastage than value, then after the sprint ends we need to do it. 

What is very important to keep in mind regarding story points is that their only legitimate function is to help with planning.   They should never be used as a proxy metric for effort or productivity.

It is a sound practice to gather story points at the end of a sprint for completed items only.   Trying to "get credit" for the completed portion of a story, and equating a SP value to it, is a wasteful and detrimental practice for a Scrum Team.   

To put this another way, it is disingenuous for a team to forecast 10 items totaling 40 SP for a sprint, finish around 75% of each item but not complete any of them in a sprint, and then declare that the data point for their velocity calculation should be 30SP (75% of 40).   Nothing was completed.   Their velocity should therefore be zero for the sprint.

And the unfinished items should be re-estimated based on the remaining effort and complexity to meet DoD, since story points only have value for planning purposes.   It is wasteful for a team to internally keep track of actual remaining effort/complexity while maintaining the original estimate (i.e. - the item says it is an 8-pointer, but really it's just a 2-pointer...).

In regards to re-estimating completed items at the end of a sprint, I will propose an analogy (apologies to Harshal, who doesn't seem to be fond of them).   I can become the greatest archer that ever lived, if I were allowed to move the target to where I shot the arrow every time.   A Development Team will not be able to assess their estimation accuracy, and improve their estimation, if they are permitted to change estimates after the fact that were deemed wrong.

1) What is the baseline for 1 story point...for example 1 story point = 8 hours ?

2) After the user story demo to PO, if there are requirment gaps identified, should the PO create new US, refine and prioritize ? What is the right approach ?

3) After the user story demo to PO, if the PO is not satified with the demo because team did not meet the acceptance critieria, in this scenoria, i am thinking below options, i would like to request to share your views and suggestions on how to best handle.

Option 1: Development team to rework on the user story and redemo again to PO?. (move the user story to in-progress state from complete state in Jira).

Option 2: Request PO to create new user story to implement the missed requirement and redemo again to PO.

Please suggest how to handle if the user story is partially completed, where the team noticed after giving the user demo to PO.

1) What ever your team decides to use.  I advise you not to equate it to hours because that has always been a recipe for disaster in my experience. It has been better for my teams to see it as relative to complexity of the work as they know it at the time of the estimates.

2) The right approach is what your teams feels works best.  But if the story is demonstrated during the Sprint, why create a new story? Why not just carry on with the one you have?  If you are meaning demonstrated at the Sprint Review, then your option might be valid.  But again, the Scrum Team should do what they feel is best and works better for them.

3) Same answer as #2

Don't get too caught in "doing what is right".  There is no right.  The team needs to decide how best to deal with these situations.  In my teams, they usually decide what to do each time this occurs and not create a "standard process" for it.  Inspect new information and adapt is the goal here as it is with everything agile.  

I've had multiple conversations on this topic since my original reply and think I've now moved towards not re-estimating. :) As your team matures and moves towards a predictable Velocity, there should be less and less carry over. All and all, I think everything will "work out in the wash". Whatever you do, spend some time to inspect and adapt, but don't get too caught up in numbers or spend time over-analyzing. 

I think giving points to "efforts" or to "work in progress of being value" should not happen..  With that said 70% of the work in  story means no value for any user. No DOD met, code is not usable. 

With that said I am into the approach of re-estimating spill-over stories. This will also help providing the correct visibility for "pending work estimations" In our backlog.

And would "kind of fit" what the scrum guide says about unfinished items being re-estimated when a sprint is cancelled.

I am just worried that at the end of the sprint (when they still have some time left) that they will not want to start any new stories, because they may think "it will not count" in the form of SP (even thou we know it will in the form of product value)

Nice discussion. I am still trying to figure it out... Sorry for my English I am Mexican. 

Have you guys noticed this part "All incomplete Product Backlog Items are re-estimated and put back on the Product Backlog. The work done on them depreciates quickly and must be frequently re-estimated" has not been included in the scrum guide 2020? Could we start considering re-estimating is considered waste? and stop doing it? 

Could we start considering re-estimating is considered waste? and stop doing it? 

If we did, what effect would that have on transparency?

Can you carry our the trash?  How often have you asked your kid or your wife has asked you that question?  When you answer you are giving an estimate.  You have to think about everything you are doing and decide if you can get it done in addition.  You have to decide whether to stop what you are doing and take out the trash.  

Do you think you can get that done next week?  Again, the answer calls for an estimate.

Do you think you think you can finish that next Sprint along with everything else that the team wants to pull in?  Estimate.

Estimates do not have to be formal numerical values or anything else that indicates some type of size.  The 2020 version of the Scrum Guide does not have the word "estimate" in it at all.  The section describing the Product Backlog states that all items will have some attributes and "size" is listed.  How you want to determine size is entirely up to the people doing the work. From the Guide (emphasis added by me)

Product Backlog items that can be Done by the Scrum Team within one Sprint are deemed ready for selection in a Sprint Planning event. They usually acquire this degree of transparency after refining activities. Product Backlog refinement is the act of breaking down and further defining Product Backlog items into smaller more precise items. This is an ongoing activity to add details, such as a description, order, and size. Attributes often vary with the domain of work.

The Developers who will be doing the work are responsible for the sizing. The Product Owner may influence the Developers by helping them understand and select trade-offs.

You asked

Could we start considering re-estimating is considered waste? and stop doing it? 

What does the team think about that and how would it impact the transparency of information to those outside of the team?