What is the purpose of estimating Product Backlog Items, according to the Scrum framework?

The Scrum Guide says that the projected capacity of the Development Team is one of the inputs to Sprint Planning. The estimation of work, along with a knowledge of past performance, is instrumental in making such projections.

In simple terms, the only value in estimation is to help a team to get its arms around how much work it thinks it can take on.

Bear in mind that Scrum makes no prescription about the method of estimation which is used. For example, a team might refine Product Backlog items to a more or less uniform size, and make projections based upon number of items completed.

The Scrum Guide says that the projected capacity of the Development Team is one of the inputs to Sprint Planning. The estimation of work, along with a knowledge of past performance, is instrumental in making such projections.

I disagree with this. The projected capacity is simply the amount of effort that the Development Team is capable of putting into the Sprint. Capacity can be reduced by holidays, vacations, mandated training programs, oand more. I have seen teams that can look at a calendar for the upcoming Sprint, identify time that team members will be unable to work on Sprint work, and then just look at a list of Product Backlog Items and consistently come up with work that can be done within the Sprint. No estimation involved. The Scrum Guide fully supports a Sprint Planning that does not utilize the estimates - it leaves the amount of work and how to choose that work fully up to the development team.

I don't disagree that some teams and organizations may find value in estimating. I have seen estimation add value to both teams and organizations. I just find it problematic that Scrum says "you shall estimate Product Backlog Items", yet does not ever say how to use those estimates. In the strictest definition, if I decide to not estimate Product Backlog Items because I do not use those estimates for Sprint Planning, I am no longer doing Scrum.

One of the Lean principles is to reduce waste, which may include extra processes and management activities. One of the Principles behind the Agile Manifesto is maximizing work not done. Spending time estimating is, for some teams and organizations, unnecessary. Yet in order to implement Scrum, one must do it and then may end up discarding the work since it's not required by any other activity, event, or artifact.

This is why I'm looking for explicit statements in the Scrum Guide. Either I have missed something, or maybe this is a problem with the Scrum Guide itself.

I have seen teams that can look at a calendar for the upcoming Sprint, identify time that team members will be unable to work on Sprint work, and then just look at a list of Product Backlog Items and consistently come up with work that can be done within the Sprint.

What information were the team considering, when looking at that list, to come up with such reliable projections? Did it allow them to plan a Sprint Backlog with which progress towards their Sprint Goal could be monitored?

What information were the team considering, when looking at that list, to come up with such reliable projections?

The team was looking at a number of things. The biggest things were planned company holidays and vacations that would eliminate one or more team member days from the Sprint. Other events were considered, too - interviews, company required training modules, office events. The focus was simply on the amount of working time. Although the team also strove to be cross-functional, everyone did recognize that different people had strengths - for example, if the better front-end developer was unavailable, it was recognized that front-end work would get done more slowly. Nearly everything was done by feeling.

Did it allow them to plan a Sprint Backlog with which progress towards their Sprint Goal could be monitored?

Yes. The Product Backlog Items brought in were tied to the Sprint Goal. As work was done, it was clear that progress was being made. More often than not, the Sprint Goals were met. Given how the Sprint Goals were established first and supporting items were brought into the Sprint second, and then the goals revisited, I'm not sure how this wouldn't have been the case, unless work not related to the goals were brought in.

What you have described is a situation where a team were successfully able to estimate their work. Whether this is done by experience and feeling, or by a more quantitative process such as planning poker is immaterial. They could cast runes or use a ouija board for all that anyone cares. The team should have a technique which works for them, such that they can reliably assess the work and determine that a certain selection will be doable in a Sprint. Scrum doesn’t make a prescription of any kind regarding how the estimation attribute ought to be implemented.

I see your point. However, that does make the wording of the Scrum Guide rather confusing:

Product Backlog items have the attributes of a description, order, estimate, and value.

and

Product Backlog refinement is the act of adding detail, estimates, and order to items in the Product Backlog.

and 

More precise estimates are made based on the greater clarity and increased detail

In my example experience, there is no attribute of an estimate associated with the Product Backlog Item. It's simple an item that contains sufficient detail that when a team looks at a collection of items, they can tell if the work is too little, too much, or just right for a Sprint based on past experience. The wording makes it seem like the expectation is that estimation happens in a more traditional sense, whether that's hours, points, t-shirt sizing, or something else.

The refinement process focuses on the requirements, edge cases, user experience, and so forth. There is no assignment of estimates to PBIs during refinement. There's also no refinement of efforts as more detail is added. The only estimate is made at Sprint Planning.

I think my confusion was around the attribute being something that was expressed on the PBI.

I do think there is room to improve the Scrum Guide in this respect. But I'm not entirely sure what it would be.

I think my confusion was around the attribute being something that was expressed on the PBI.

It is logically expressed on the PBI in so far as if different items are moved into or out of projected Sprint scope, then the total amount of work in the forecast may be expected to change. This does not imply that each “estimate” attribute has to be some sort of quantity which is arrived at discretely and independently. Estimates might only ever be assessed in aggregate. In some cases, such as when items are tightly coupled, that might be the only meaningful way to estimate a body of work at all. In other cases there may just be no advantage in bothering with anything more elaborate.

Great conversation!

I would only add that from a measurement/metrics perspective, providing estimates (in hours, points, t-shirt sizing, etc.) can provide the team with some visibility into whether they are improving their ability to complete work within a sprint.   Allowing a team to subjectively evaluate whether they can forecast sprint items or not doesn't provide any data to indicate whether the team is able to increase their forecast over time or not.

It is logically expressed on the PBI in so far as if different items are moved into or out of projected Sprint scope, then the total amount of work in the forecast may be expected to change. This does not imply that each “estimate” attribute has to be some sort of quantity which is arrived at discretely and independently. Estimates might only ever be assessed in aggregate. In some cases, such as when items are tightly coupled, that might be the only meaningful way to estimate a body of work at all. In other cases there may just be no advantage in bothering with anything more elaborate.

Hearing this makes perfect sense. I think I got stuck in the thought process that the estimate of a PBI is a discrete and independent quantity. I wonder how many people are also stuck in this kind of thinking and trying to estimate every PBI on their own when other methods may be more appropriate. I hope that a future revision of the Scrum Guide can work on the discussion of estimating PBIs to make this more clear.

I would only add that from a measurement/metrics perspective, providing estimates (in hours, points, t-shirt sizing, etc.) can provide the team with some visibility into whether they are improving their ability to complete work within a sprint.   Allowing a team to subjectively evaluate whether they can forecast sprint items or not doesn't provide any data to indicate whether the team is able to increase their forecast over time or not.

There is some level of risk in not providing some level of estimate on the PBI, I agree. I think it takes a really motivated team to continually push themselves and improve to new levels, or know when to back off because the workload is not sustainable. It also depends on the business environment that the team is working in. 

What an eloquent conversation!

I would merely like to add that I would struggle to see how the Product Owner could fulfill his or her role correctly without an estimate of the work involved.

Particularly:

The Product Owner is responsible for maximizing the value of the product resulting from work of the Development Team.

I would merely like to add that I would struggle to see how the Product Owner could fulfill his or her role correctly without an estimate of the work involved.

The Product Owner simply prioritizes the Product Backlog in terms of business value. The Development Team can identify any technical dependencies that are either necessary or preferable. Together, the Scrum Team (PO + Dev Team) can ensure that Product Backlog Items are in a logical order, considering both business value and technical dependencies. Various tools and techniques can help to ensure linkages between dependencies aren't broken. All of this happens during Product Backlog Refinement.

This is something that takes a (very) mature Scrum Team. Not necessarily in terms of the Scrum framework, but in conducting their roles as Product Owner and Development Team. The Product Owner needs to be comfortable understanding (at a high level) why there may be technical dependencies between PBIs and the Development Team needs to be very in tune with the product and business to make sure they are able to meet demands.

Personally, I find the idea of moving away from individual estimates on PBIs and focusing on decomposing them into the required work (which may or may not be represented by creating new PBIs, depending on if the work is independent, valuable in isolation, and testable) to be helpful. After we started focusing on decomposing the PBIs, the estimates lost value as most stories became in the range of 3-8 points. The options were to either recalibrate the points or try something new - the team chose something new (dropping individual PBI estimation). I've since left the team, but I wouldn't be surprised if they were able to move to a more continuous flow model. We also saw less discovered work in the Sprint - rarely did a Sprint Backlog Item uncover a significant amount of new work that fell into the criteria that needed to be tracked by a new backlog item (either in the Sprint because it blocked planned work or in the Product Backlog for future). That led to a whole lot more consistency in the two week iterations between plan and done.

the estimates lost value as most stories became in the range of 3-8 points

 

You can use any kind of gauge, even very straight forward like this one with only 3 levels : https://pbs.twimg.com/media/B7PbmFlCIAEHtun.png 

Isn't a part of determining value, as the Product Owner, knowing the 'size' of the task at hand?  If I was a Product Owner, I think that would be an integral component of determining the order of my backlog.  

I'd also consider it vital in determining the potential completion date of a goal or mission, as described by the Scrum Guide.

Isn't a part of determining value, as the Product Owner, knowing the 'size' of the task at hand?  If I was a Product Owner, I think that would be an integral component of determining the order of my backlog.  

I'd also consider it vital in determining the potential completion date of a goal or mission, as described by the Scrum Guide.

In my experience that is rarer than you might suppose, at least with more mature teams, where a Product Owner may value a Product Backlog which is ordered in terms of projected goals. During refinement a team might then estimate work in aggregate (e.g. that portion would probably be doable in one Sprint to achieve such-and-such an MVP).

Estimating items individually (e.g. by planning poker) is of course the more usual case and it can make budgeting easier, It also has the distinct advantage of causing teams to discuss and further refine the work being considered - arguably the real purpose of this technique.

Unfortunately once items have individual estimates, it can cause goals to become devalued and Sprint Planning to degenerate into more of an accouhting exercise. This antipattern is very common in less mature organizations. Hence the ambition to “deliver points”, instead of increments which meet goals, is so widespread and needs particular guarding against.

Unfortunately once items have individual estimates, it can cause goals to become devalued and Sprint Planning to degenerate into more of an accouhting exercise. This antipattern is very common in less mature organizations. Hence the ambition to “deliver points”, instead of increments which meet goals, is so widespread and needs particular guarding against.

After the team stopped using story points and giving PBIs individual estimates, the ability to consider goals rather than "our velocity was averaging X over the last few sprints, and we have about the same capacity, so we should bring in approximately X points" became the focus of planning. However, the goal setting was an iterative process. The Product Owner had certain goals in mind. Sometimes, there were technical dependencies that needed to be done before some of the goals. Othertimes, the goals were too lofty to fit into a Sprint and needed to be decomposed.

No matter what, the team settled on between one and three goals, each of which was represented by a subset of the PBIs that the team took into the Sprint.

Estimating items individually (e.g. by planning poker) is of course the more usual case and it can make budgeting easier, It also has the distinct advantage of causing teams to discuss and further refine the work being considered - arguably the real purpose of this technique.

I would agree that estimating PBIs individually is easier for budgeting. It also lets you have a rough idea of projecting completion and determining (assuming no changes to the Product Backlog), how long it would take to get through certain amounts of work. Sometimes, the person paying the bills would like to have this rough idea in mind, perhaps for their own planning and budgeting purposes. There are a lot of caveats with using the information this way, but it can be done.

I'm not convinced that estimating PBIs individually leads to further refinement than not. In my experiences, a mature team can appropriately decompose PBIs (either by refining the definition of the work or by actually decomposing into new PBIs) without assigning an estimate to each one individually.

Estimating individually reveals differences in interpretation of PBIs among team members.

Therefore it helps discovering refinement opportunities.

And it simply adds some information on its own that could help with reprioritizing, budgetting etc.