Backlog refinement and Sprint Planning. Should it be separate events?

The scrum guide states under Sprint Planning: "The Product Owner can help to clarify the selected Product Backlog items and make trade-offs. If the Development Team determines it has too much or too little work, it may renegotiate the selected Product Backlog items with the Product Owner. The Development Team may also invite other people to attend to provide technical or domain advice."; isn't this the objective of backlog refinement? i.e. to clarify the details of the story, define the acceptance criteria and gaining consensus on story points?

Do you think that the clarification of scope ever really stops?

If the whole team participates, can we not therefore use 1 event to plan for the upcoming sprint?

In Scrum, there only is one event for planning the upcoming Sprint, which is the Sprint Planning session. Product Backlog refinement is an ongoing activity which ensures that items are in a "ready" state for planning into future Sprints.

@Ian Mitchell,

You are right, clarification of scope is welcome in agile and I understand that refinement is an ongoing process, however, is it right/wrong to practice refinement in such a way that it ends up being a planning session?

Let me try to be a little more clear. For us, a new sprint starts every Wednesday and refinement sessions are held every Monday. What happens is that, during refinement, work is allotted to individuals within the team for the next sprint. Let us assume that as a result of this, for a 2 week sprint, we would have had two refinement sessions and enough work planned for the next sprint. Then, does this mean that we can eliminate the Sprint Planning session?

@Ian Mitchell

As much as I'd like to know if Sprint Planning can be eliminated based on the above explanation (not that I want to), when you suggested if we could have 1 event to plan, are you suggesting using a 4 hour planning session for a 2 week sprint?

My team would obviously not want that and I am sure they'd opt for the smaller refinement-planning session.

In your opinion, should the refinement session include the whole team?

What happens is that, during refinement, work is allotted to individuals within the team for the next sprint

Neither sprint planning nor backlog refinement should used to allocate work to individuals.  It's for the team to decide dynamically during the sprint how to progress the work.  This sort of pre-allocation can lead to excessive work in progress which in turn can lead to bottlenecks and unfinished work.

> is it right/wrong to practice refinement in such a way that it ends

> up being a planning session?

The purpose of refinement is to ensure that work is ready for Sprint Planning at some future point. That’s when it’s time for the team to plan what work will be brought into progress and how they will implement that work to meet a Sprint Goal. It would therefore be hard to see the practice you describe as being right.

> Let me try to be a little more clear. For us, a new sprint starts every Wednesday and

> refinement sessions are held every Monday. What happens is that, during

> refinement, work is allotted to individuals within the team for the next sprint.

That isn’t refinement, that’s a precocious attempt at planning. Deciding who does what within a particular team before Sprint Planning is not timely or appropriate, as it pre-empts the very teamwork members are expected to demonstrate, as well as the decision to bring that work into progress at all.

It is reasonable, during refinement, to anticipate what is likely to be done by another team if doing so helps to mitigate external dependencies and to order the Product Backlog efficiently.

> Let us assume that as a result of this, for a 2 week sprint, we would have had two

> refinement sessions and enough work planned for the next sprint. Then, does this

> mean that we can eliminate the Sprint Planning session?

No, because Sprint Planning is exactly the time when the team ought to start making decisions about how they will do the work. Refinement should be sufficient to enable the work to be sized and understood for planning purposes.

> In your opinion, should the refinement session include the whole team?

If refinement does not include the whole team, is the work likely to be understood sufficiently for it to be planned and actioned by that team in a collaborative manner?

I'll try to add a few things from my perspective (in addition to Ian and Steven's valid points).

• Backlog refinement and sprint planning are (and should always be) separate.
• In essence, refinement is trying to eliminate most risks/uncertainty when it comes to the business requirements set forth for the development team to cover. Because requirements will never be perfect, you need to have conversations on each particular user story in order to determine the effort required, value for busines, priority at a certain point in time (which can drastically change in just a couple of days/weeks), etc.
• Planning is an opportunity to select work to be included in the sprint that's just started. It consists of two parts: the what (namely what areas will be covered by the development team) and the how (basically describing the steps needed to create/implement the said areas from what). To the extent you have regular refinement sessions, your backlog should be in a pretty good state which means the PO (in fact the whole Scum team) could be already aware of what's likely going to be selected during planning for the next sprint. So while a tentative sprint goal may already be known before the sprint planning, it is only created during the planning meeting itself, following discussions within the Scrum team.
• There is no redundancy (in my view). You'd want to ensure the backlog (its top and middle to be more precise) is as accurate as possible (it will never be perfect - someone correct me if you've experienced a perfect backlog). You can't have a reasonably accurate backlog without constant refinement sessions.
• When it comes to participation in backlog refinement meetings, now things get tricky. Per the official Scrum guide, the accountability belongs to the development team as a whole, which means all (or most) activities should have the full participation of all those forming the development team. In reality however, you'll find cases where participation won't be complete.

I may be wrong, but it eems to me Scrum isn't fully/correctly understood (nor implemented) in your team.

during refinement, work is allotted to individuals within the team for the next sprint.

Steve, this should be recognized as a wasteful activity, and as Ian pointed out, a misguided attempt to insert planning into the refinement process.   

I classify it as wasteful, because there is an Agile concept called the Last Responsible Moment where decisions should be made.   Trying to determine who may work on specific items in advance of Sprint planning does not embrace this practice.   If "Joe" were allocated to work on an item a week in advance, but then falls ill around Sprint Planning time, the effort to identify and allocate "Joe" to the item is therefore 100% wasteful activity.

are you suggesting using a 4 hour planning session for a 2 week sprint?

The Scrum Guide time-boxes the ceremony at a maximum of 8 hours for a 4-week sprint; however, the length of Sprint Planning does not have to follow the same ratio for shorter sprints.   If you are able to complete Sprint Planning in a much shorter time frame, that is perfectly acceptable, provided you complete all of the Sprint Planning deliverables (Sprint Goal, Sprint Backlog, Kaizen (improvement item)).

In most scenarios, I tend to go for refinement meetings on a regular basis, where the development team comes together to dig into product backlog items explained by the product owner and also estimating them. If needed, additional experts are invited to clarify open topics.

I found two big benefits in doing so:

1. If most of the backlog items have been refined and estimated before the planning, the actual sprint planning can mainly focus on creating an actual plan for the sprint instead of discussing open questions and estimating backlog items. 
2. As the upper part only works if the backlog is always in good shape and order, the product owner is encouraged to do so, as the whole speed boost get's lost, if priorities tend to change to much between the last refinement before the sprint planning.

In the guide I read that the sprint planning should cover 3 topics and here is the third

Topic Three: How will the chosen work get done?

For each selected Product Backlog item, the Developers plan the work necessary to create an Increment that meets the Definition of Done. This is often done by decomposing Product Backlog items into smaller work items of one day or less. How this is done is at the sole discretion of the Developers. No one else tells them how to turn Product Backlog items into Increments of value.

That sounds like a refinement to me or am I wrong?

Sprint Planning is timeboxed to a maximum of eight hours for a one-month Sprint. For shorter Sprints, the event is usually shorter.

If PBIs are already refined and they just need to be selected why would it take so long?

In my experience, it doesn't always.  Refinement is done differently across teams.  Some will refine the story to the task level making sure that it is well defined what has to be done to satisfy the need.  Others will refine to a level that the general work is stated and define more detailed task level during Sprint Planning.  

I have seen that the ones that do the "general" level refinement often have an easier time completing the work. This is because the item may have been refined weeks before.  If the tasks are created Sprints before the actual work is pulled into a Sprint, there could be new information discovered that would modify the detailed tasks.  So waiting until the work is actually going to begin allows them to adapt easier. And even in the "general" approach, it is not uncommon to discover new information that would modify the understanding of how to implement the need. 

From the Scrum Guide

Scrum is a lightweight framework that helps people, teams and organizations generate value through adaptive solutions for complex problems.

Complex problems often, but not always, require a lot of adaption as the work is being done.  The suggested timeboxes in the Scrum Guide take into account the possibility that your work will be changing quickly and constantly. 

"Should" -no. "Can"-yes

What to do is entirely up the team. Experiment with different timing and settings until the optimal one is found

That sounds like a refinement to me or am I wrong?

Product Backlog refinement can be thought of as the art and science of de-risking Sprint Planning. When the Developers enter Sprint Planning, the Product Backlog items they choose to do are ready. They know that work can be Done that Sprint.

Sprint Planning is timeboxed to a maximum of eight hours for a one-month Sprint. For shorter Sprints, the event is usually shorter.

If PBIs are already refined and they just need to be selected why would it take so long?

Perhaps it's a life-critical system and the task planning required to meet the Definition of Done is exhaustive. That's one possibility.

It's possible it won't take that long. That's the great thing about a timebox: it defines the maximum allowed time. If people are finished before the timebox is up they can leave early. They don't have to stay until the last pip.

If PBIs are already refined and they just need to be selected why would it take so long?

What if they are not already refined?

What if what is most valuable has changed or has just been uncovered during Sprint Review? While the team may have been refining what they believe to be most valuable leading up to Sprint Planning, it is entirely possible that something more valuable is shared or discovered at Sprint Review. Product Backlog may have been adapted during Sprint Review, including new PBIs. Sprint Goal may require new PBIs or may require PBIs that are further down in order, and not refined yet.

Refinement can and may occur during Sprint Planning. There is even mention of it under Topic Two…

The Scrum Team may refine these items during this process, which increases understanding and confidence

Yes, backlog refining and sprint preparation should be separate events in Scrum.

Backlog refinement is a continual process that involves analyzing and updating the product backlog to ensure that it includes all that the team needs to work on. This action can be completed by the product owner and development team at any time during the sprint. Backlog refinement is done to ensure that everyone on the team understands what work has to be done in the following sprints.

Sprint planning, on the other hand, is a timed event that occurs at the start of each sprint. The entire development team works together to plan and commit to the work that will be completed during the next sprint. Sprint planning is going over the product backlog, deciding which items will be worked on during the sprint, dividing the work down into smaller tasks, and estimating the effort required to complete each task.

By separating backlog refinement and sprint planning, the team can guarantee that the product backlog is well-prepared before planning the next sprint. This allows the team to focus on the details of the forthcoming sprint during the planning session rather than discussing and refining the product backlog. Furthermore, separating these events allows the team to adjust the product backlog as needed throughout the sprint.

Refining can be done at any time, according to the Scrum guide, it is simply suggested, for obvious reasons, that it should be somewhat in the middle of the Sprint.

Sprint Planning on other hand is one of the five(including Sprint itself) required events in Scrum

If you are doing Scrum and not just some own project with elements of Scrum. So oy cant be "eliminated", as someone suggested.

Refining can be done at the Sprint planning when neccessary. It can be part of the discussion or not, any anyone in the team can suggest it. 

In my opinion, after building large products in a dynamic environment like e-commerce, I must say "yes," backlog refinement and sprint planning may happen during Sprint Planning event.

While it's acceptable for some refinement to happen during Sprint Planning, it's crucial to ensure that it doesn't overshadow the primary purpose of the planning session. Sprint Planning should primarily focus on selecting Sprint Goals and deciding which backlog items will be included in the upcoming sprint. Keeping refinement activities at a minimum during Sprint Planning helps maintain the efficiency and effectiveness of the session.

In our process, we conduct backlog refinement in dedicated sessions. However, between the last refinement session and the next sprint planning, new requests may arise that need to be incorporated into the sprint. Due to time constraints, we often refine these new stories during the sprint planning event. Additionally, while selecting stories for the sprint, we remain open to incorporating new insights or clarifications that may emerge during the discussion.

Once more, this approach must not detract from the intended focus of the sprint planning event, but for us it’s very useful.