Sprint Review and UAT...

Hello,

I notice that many teams are experiencing similar problems.
The problem is that your Definition of Done is not complete. You still have remaining work left (UAT). So by the end of the Sprint actually you don't have a useable increment of Product.
You should encourage the Scrum Team to do their best to include UAT into the Sprint, that's the Goal.

Posted By Illya Pavlichenko on 19 Nov 2013 08:04 AM
Hello,

I notice that many teams are experiencing similar problems.
The problem is that your Definition of Done is not complete. You still have remaining work left (UAT). So by the end of the Sprint actually you don't have a useable increment of Product.
You should encourage the Scrum Team to do their best to include UAT into the Sprint, that's the Goal.

Hi Illya,

Thanks for the quick response.

In my opinion the stories that the team delivers are potential shippable. The “problem” is that when we push our deliverables into “Acceptance” phase, new stories arises that (most of the time) has higher priority than the stories in the current Sprint.
As a result, the PO wants the Development Team to work on the new stories instead of the ones in the current Sprint.

I’m not sure if it's a matter of completing the teams DoD.

The dev team can always refuse to insert the new stories mid-sprint. They own the sprint backlog.

What Illya is suggesting is to incorporate the Acceptance phase into the sprint. The PO could be inspecting the stories before the sprint review so that s/he has the new stories before sprint planning not after.

This is a requirements problem. The PO needs to understand that even if he trades these more important stories for other stories in the sprint backlog (and the team agrees to do this) there is an overhead cost in that planning has already occurred for the stories s/he's taking out, and planning needs to be done ad hoc for the stories being inserted.

The stories are potentially shippable by the end of the Sprint if you can deploy them production. There should be no remaining work left.
Does your PO invite all key stakeholders including business users to the Review meeting? Maybe that's the case if he/she does not. The output of the Review should be an updated Product Backlog.

Posted By Robert du Toit on 19 Nov 2013 08:29 AM
The dev team can always refuse to insert the new stories mid-sprint. They own the sprint backlog.

The Development team can indeed refuse insertion of new stories in their Sprint, but the PO has the authority to cancel the entire Sprint. I want to make sure that it’s going to be a Development vs. PO thingie.

What Illya is suggesting is to incorporate the Acceptance phase into the sprint. The PO could be inspecting the stories before the sprint review so that s/he has the new stories before sprint planning not after.

This is actually my suggestion towards the team as well. The only thing we need to encounter is that these “sneak-preview” moments are just for looking. If bugs are found, fine it needs to be fixed, if new features arises due to these sneak-previews new stories needs to be created. This does make the ‘Acceptance Criteria’ even more important than ever! Plus the team needs to be “strong” enough to stand up against a dominant PO in our case. Especially if your team consists of junior developers.

This is a requirements problem. The PO needs to understand that even if he trades these more important stories for other stories in the sprint backlog (and the team agrees to do this) there is an overhead cost in that planning has already occurred for the stories s/he's taking out, and planning needs to be done ad hoc for the stories being inserted.

I often show this picture to our PO. Love it!
http://whatsthepont.files.wordpress.com/2013/09/20130903-233825.jpg

Posted By Illya Pavlichenko on 19 Nov 2013 08:30 AM
The stories are potentially shippable by the end of the Sprint if you can deploy them production. There should be no remaining work left.
Does your PO invite all key stakeholders including business users to the Review meeting? Maybe that's the case if he/she does not. The output of the Review should be an updated Product Backlog.

Yes, all key stakeholders are present during the Sprint Review.
When the stories are presented, the stakeholders usually “plays” with the implemented features. As a result, it’s either a Go or No go.
No goes refers to findings that are found during the review session. Usually when a story is presented that doesn’t match the Acceptance Criteria or fatal error occurs.
In our case, most stories gets the green light. The problem occurs when the stories are put into Acceptance phase, new stories occurs. These new stories usually have a higher priority than the stories in the current Sprint.

So how does the Review session differ from the Acceptance? Why doesn't Review session generate a list of new stories/requirements/desirements?

> In my opinion the stories that the team delivers are potential shippable.

That's fine, because the Definition of Done should be robust enough to ensure completed stories are *potentially* shippable. However, it's the PO's opinion that matters as far as an *actual* release decision is concerned. You can therefore have different opinions about releasability without coming into conflict.

> The “problem” is that when we push our deliverables into “Acceptance”
> phase, new stories arises that (most of the time) has higher priority
> than the stories in the current Sprint.
> As a result, the PO wants the Development Team to work on the new
> stories instead of the ones in the current Sprint.

The Development Team wholly own their Sprint Backlog. The Product Owner can add any new stories to the Product Backlog and prioritize them accordingly (e.g. for the next Sprint), but he or she does not have the authority to change the Sprint Backlog without the team's say-so.

As others have said, User Acceptance needs to be brought in-sprint. However I think you need to be clear that it is a Product Ownership responsibility to do so, since:

- it is UA that is influencing the PO's decisions regarding prioritization and acceptability
- it is UA that is interfering with the PO's responsibility to ensure that "the Product Backlog is visible, transparent, and clear to all, and shows what the Scrum Team will work on next" as the Scrum Guide puts it.
- by treating UA as an equivalent stakeholder, product ownership has become the sort of committee that Scrum prohibits.

Posted By Illya Pavlichenko on 19 Nov 2013 09:06 AM
So how does the Review session differ from the Acceptance? Why doesn't Review session generate a list of new stories/requirements/desirements?

The feedback that I get back from the PO and key stakeholders is that they do see the benefits of a Sprint Review. However most stakeholders are business people who goes into the Sprint Review all blank and not able to test the stories on-the-fly. They need more time to absorb the stories and go through the stories at their own pace.

The Development Team wholly own their Sprint Backlog. The Product Owner can add any new stories to the Product Backlog and prioritize them accordingly (e.g. for the next Sprint), but he or she does not have the authority to change the Sprint Backlog without the team's say-so.

The team may be owners of the Sprint Backlog, but the Sprint itself is own by the PO. Our PO is a bit stubborn and persistent so if the new stories aren't added in the current Sprint, I foresee that he will terminate the current Sprint and create a new one. In the end, we all have a common goal which is deliver the most important increments first…
I'm afraid if we go to extreme, it will only destroy the relationship between PO and team.

As others have said, User Acceptance needs to be brought in-sprint....

I'm going to think about this option. Perhaps we can include the PO more during the Sprint. However there is still no guarantee that once it has been delivered, new (more important) stories will be created.

The PO needs to be coached to understand the impact of frequently changing directions. You (I'm assuming your'e the Scrum Master) could try the aircraft carrier analogy (tough to turn, but when going in the right direction can't be stopped). You could help him understand that any time he inserts a new story, time is wasted understanding that story and the time spent understanding the story that now won't be completed is waste.

To get user acceptance into the sprint, you could have a new status Ready for Acceptance. The PO would then be able to review the story as soon as it is ready and that could give enough time for him to come up with the new stories just in time for the new sprint. I'm a bit reluctant about that, though, because that means the team cannot complete the Definition of Done on their own.

I think a UA period after the sprint is fine, but the expectation has to be set that anything coming out of UA is addressed in the next sprint, not the current one. We have UA, and as a PO I queue new stories for the following sprint. I sometimes sneak in bugs but that is actually bad and the team or Scum Master should probably be stopping me. I wouldn't debate the point because scrum is very clear that the team owns the Sprint Backlog.

The conflict should not occur between the dev team and the PO. It is the Scrum Master's responsibility to ensure that scrum practices are being followed. S/he should be protecting the dev team from the PO in this situation so that if there is any conflict it is between the SM and the PO. While it would be drastic for the PO to cancel sprints in these circumstances, it wouldn't be very productive, and it sure as heck would be visible. You have management support for scrum?

I'm a bit reluctant about that, though, because that means the team cannot complete the Definition of Done on their own.

I guess technically the team can still complete DoD since the PO is a member of the team, just not the dev team.

One team I worked with included an 'empowered UAT' test advocate who was empowered by the business to conduct preliminary UAT on their behalf; the Definition of Done for these Sprints allowed for Done = 'Full UAT ready' by the tester. Formal UAT by actual business users was still conducted, but became part of the Definition of Done for a subsequent Sprint. The team was still releasing increments that were 'close enough' to potentially shippable every Sprint, and the actual UAT itself took about 80% less time and 50% less people to get 'Done' than the previous Waterfall delivery method -- one reason being, no defects were found. I should note that this arrangement suited a Scrum team working for the first time in a secure enterprise environment that had been built along Waterfall architectural principles.

However most stakeholders are business people who goes into the Sprint Review all blank and not able to test the stories on-the-fly. They need more time to absorb the stories and go through the stories at their own pace.

Sprint Review is not a "test workshop" or "acceptance workshop".
They are no "acceptance phase". They are no phase in Scrum.

The increment is already potentially shippable before the review, because acceptance testing (actually any kind of testing like unit ; security ; performance ; usability ; portability...) was done item by item during the sprint.
The review is a opportunity to inspect the increment with high level of transparency, to get feedback from the key stakeholders and to adapt the Product Backlog accordingly.

Good. I was trying to understand the aspect of inspect in sprint review by stake holders. This discussion has helped me understand that stakeholders involvement in sprint review.The product owner who understand the needs of the stake holders and also understand the sprint goal is the person who might be capable and fulfiling the acceptance test sign off role.The stake holder can only allign with the product owners perspectives.

In situations where there are larger organizational impediments at play (separate business uat teams, uat involving multiple streams who may or may not be on agile, agile/scrum being pushed ground up with the broader org not signed up), the more practical approach is to have some capacity allocated 5o manage the urgent requests. You can either have this reflected as non-feature user stories or as an agreement with the PO and employ lower velocity/ capacity during sprint planning. You start with this as a short term approach and continue to work with PO and broader org to move to the ideal scrum recommended ways.

As a Scrum Master observing this situation, I would consider measuring the cost of canceling a Sprint. Ask the people department a.k.a HR and ask them to furnish the team cost for a Sprint of time. Calculate a daily cost of development. Calculate the cost of rolling back un "Done" work based on the days needed to make them "Done" e.g. 1 day on a 6 person costs X dollars. Consider that cancelling a Sprint also triggers immediate Sprint Planning which is also team time at the daily team rate/salaried equivalent. Also, measure the team reaction to shifting priorities and lost focus. Do these activities piss off your highly paid developers? Will this affect their work in the future? Will they start looking for a more stable job in the future!?

For good reason, the Scrum Guide says, "Sprint cancellations consume resources, since everyone has to regroup in another Sprint Planning to start another Sprint. Sprint cancellations are often traumatic to the Scrum Team, and are very uncommon."

If the work generated from UAT is so valuable that it makes sense to continually cancel Sprints. Chances are though, the new work can wait until next Sprint. Your developers will thank you :).