Question about UAT
Hey everyone.
I have about 8-9 years experience with SCRUM and as many others my experience varies from
10% implemented to 90% implemented, but never 100%.
It kind off makes sense to me because I have almost never met a client willing to play the "oblivious to everything else" religion
that SCRUM requires.
A thing I always encounter is the fact UAT is performed by business/client and never inside the development team, which means
after a 14 days sprint we deliver a release to review. SCRUM dictates that this is a product and ready for release, however in reality the clients might not agree.
After review this release is placed on a stage environment where the client spends days testing it meanwhile we start a new Sprint.
As a result it looks a bit like this:
I'm wondering a few things
1. How do you guys compensate the SCRUM Process if you cannot force the UAT testing into the actual sprint
2. How to you facilitate that bugs from the previous sprint will come from UAT testing and will always be important
(I have tried all 3 methods below
A: making room for bugs which is a bit problematic because you dont know if you leave the right amount and it makes a fake filled sprint
B: Not count bugs, which just ends up causing us to not hit sprint goal
C: Adding tasks that are swapped for bugs, which of course is also a bit of a mess because then the sprint goal changes
I'd like to hear how people handle this in the real world :)
Scrum is not implemented for its own sake, but to establish empiricism under complex conditions of high uncertainty. That's what the framework is for.
What you're trying to "compensate" for, really then, is a lack of empirical process control. But if your situation is a complex one, the truth is that you can't. Empiricism is as "real world" as you can get. There's no compensation to be had.
So, what are the consequences of the problems you describe in terms of empirical feedback and validated learning, Sprint by Sprint? Trying to implement Scrum does at least increase transparency over these issues.
I dont say this to belittle you, but I tend to stay away from forums such as these because whenever someone throws a scenario that is almost always out of their hands, you can be sure instead of help someone will instantly respond "THATS NOT HOW SCRUM IS SUPPOSE TO BE!".
I dont challenge how SCRUM is written, however I have worked on many multi million dollar projects, including nation wide projects, in some of the most regulated business in the world, banking and also flight industry.
I have yet to see anyone implement a 100% real SCRUM, in fact I have yet to hear of anyone doing it
That being said, while I have now clicked on your profile and realise you know a lot more about SCRUM than me, I would still like to challenge what you write.
As long as Product owner approves during the review there is absolutely nothing in SCRUM preventing them from executing UAT tests afterwards and reporting in product backlog items.
So instead of shouting thats not how you are suppose to do it, it would be refreshing if another approach was chosen, because there is absolutely nothing I state in the above that can be stated directly opposite in SCRUM
My advice is to put to one side how Scrum is or is not meant to be, and to avoid projecting that concern. Instead, think about empiricism and complexity, and the consequences of the problems you describe in terms of empirical feedback and validated learning.
In one of Ken Schwaber's books, he reminds us of this:
…the ScrumMaster has to operate within the culture of the organization.
The ScrumMaster walks a fine line between the organization’s need to make changes as quickly as possible and its limited tolerance for change.
…sometimes these changes are culturally unacceptable and the ScrumMaster must acquiesce. Remember that Scrum is the art of the possible. A dead sheepdog is a useless sheepdog.
Sometimes you can't fix the problem or impediment and have to live with them, but it does not mean we go along with the status quo if we can cause change.
What I might do if UAT is part of the required process is to make the situation transparent to all concerned including your customer. There's a good reason why Scrum requires the product increment to be in a 'Done' (i.e. 'potentially releasable') state every Sprint. Undone work is more expensive to fix outside of a Sprint in complex environments like software development. Undone work usually negatively impacts the release timelines. The later a defect is found, the more expensive it is to fix. And to your point about UAT bugs from prior Sprints disrupting the current Sprint, there's a huge cost to context switching for the Developers. It also impacts team health.
Perhaps you're working on a fixed cost/timeline so the customer thinks they don't care about cost. So they "think". In the long run, it impacts your team's health or they might be cutting corners elsewhere to fix the UAT bugs (i.e. building up tech debt). We call this a squirrel burger.
Coming from experience in regulated industries or serving customers in regulated industries, I have to deal with UAT and customer validation activities regularly. My advice is always the same: move it out of the Sprint.
It simply does not make sense to consider UAT within the context of the Scrum framework. UAT, when done right, is not only outside the control of the Scrum Team(s) building the product, but outside of the development organization entirely. Why would you put the constraints of the Scrum Team - mainly their way of working and schedules - on your customers and users? You wouldn't.
Instead, I treat delivery to UAT in the way that many teams in other contexts consider delivery or deployment to production. That is, UAT is not a defect-finding activity. Customers should not be finding significant amounts of defects during UAT, and those that they do find should not generally prevent their acceptance of the system under test. The feedback that comes out of UAT should be, more often than not, the kind of feedback that can go onto the Product Backlog for the Product Owner to order among all of the other work, then go through the refinement process and eventually enter Sprint Planning for development.
Depending on the team's ability to deliver a quality product and the frequency of delivery and UAT, the team may opt to reduce the amount of capacity dedicated to the Sprint Goal around the time UAT is happening. This can give the team the opportunity to respond to any feedback or defects that do come out of UAT yet still be able to achieve their Sprint Goal.
If UAT finding critical issues that disrupt the team's planned Sprint is a recurring issue, the team should be using their Sprint Retrospective to understand why the quality of the product entering UAT is insufficient to meet customer needs and then address those problems.