Scrum problems: Need suggestions for improvements

The team does not develop features for the final product. We develop infrastructure components. This means we are "between" several other teams (including teams that develop features for the product itself). In this position we are involved in a lot of things, what makes it quite interesting. The downside is that we are faced with A LOT of dependencies that cannot be completely resolved before we start working on something. In fact, the majorities of these dependencies only become obvious while working on the backlog items that are blocked by these dependencies.

It sounds like you may have a platform team. There's not enough information here, but it is something to look at, perhaps for inspiration. You don't need to be working on the final, customer-facing product to apply product development techniques. You can consider the infrastructure as the product that is delivered to multiple other teams, who are the customers and/or users.

I'm not sure where the dependencies are coming from, though, but that's something to look at and understand.

Furthermore, the heterogeneity of our tasks and the fact that the priority of the backlog items most of the time prevents the clustering of similar tasks, makes it nearly impossible for us to define a meaningful sprint goal. Dev A and B will work on topic X, Dev C on topic Y and Dev D on topic Z. Topic X, Y and Z may have nothing to do with each other.

A Sprint Goal does not have to unify all of the work that is undertaken in a Sprint. I find it very helpful to think about the Sprint Goal as a focusing tool. Consider that you have work for X, Y, and Z planned for the Sprint. However, during the course of the Sprint, unexpected things come up and the team will have to shuffle around and will only be able to finish one. Which is the most important thing to deliver? That becomes your Sprint Goal.

For example, if you decide that delivering Y is the most important and most valuable thing to do, then if Dev C has an unexpected absence or needs help, then the other developers would be expected to prioritize helping Dev C get Y done before the end of the Sprint timebox.

I would, however, recommend focusing on outcomes over outputs. Rather than making goals about getting certain amounts of work done or delivering a certain feature, the goal should be about enabling stakeholders to achieve some kind of objective.

We have taken steps to actively spread knowledge between the team members. By now, this has led to T-shaped individuals. And this has worked quite well. But it is the nature of the tasks that we work on that even task we have done multiple times before, and topics that are basically known to everyone, almost always have some new/unusual circumstances. This makes it hard to estimate these tasks, and even routine tasks require a relatively large amounts of research. We use research backlog items (spikes) quite often, and in our opinion, it does not make sense to timebox them or to use rather small timeboxes. Most of the time this timebox would be over before we even know how to solve the task resulting in another spike...

It's not clear what is going on here, but this could be related to what you've identified as Problems 3 and 3a, so I'll elaborate there.

The backlog items are sometimes poorly described. We have discussed this with our PO, and he understands the problem. These discussions result in an improvement, but after a short period of time old habits return. How can we improve this in a sustainable way?

We use regular refinement meetings to work on the description and the understanding of backlog items. I’ve often read that user stories should not be described in too much detail, but if we don’t do that, the Devs often do not understand what exactly is required to be done. (By this, I do not mean the DoD, but the actual work required to achieve the DoD).

In my experience, the vast majority of refinement doesn't happen at a refinement meetings. It is useful to get the Product Owner and all of the Developers together to synchronize, reviewing the state of the Product Backlog, making sure that enough of the top of the backlog is well-refined and ready, and making plans to figure out how to go about making sure that enough is refined and ready if it isn't. However, most of the refinement actually happens individually, in pairs, or in small groups.

Making sure that you're spending enough time in refinement and that you're using that time effectively is key. If you consider doing research to understand the work as part of refinement, you may need to spend more time in refinement. Just be sure to focus on answering the important questions to reduce the risk sufficiently to make it tolerable to your organization, since different teams and different organizations have different tolerances for risk and ambiguities.

This is our biggest issue, and I strongly believe that it is a consequence of the problems I mentioned before, especially problem 1. Basically, in every sprint we are unable to finish all the backlog items that we planned for the sprint during our planning meeting. Yes, the Devs do estimate the user stories and we plan according to our velocity and capacity (illness, vacation, etc.). We tried to reduce the number of backlog items, but this resulted in some team members having nothing to do at all. We tried to slice the backlog items in such a way that multiple Devs can work on different aspects of an item. However, most of the time this is not feasible because the nature of the task renders it pointless for more than one person to work on it.

If we start the sprint with enough backlog items so that each Dev has something to do, often one or two Devs get blocked and new items have to be pulled from the product backlog into the sprint backlog. Often it is not clear whether the reason for blocking will vanish or not. Therefore, the blocked items remain in the sprint. If the item is still blocked during the planning of the next sprint, it will stay blocked in the product backlog.

At the moment, we carry other incomplete backlog items from sprint to sprint until they are really done. Since we do not count partial work, our velocity is rather low for a couple of sprints, followed by a comparatively huge peak in which a bunch of items carried along are completed.

Another reason for unfinished work are quite large backlog items that cannot be split. We tried splitting them and realized: Splitting the items produces so much administrative overhead during the execution that we do not gain any advantage.

The Sprint should not be about completing all of the selected Product Backlog Items. The purpose of the Sprint is to deliver value, usually by achieving the Sprint Goal. It's also not about making sure that all individuals have things to do. If everyone's working on something that is equally important, there's no room to shift work around to focus on the most valuable or most important thing.

I'd recommend looking at crafting appropriate Sprint Goals and using past historical data to understand how much work you can actually achieve in the Sprint timebox. Don't pull in more work than you can actually complete. And don't be overoptimistic in how many hours you have - you aren't doing 8 hours of hands-on product work in an 8 hour workday, you're doing a fraction of that.

we are not able to change the environment around us

If you can't, how will the situation ever improve? Who wants Scrum here and why?

Consider team structure as an example. Would teams choose to self-organize along the lines you describe, with those extensive dependencies...or would they come up with smarter arrangements for creating Done increments more transparently?

If some other authority makes these decisions, what outcomes are they hoping to achieve by denying teams the ability to control their environment?

It sounds like you may have a platform team. There's not enough information here, but it is something to look at, perhaps for inspiration.

Thank you, I'll have a look.

You don't need to be working on the final, customer-facing product to apply product development techniques. You can consider the infrastructure as the product that is delivered to multiple other teams, who are the customers and/or users.

That is already what we are trying to do.

I'm not sure where the dependencies are coming from, though, but that's something to look at and understand.

There are many reasons for the different dependencies. For example, the delivery cycles are not in sync: Team A does a four-week sprint and starts on Monday, Team B does a two-week sprint and starts on Wednesday, the actual product cycle is also quite different, and so on.

We need input from many other teams, and normally I would suggest that we wait to implement until all the deliverables from other teams are ready. However, the reality is that we need to start our implementation even if we get blocked halfway, otherwise we will most likely not have enough time to finish our implementation in time. Also, if we found a bug in another team's implementation, we have to wait for a release cycle before we can resume our work. The same problem occurs when we find a missing part in an API.

A Sprint Goal does not have to unify all of the work that is undertaken in a Sprint. I find it very helpful to think about the Sprint Goal as a focusing tool. Consider that you have work for X, Y, and Z planned for the Sprint. However, during the course of the Sprint, unexpected things come up and the team will have to shuffle around and will only be able to finish one. Which is the most important thing to deliver? That becomes your Sprint Goal.

We usually don't have a most important thing and as I described, most of our backlog items can hardly be done by more than one dev.

For example, if you decide that delivering Y is the most important and most valuable thing to do, then if Dev C has an unexpected absence or needs help, then the other developers would be expected to prioritize helping Dev C get Y done before the end of the Sprint timebox.

We try to help each other whenever and wherever possible.

I would, however, recommend focusing on outcomes over outputs. Rather than making goals about getting certain amounts of work done or delivering a certain feature, the goal should be about enabling stakeholders to achieve some kind of objective.

I don't see how this can help us in creating a meaningful sprint goal.

In my experience, the vast majority of refinement doesn't happen at a refinement meetings. It is useful to get the Product Owner and all of the Developers together to synchronize, reviewing the state of the Product Backlog, making sure that enough of the top of the backlog is well-refined and ready, and making plans to figure out how to go about making sure that enough is refined and ready if it isn't. However, most of the refinement actually happens individually, in pairs, or in small groups.

Making sure that you're spending enough time in refinement and that you're using that time effectively is key. If you consider doing research to understand the work as part of refinement, you may need to spend more time in refinement. Just be sure to focus on answering the important questions to reduce the risk sufficiently to make it tolerable to your organization, since different teams and different organizations have different tolerances for risk and ambiguities.

Thank you, the tip about small groups is great, I'll give that a try.

The Sprint should not be about completing all of the selected Product Backlog Items. The purpose of the Sprint is to deliver value, usually by achieving the Sprint Goal.

Sorry, but I have to disagree on this one. As far as I understood Scrum, the purpose of a sprint is to deliver value by completing all selected items. If the work is not done by the end of the sprint, something may be wrong. As mentioned before, I still find it difficult, if not impossible (maybe it is possible, but I don't know how), to define a sprint goal.

It's also not about making sure that all individuals have things to do. If everyone's working on something that is equally important, there's no room to shift work around to focus on the most valuable or most important thing.

Of course, it should ensure that each individual has something to do. No one said that all tasks are equally important, but we can't have some people sitting around waiting for something unexpected to happen...!?

Don't pull in more work than you can actually complete.

Sorry, but that's just a platitude. That is exactly what I was trying to say: We don't pull in more work than we can complete, but often we need to add new work or some developers will have nothing to do because their former items are blocked.

If you can't, how will the situation ever improve? Who wants Scrum here and why?

I don't know, that's why I'm here. The management wants it. Actually, I wonder if we should use scrum at all or try something else and call it scrum just for management^^.

Consider team structure as an example. Would teams choose to self-organize along the lines you describe, with those extensive dependencies...or would they come up with smarter arrangements for creating Done increments more transparently?

I don't know because I am not a technical expert, but I would assume that this whole structure is naturally grown.

Thank you again!

Dave

May I ask why no one is adding more responses? I did not intend to offend anyone by rejecting some suggestions. If that's how it sounded, sorry.

There are many reasons for the different dependencies. For example, the delivery cycles are not in sync: Team A does a four-week sprint and starts on Monday, Team B does a two-week sprint and starts on Wednesday, the actual product cycle is also quite different, and so on.

We need input from many other teams, and normally I would suggest that we wait to implement until all the deliverables from other teams are ready. However, the reality is that we need to start our implementation even if we get blocked halfway, otherwise we will most likely not have enough time to finish our implementation in time. Also, if we found a bug in another team's implementation, we have to wait for a release cycle before we can resume our work. The same problem occurs when we find a missing part in an API.

I would strongly recommend synchronizing the cadences of the teams. Even if you can't fully synchronize them, having them be multiples of each other and share a common start and end day will go a long way to making teams be able to coordinate. Although it does take some freedom away from the teams, there are advantages to having certain commonalities once you're in a scaled environment that enables the teams to be more successful in other areas.

Improving overall quality through strict Definitions of Done help. It may not help if a team is not able to finish work because of conflicting priorities, but a focus on quality can help to reduce bugs or unexpected missing parts of APIs. These really shouldn't be a surprise.

Sorry, but I have to disagree on this one. As far as I understood Scrum, the purpose of a sprint is to deliver value by completing all selected items. If the work is not done by the end of the sprint, something may be wrong. As mentioned before, I still find it difficult, if not impossible (maybe it is possible, but I don't know how), to define a sprint goal.

You do not understand Scrum correctly. A Sprint is not about delivering value by completing all selected items, but by achieving the Sprint Goal.

Of course, it should ensure that each individual has something to do. No one said that all tasks are equally important, but we can't have some people sitting around waiting for something unexpected to happen...!?

Sometimes, someone being busy on less important work is more wasteful than having them be idle. So yes, maybe consider having people idle. Instead of having them be fully idle, maybe consider having them pair or group up and work on fewer things.

This also goes to the next point. If you have work that is blocked, don't reach for new work. Consider doing something to get unblocked instead of starting something new and having more work-in-progress. WIP can be very wasteful.

Thank you again!

You do not understand Scrum correctly. A Sprint is not about delivering value by completing all selected items, but by achieving the Sprint Goal.

I looked it up in several books (succeding with agile by Mike Cohen and some others). It is true that the Scrum Guide does not mention the completion of all backlog items by the end of the sprint. It is also true that the sprint goal is the most important part of a sprint. However, in each of the different books it is explicitly or at least implicitly stated that a major goal of the development team should be to complete all the work they have committed to. Yes, achieving the sprint goal is key, but this should be done by completing the committed work. Since developers choose the backlog items to achieve the goal, I would argue that this whole discussion is two sides of the same coin. 

Sometimes, someone being busy on less important work is more wasteful than having them be idle. So yes, maybe consider having people idle. Instead of having them be fully idle, maybe consider having them pair or group up and work on fewer things.

How can working on less important tasks be more wasteful than doing nothing at all? If developers can contribute to other important tasks, they will. But doing nothing affects the morale of the team.

This also goes to the next point. If you have work that is blocked, don't reach for new work. Consider doing something to get unblocked instead of starting something new and having more work-in-progress. WIP can be very wasteful.

As I have mentioned several times, in our particular situation we cannot actively do anything to unblock these items. So if doing nothing is not an option (I can hardly explain to others (management) that part of the team is surfing the internet because they have nothing better to do), we need to add more work to the sprint.

Don't get me wrong, I understand where your suggestions are coming from. But the reality in the team is different. I have noticed that in my organization an important part is missing: every team uses Scrum (sort of), but the whole structure is not some form of scaled Scrum. So that could be the main problem: no overall alignment between the different teams.

Maybe we are just not able to improve "scrum" in this environment