Forecasting
Hi all
I have had a request from our project management to improve the way we forecast with story points.
Scenario: We have some stories in-progress at the end of the sprint. Lets say the story points add up to 20. We would discuss and carry forward the stories into the next sprint knowing that they would be completed in the first few days. We don't refactor the story points.
Management feel that the forecasting time line is skewed because we don't count the story points until they are done! So when they add up the outstanding story points for the project and divide this up by the average velocity to see hopw many more sprints are required, they feel that it is being pushed out because they are counting 20 story points of partially completed stories.
They are after some sort of metric to adjust timelines on carry-forward stories. I am a new Scrum Master and don't think its worth the time and effort to implement such a thing but rather focus on trying to resolve carry-forward stories.
I don't think I am the only one who has been asked for such a metric and I hope you can share some insights on how to best manage the situation.
Velocity is only a forecast and an internal team metric that should not be used by project managers to forecast the expected sprint for there project. The project manager could just ask the product owner how long he things it will take to deliver the value of the project mangers project. And give a forecast like probably 6-8 sprints for instant.
I think it would be wise to teach the projectmanagers a little more about scrum to let them understand
Velocity is the rate at which work is Done.
A measure of work that is not Done will indicate the rate at which technical debt is being accumulated. One symptom of technical debt is indeed the so-called "rolling over" or "carry forward" of work from one Sprint to the next.
In Scrum, any work that is not yet Done for any reason should be re-estimated on the Product Backlog, so that artifact truly reflects the amount of work that is thought to remain. The outstanding work may or may not then be planned into a future Sprint.
Stakeholders outside the team ought to care about the delivery of valuable increments, not velocity. The only purpose of estimation is so Developers can gauge how much work they can take on each Sprint so a Sprint Goal is met and at least one immediately usable increment delivered.
It seems to me that Story Points might be getting in the way more than they are helping you. Realistically you can’t use then to forecast as they aren’t real metrics or even real numbers. The basic problem is that they are not immutable which means for example 5 always equals 5 in every circumstance. 2+3 = 20/4 = 8-3 = 50 / 10 = 5. Story points aren’t like that, if you break down an 8 you might get a 5 and a 3, but you might get a 5, another 5 and a 3, and another 3. You decide on the story points before you do the work and as the work is complex (Scrum is for complex problems) that means you have to guess how big a complex problem is before you solve it, which is by definition unknowable. Then the story points get added together and compared with other teams (which is meaningless as everyone guesses differently) and all sorts of weirdness happens.
So after all of that what do I suggest? Move to some better metrics. Start measuring your cycle time from start to done on each PBI. Measure how many PBIs you finish per day (we call that throughput). Once you have 11 new you can do SME probabilistic forecasting that will be infinitely more accurate, and doesn’t involve any guesswork along the way. If you want to move to the gold standard, then Monte-Carlo simulation is your best bet. If you want to do a little more reading on this, then I would suggest the work of PST Daniel Vacanti - Actionable Agile Metrics which is a great starting point for understanding how to use metrics effectively in a scrum team. I’d also recommend the work of Troy Magennis too. You don’t have to be a maths guru to use this stuff, but it will give you so much better forecasting information to let you plan much better. if you are interested in a class I’d recommend PSK - Professional Scrum with Kanban which will get you started on metrics.
I hope that helps
Sven, Ian, Dan, thank you very much for your insights and suggested reading. It gives me a good starting point to take this forward.
Kind regards
Jack
Agree with what Sven, Ian and Dan shared above.
Sizing, or estimates in a backlog have a shelf life and this is one reason I have found them to be very inaccurate for forecasting or prediction. We learn through our empirical process and estimates made yesterday can change today or tomorrow based on what we learn.
Have you ever looked at past Sprints to see how long your backlog items take to complete (cycle time)? I bet you will find a lot of randomness and you will see how untrustworthy those point values really are.
Plot out your cycle-times and group them by point value and you may be surprised by the results. I have been doing this for the past few years across a number of different teams (not just my own) and the result is similar. You will find 1’s taking as long or longer than 5’s or 8’s, you will see clustering of cycle times averaging out across each estimate value and you will see a whole lot of inconsistencies in each range.
I have found throughput based forecasting to be better as it is binary measurement of Done work for a time period.
Using throughput, if we complete, on average, say 5 backlog items a week, we can convert this to 1 / day. If you have 10 backlog items left in your backlog you could forecast 10 days to complete. If you add backlog items, the prediction changes. If you increase or decrease throughput, the prediction changes.