requirements organization in agile when backlog is not enough?
I find pretty hard to have a clear view of the requirements through backlog user stories alone.
Our project is a complex web application for finance. The business logic is very complex, changeable and subject to endless business discussion: the final implementation appears to be the result of many stories spread over many sprints
As a small example, we have a field that calculates the interest rate:
- in the first story the formula is provided
- next commissions are added
- then limit and validation rules are needed
- everything is changed by a new formula
- finally we realized that the formula must depend on the client and parametrisation is applied
...and so on.
When somebody asks me how the interest rate is calculated, I have to "replay" all the story. Or better, ask developers to look at the code.
Attempts to keep documentation updated in a wiki are failing because of the many parties involved in the discussions...stakeholders, business analysts, actuarials, product owners, etc....The example above it's just one happening in a small module of the whole application.
Testing coverage is also sketchy due to the complexity of the calculations and the frequent business logic changes.
Conclusion: we depend heavily on "individual and interactions"...as the agile manifesto says....but it doesnt look sustainable to me at all.
I'd like to propose some kind of structure, but it's not clear the value, it's hard to ask people to maintain it, everybody has its own "wiki" or knowledge base. Without clear requirements is also difficult to explain how we are spending the money and on what.
The Manifesto for agile software development says
That is, while there is value in the items on the right, we value the items on the left more.
It doesn't say that you shouldn't do the things on the right. The right hand side of "Working Software" is "comprehensive documentation". There IS value in documentation, but the trick is to finding the right balance. Comprehensive documentation is cumbersome to maintain, especially in an environment you mention. However, there could be a balance where certain things would require updated documentation. For example, if the business frequently asks how interest rates are calculated, then I would see that as a documentation requirement that needs to be met by the Scrum Team. Remember that in agile and Scrum, information is freely and readily available.
I suggest that you work with your stakeholders to determine the minimum required documentation that needs to be maintained. Make stories to create or update it. Then make it part of the Definition of Done that any documentation related to the code that was under change must be updated to reflect current state.
the final implementation appears to be the result of many stories spread over many sprints
When working on a complex product, it's the implementation each and every Sprint which allows empirical process control to be established.
Without clear requirements is also difficult to explain how we are spending the money and on what.
What then is being done to make sure work is Ready for Sprint Planning? It doesn't sound as though work is being refined enough to make Done Increments achievable.
When somebody asks me how the interest rate is calculated, I have to "replay" all the story. Or better, ask developers to look at the code.
Describing a Product Backlog item with a user story is only one way to make the requirement transparent. Has your team looked into Specification by Example? There are lots of examples found on Google.
I find pretty hard to have a clear view of the requirements through backlog user stories alone
We have a saying that the Product Backlog is "flat". Some people prefer to use a story mapping (see Jeff Patton's book or web site) session to make the Product Backlog more transparent, and carve out releases and an MVP. Again, plenty can be fund on this topic.
Remember you just need enough "ready" Product Backlog for the next Sprint. We have a saying that we refine at the last responsible moment.
Daniel hits it right on the head - work with the stakeholders (and I'm including the Developers as stakeholders) to understand not only what the minimum useful set of documentation is, but I'd also add talk about how to present it in an effective way so it would be useful rather than just checking a box.
In software, automated tests are a good form of documentation, especially if you can capture the assertions of the test in a format that is readable by stakeholders outside of the Developers. One example of this is Gherkin language. Not only do you have human-readable statements about the expected behavior of the system, but you also have automated tests that can be executed to assert that the true behavior matches the expected behavior.
You may need other formats. If you couple visual (graphical, tabular) models of capturing requirements (see Visual Models for Software Requirements for plenty of examples) with Agile Requirements Modeling (iteratively and incrementally capturing requirements, just-in-time requirements engineering and modeling), you can build up other useful representations of the current "as-build" requirements as the system changes over time.
You can include updating your system models as part of your Definition of Done. Whenever a Product Backlog Item changes the system in a way that is represented in a model, make sure that the model is updated concurrently with the design, development, and deployment of the change.