Scrum - where Non-functional work is considerable.

> Can a project like that work well using Scrum principles...

That depends upon the risks. If the process is well understood and the risks are low, then there may be little point in using Scrum.

Complexity often brings risk however, and so if the process is a complex one a Product Owner may reasonably seek empirical evidence of risk control. Supplying a PO with analysis, design, and mapping data is not empirical evidence, as these are promissary notes used in place of value and are not increments of product that prove value. They are typical of staged waterfall methods, but not of agile practice where the only acceptible evidence of risk mitigation is the delivery of valuable increments of working product.

> ...the PO will have a very limited part to play in creating the
> product backlog items.

If that is indeed the case, then there is no point in attempting agile delivery as the PO will not be in a position to benefit.

The question then is, does the PO want to see risks being controlled by means of incremental delivery? Would this provide valuable assurances to stakeholders? If so, what would those increments look like?

Along with what Ian posted, these are my observations:

I am confused as to why you believe the PO will have a limited say in creating the backlog items. If the report requirements come from the PO, and if each story represents business value to the PO, then why wouldn't the PO be involved in crafting and offering the stories to the team?

Ashish, you have laid out a typical waterfall (phased) approach to developing these new 4 reports. Each step does not result in functionality that the end-user can benefit from. It is a technical approach, which unfortunately does not represent effective story mapping and writing.

Database table creation means nothing to the end user. Data mapping also means nothing. The only value to the business unit are the 4 new reports to be created, and the accuracy of the data contained.

Therefore, you need to ask yourself if there is a different way to look at the customer-facing items.

Is there value in providing the business unit with what each report will look like? Can this be done without ensuring the accuracy of the data shown?

Do all 4 reports need to be done at once? Can each report be delivered separately?

Do all of the attributes for each new report need to be done at the same time?

Ask yourself - What is the smallest piece of functionality that I can actually deliver to the business unit that has value to them?

What is critical is making the feedback loop from delivery of functionality to story requirements (and then back to delivery) as short as possible. That approach will do more than anything to help your organization transition to Agile.

There is a tendency for those in IT to "batch" items together. This is incredibly inefficient. Read up on Lean concepts to learn why.

Try not to think of the "work" from a development perspective (I'm opening this code, so it makes sense to batch up all of my changes to update it only one time). Think more holistically (end to end). What is the testing effort? Release and change management considerations?

Good luck to you.

If you are producing statutory reports, putting data that already exists in the system into specific places on the form, go waterfall.

On the other hand, you may be falling into the trap of starting at the solution (I need a report) instead of a user story: "when I'm trying to do X, I need to decide Y or Z...if I can see ABC, that tells me...."

You get the picture.

That user story gets built by asking "When do you run the report? Why do you run it? What decisions give you the most struggle? Why? How often do you run it? Does it have to be a report?" We turned on logging in order to cull out unneeded reports, and I found that one user ran a particular report over and over all day long...sometimes more than once in a minute. After discussing what she was trying to accomplish, we gave her a way to view inventory with lots of columns that she could filter. And then we killed her old report.

Thanks for all the comments.

@ Timothy, to your points :

1) 'Limited' because yes each report can taken as a separate story, however it cannot be decomposed any further. So there would just be 4 stories in all.

2) Yes, each report can be delivered incrementally, however the sequence of steps would still remain the same. So a lot of activities as mentioned in my steps 1 & 2 would be done for each report, irrespective if they are all carried out once for all reports or carried out individually for each report.

My question however is a more generic one and I used this example to make my point that for projects like these where non-functional work is far more than what the user would get as functionality, would Scrum still be effective.

Ashish,

I do not understand your assertion around non-functional work. It seems all of the work you are describing is going towards functional requirements (new reports).

If your question is whether Scrum is appropriate for a lot of development and testing work that results in a small amount of functionality (according to your opinion), the answer is yes.

There has been a lot of valuable information and advice posted in this thread. Take it all to heart, and good luck to you.

If i understand correctly.. the User story will be what has been mentioned before "When do you run the report? Why do you run it? What decisions give you the most struggle? Why? How often do you run it? Does it have to be a report?" "

During your sprint planning the following are tasks: 1) Design & Creation of a new database tables.
2) The input data for the reports come from external systems. That input data has to be mapped to the data tables created.
3) The data from the new tables has to be extracted and mapped to the XML format, before that is used to produce the reports.

Once you have them, estimate and do your sprint planning. If the same database, tables are being used for multiple reports then you can accordingly split your sprints in the first being completing the common task which will be used in all reports.

Ashish,

As rightly highlighted by Rishi - scrum framework can very well be used and applicable for scenario you have described -

I feel that the scenario described by you first needs to have Spics and Stories properly defined - In your example- Generating Reports seems like a "Epic" - which can be potentially be broken down into stories - e.g.

1) Story -1 As a "role" - I should be able to capture "transaction Data" and save the same for subsequent retrieval and summary/detailed reporting/analysis.
2) Story-2 As a "role" - I should be able to able to query the information for defined Criteria so that "benefit/value"
etc

As part of Product Grooming/refinement effort - each of the story can then be sized and assessed for level of completeness (Ready) as per the common understanding of the Team.

PO should then help with prioritization of each story using associated business value, risk, size and other applicable factors.

As part of Sprint Planning session - Development team should then agree on the PBI's for the next sprint in accordance with the team's velocity.

All the Engineering activities around Design/Architecture, Analysis, Coding, Integration and testing are nothing but Tasks against each PBI and should form the part of the Sprint Backlog as part HOW part of sprint planning. (Create Database schema, Define Index, Create Views, XML structure, Coding etc)

Elements like NFRs - for accessibility, performance, security etc should be part of DoD (Coding completed, Code Reviewed, Automated Test Cases Created, Unit Test Passed, System Testing, Integrated and Tested, OAT completed, Performance Testing Completed, Documentation completed etc). Once all the Tasks identified during Sprint Planning (and any additional tasks identified during the course of work/Daily stand-ups) and DoD check points are completed - should be Backlog item be marked "Done".

Sprint Review meeting should then be used for demonstrating the functionality/capability of "Done" PBIs to PO/Stakeholders for gaining final acceptance.

Hope this helps.
Pankaj