Unrealistic assumption?

The exact statement mentioned in the software engineering course is like this:

Agile methods are not mature enough:

Unrealistic assumptions (e.g. Scrum)

Lack of scalability (All, more or less)

Lack of a specific, unambiguous process (e.g. XP, Crystal)

Actually the slides are dated as the references are:

- Webster, S.,  “On the evolution of OO methods”,  Bournemouth University, 1996.

- Graham, I., Object-oriented Methods: Principles and Practice (3rd Edition), Addison-Wesley, 2001.

- Abrahamsson, P., Warsta, J., Siponen, M. T., Ronkainen, J., “New Directions on Agile Methods: a comparative analysis”, Proceedings of the International Conference on Software Engineering – ACM/ICSE 2003, 2003, pp. 244-254.

Scrum is very good at highlighting organizational impediments to agile practice. Sometimes the evident need for deep and pervasive change is held to be unrealistic.

Could we say that the unrealistic assumptions are thinking of constant refactoring as an effective way for code production and that the technical dept can be payed in the future?

reference: https://effectivesoftwaredesign.com/2014/03/17/the-end-of-agile-death-b…

Seeing how those references are all more than 20. years old it might make sense to look at things much more recent and also look at what may cause the issues which is generally not the "process" but instead the people doing the work.  Check out this report which is an excellent depiction of the how vs. the what.

I find that Agile works well in smaller organizations, like start-ups, where the decision maker is in the C-suite and can say "make it happen" and all teams (developers, information security, compliance, architecture, data warehouse, sales, etc.) HAVE to work together to get it done. An exception to this would be visionaries like Elon Musk.

Personally, I have not Agile (and really product development/management best practices) work well in larger organizations because attempts at building something valuable is left to middle-managers who don't have the authority to get anything done outside of their teams. This is how impediments are created and then a patchwork of managers and directors come together to create a messy product with unnecessary integrations which leads to technical debt. Add to this that MBAs (as Elon Musk recently pointed out) and not engineers are running large product companies. These folks at the Director-level roles are taught that project management works well because of the planning and structure in place. So they are terrified of innovation due to the fear of failure. Best to stick with business as usual since that leads to a steady paycheck.

I understand that any methodology or fremework applied in a wrong way will generate bad or unexpected results. Waterfall has not been so efficient in the last few decades to meet the demands of complex solutions in a constantly changing market, which is why the agile manifesto and scrum were created. I realize that many organizations implement Scrum in a hasty manner, generating frustration and false impressions about its efficiency.

 

Dealing with scale challenges is complex in any context, and agile and scrum is no different. there are several frameworks for scaling scrum teams like Nexus and Safe. Anyway, big companies around the world use scrum with excellent results, even with multiple teams in a single product. There are great books and articles on scale scrum cases.

I sincerely understand that there is the old way and the current way, which is agile and scrum. that simple.

For sure, different agile methodologies are applicable for different development scenarios which are well described based on the categories of problem situations proposed by the Cynefin Frameowork. i.e. Simple (obvious), Complicated, Chaotic, Complex and Disorder. Lighter agile methodologies like Scrum are best suited for Complex domains whereas not best suited for complicated domains where heavier agile methodologies such as DSDM or DAD are better suited because their analysis and design processes are based on expert knowledge and good practices.

That's what I learned from the articles I read but not by practice. So please correct me if I am wrong!

Lighter agile methodologies like Scrum ...

That statement is part of the unrealistic assumption.  Scrum is a framework. It is not a methodology. Methodologies provide methods for how things are done.  They provide details on what has to take place and how those things are done.  A framework is something that supports.  An analogy I used often is that a house has a framework.  The lumber that holds up the roof and walls.  But a house is built with rooms.  Some of them have specific purposes such as a bathroom.  But other rooms can be multi-purpose depending on the needs of the occupants.  For example, one of the "bedrooms" of my house is actually an office for my wife and I.  Our formal dining area is actually filled with curio cabinets that my wife uses for her craft collections.  There is no table in it at all.  

Many people consider Scrum fails because they try to implement it as a methodology using the words in the Scrum Guide as absolute rules.  It is not meant to be used in that way. It is a framework of events, roles, artifacts that when used can help an organization be more disciplined and successful at adapting to constantly changing needs.  But the process, the tools, the people you use within the framework are entirely up to you to decide.  As @Mark Adams points out not all organizations work the same and some are successful at implementing Scrum while others are not.  But for Scrum to be successful, people at all levels of the organization have to be willing to change and accept the principles and guidance of the Scrum framework.  They do not all have to follow the framework but they do have to understand how interacting with portions of the organization that are following Scrum can be effective or harmful. 

I have worked with large and small organizations that have Scrum being used effectively.  I have also worked with large and small organizations have failed to use Scrum effectively.  Size of the organization has nothing to do with success.  Willingness of the organization to experiment and realize that there are many ways of being successful is absolutely necessary.

I hereby thank all for your precious thoughts and ideas. I will use them in my future researches.

Regards, Alireza

I think organisations, particularly larger ones in my experience, don't like what they see when they look in the mirror. Scrum holds the mirror up, but it is the organisation's choice whether to look at the evidence and adapt because of it.

I often find this is due to a frozen middle management. Scrum Teams are overseen (sometimes from afar) by a manager of sorts, and is their escalation point if needed. If that manager isn't able to affect any positive change, then you'll find that inspection without timely adaptation has less value.

It all comes down to sponsorship. The sponsor of the change (to Scrum or another framework) needs to be visible but in a large organisation this is rarely the case. Success is more than possible with Scrum, but it needs a culture for the self-managing team to operate in. In many organisations this is held to be unrealistic.

Here is my summary of the main problem large organisations have with Scrum:

Scrum is a lightweight framework that helps people, teams and organizations generate value through adaptive solutions for complex problems.

Most large organisations have only a small number of isoldated teams working to develop solutions to complex problems (products). The rest of the organisation does other things to (such as: markets, sells, moves, reports on) those products, or "manufactures the product without introducing any change at all."

Source:

Assumptions Underlying Agile Software-Development Processes

Daniel Turk, Colorado State University, USA

Robert France, Colorado State University, USA

Bernhard Rumpe, Braunschweig University of Technology, Germany

The assumptions identified in this section that lie behind the principles of the Agile Alliance:

1. The Visibility Assumption: Project visibility can be achieved solely through delivery of working code.

2. The Iteration Assumption: A project can always be structured into short fixed-time iterations.

3. The Customer Interaction Assumption: Customer teams are available for frequent interaction when needed by developers.

4. The Team Communication Assumption: Developers are located in time and place such that they are able to have frequent, intensive communication with each other.

5. The Face-to-Face Assumption: Face-to-face interaction is the most productive method of communicating with customers and among developers.

6. The Documentation Assumption: Developing extensive (relatively complete) and consistent documentation and software models is counter-productive.

7. The Changing Requirements Assumption: Requirements always evolve, because of changes of technology, customer needs, business domains or even acquisition of new customers.

8. The Cost of Change Assumption: Cost of change does not dramatically increase over time.

9. The Team Experience Assumption: Developers have the experience needed to define and adapt their processes appropriately.

10. The Self-Evaluation Assumption: Teams are able and willing to evaluate themselves.

11. The Self-Organization Assumption: The best architectures, requirements, and designs emerge from self-organizing teams.

12. The Quality Assurance Assumption: Evaluation of software artifacts (products and processes) can be restricted to frequent informal interviews, reviews and code testing.

13. The Application-Specific Development Assumption: Reusability and generality should not be goals of application-specific software development.

14. The Continuous-Redesign Assumption: Systems can be continuously redesigned (refactored) and still maintain their structural and conceptual integrity.

From the discussion in the previous section it should be clear that the assumptions underlying agile processes do not hold in all software development projects and environments. This should not be surprising: Agile approaches are not process silver bullets. Because these assumptions are not met in all organizations and/or development environments, agile approaches, in their current forms, do have limitations. It is possible to extend agile processes to address their limitations. Such extensions can involve incorporating principles and practices often associated with more predictive, plan-based, or “traditional” development processes into agile processes. In general, users of agile processes need to ensure that practices based on assumptions that are not valid in their development environments are modified accordingly