Background
On November 22nd, 2019, I gave the closing keynote at Scrum Deutschland, a talk called ‘The Four Things You Do To Prevent Value Delivery.’ In this keynote, I discussed the trends I found during my research at multiple organizations.
For background, I’m often called into organizations to investigate their ability to deliver. Such a study is holistic; it involves looking at the organizational design, technical capabilities, culture and knowledge, and type of control and metrics used to define success. I get these kinds of request, usually because of three main reasons:
- Something went wrong, and the organization wishes to know what happened where and when.
- Work (or sometimes even a transformation) is ongoing, and the organization wants a second opinion on risk or improvement factors.
- The organization expects an inquiry or audit into its delivery capability and wishes to be prepared.
As expected, repeated studies have allowed me to observe some patterns. Some of these have to do with ‘the system,’ the whole of hierarchy, setups, technology, and so on that make up an organization. The impact of the human element in all of this surprised me. Not because of its presence, but more how it impacted everything. I didn’t find any malicious behavior at any point. What I found where people trying their best, making an effort to be helpful, and yet this often resulted in issues in the delivery capability. It is these observations that I shared in my (happy to say well-received) keynote. And now in these four articles.
Example
Some time ago, I was called into an organization to perform a study. A new product had been developed to replace an older, third party solution that was no longer tenable. The launch was less than satisfactory. The UI was a mess, many features were bare-bones or wholly absent, and the product was so unstable that it was down more than up. The organization had to revert to the old, third party solution at significant cost, after several (costly) years of development. In my research, I found many issues throughout the development process, but one stuck out in particular. In Scrum, product accountabilities are clear: The Product Owner is accountable for the business of the product (strategy, releasing, profit & loss, etc.) and the Development Team is accountable for the quality of the product (releasability, maintainability, stability, documentation, architecture, etc.). While the organization claimed to have used Scrum, these accountabilities were not secured in this way. I spent many interviews trying to find out who was accountable for things. Product Management would point to Development Teams; Development Teams would point to Architects, Architects would point to Project Managers, back to Product Management, and so on. Every time I would get the same answer:
‘That’s on us? I thought that was on them.’
Elaboration
Before we dive into an explanation of how these things happen and how they can be avoided, it’s worthwhile to look at the history of this company. Because there wasn’t much special about it. In all honesty, there are not that many unique companies. This company had, like many others that reach a certain size, been set up bureaucratically and was in the initial stages of moving past that paradigm into one more suited for creative knowledge work. So let’s look at what that word means. What is bureaucracy?
First off, it’s important to note that bureaucracy isn’t bad. It is undoubtedly better than the system it replaced, namely various flavors of despotism. In many ways, it is a vital part of modern civilization. As with most things, it takes people to turn it into something bad. It is for this reason and one other that I’ll explain later, that most people I’ve worked with over the years (and yes, that is an anecdotal observation) tend to dislike the system.
German sociologist Max Weber, who was the first to study bureaucracy formally, listed the following traits:
- Hierarchical organization
- Formal lines of authority
- Fixed areas of activity
- A rigid division of labor
- Regular and continuous execution of assigned tasks
- All decision and powers specified and restricted by regulations
- Officials with expert training in their fields
- Career advancement dependent on technical qualifications
- Qualifications evaluated by organizational rules, not individuals
The resulting system is best described by Weber’s assertion that ‘The fully developed bureaucratic apparatus compares with other organisations exactly as does the machine with the non-mechanical modes of production.’ We know now that this describes a system that assumes a simple context, with high predictability and high repeatability. The people in this system are encouraged to specialize and, thus, have limited competence. In other words, they are competent in their field, but not in any other. It is a system of managed incompetence.
But this company was changing.
Explanation (Why this is wrong)
The field of product development is not a simple one. In this context, complex work is done. Meaning that there are more unknowns than knowns, and you don’t know what you don’t know. There is very little predictability to be had, and virtually no repeatability as you never solve the same problem in the same way twice. Speed is the name of the game here, as the more often you deliver, the more often you can check assumptions and adjust course. The First of Four article explains these things further. For this article, let’s see what this does to the traits of the bureaucratic system if one is trying to be effective in a complex environment:
- Hierarchical organization - There simply isn’t time for decisions to go up and down the ladder. For speed’s sake, authority is pushed down to the work floor for all but the most impactful decisions.
- Formal lines of authority - Forget the lines. In creative knowledge work, domain knowledge is the highest a the level where the actual development is being done. Meaning that accountability remains clear, with the change that the people doing the work are the people accountable for that work.
- Fixed areas of activity - As new insights emerge, products might shift in the market they serve or the technologies they use.
- A rigid division of labor - Forget it, while accountabilities are clear (and relatively broad, again for speed’s sake) collaboration is highly encouraged. DevOps is a clear example.
- Regular and continuous execution of assigned tasks - Creative knowledge workers are problem solvers, not solution implementers. Assigning tasks to them is a waste of brainpower.
- All decisions and powers specified and restricted by regulations - Due to the unstable environment, organizations are constantly forced to make decisions and perform actions in situations that often have no clear regulations (or to which the regulations no longer apply).
- Officials with expert training in their fields - This one is very tricky, again due to the unstable nature of the context. Often, officials are expected to learn, explore, and invent as they go.
- Career advancement dependent on technical qualifications - The traditional career path is abandoned in favor of other incentives because content and quality authority are already at the ‘lowest level.’
- Qualifications evaluated by organizational rules, not individuals - Due to the lack of standardized tasks and continuously evolving requirements, qualifications include not just an objective measure of hard skill, but increasingly the ability to collaborate productively in a high-pressure environment. The required soft skills can only be evaluated by the individuals one will interact with.
Yeah, that’s quite a lot of text. In simpler terms: While a bureaucratic system works great in a simple and complicated context, it is an extremely inefficient approach to organizational control in a more complex one. So change is needed. And not just operationally. It is a wholly different paradigm for an organization to move into.
So what happened?
It is said the road to hell is paved with good intentions. That is, to some extent, what happened here. The organization had, in their transformation to a more Agile way of working, decentralized control and encouraged self-organization. They had let go of many things that used to be clearly defined. And they had realized that as the company was learning to deal with this, it was going to happen with some struggles. They had embraced the value of failure. But there is failure, and there is failure.
What this organization had done was use something I call the Underpants Gnome approach to transformation, after the South Park episode that popularized the concept. It consists of 4 simple steps:
- Increase the autonomy and responsibilities of those that deliver the value.
- Accept that change is hard, so increase the tolerance for failure.
- ??????
- Success! Value!
If you’re feeling somewhat uncomfortable with this approach, you’re right. Because some failures are great. They happen when, to the best of your knowledge, experience, and education, things ‘should’ work, but don’t. It means there’s something new to be learned. But bad failures happen when you come at things completely unprepared. In this case, you can’t distinguish between new and novel information, and the blatantly obvious. The Underpants Gnome approach is terrible because it allows people just to stumble around, because all failure is tolerated, even the preventable kind.
Solution
It was during this study that I was sent an article by Gary Pisano called ‘The Hard Truth About Innovative Cultures,’ published in the Harvard Business Review. It’s worth a read. What stood out to me was the very first, somewhat paradoxical statement:
‘Tolerance for Failure but No Tolerance for Incompetence’
And there we had it. While there were other issues at play, this explained the situation I described earlier. The organization was committed to moving to a new paradigm. They had embraced Agility to run their business and encouraged Scrum at the team level. But in their eagerness, they had not assured themselves that the people given broader accountabilities were competent enough for those accountabilities. In other words, they went from the bureaucratic system of managed incompetence to unmanaged incompetence, instead of the Agile system of unmanaged competence.
The Scrum Teams barely knew Scrum; they certainly had no formal education. The Development Team had zero experience in maintaining architecture, robust testing, and thorough documentation, given that until that point, they had just been coding requirements. Leadership didn’t know what they ought to be doing, just many things they felt they shouldn’t be doing anymore. What they did do, was tolerate incompetence, including their own.
It is essential for any organization that deals with creative knowledge work to have a high degree of tolerance for failure. It is equally important not to tolerate incompetence. This doesn’t mean getting rid of everyone that is ‘incompetent.’ What is means is assuring that people receive the education, support, and experience to be competent for the accountabilities they have—guiding them until they no longer need guidance because this assures and increases the quality of failure.
Stop tolerating incompetence
Sources
Wikipedia - Bureaucracy
Gary Pisano - The Hard Truth About Innovative Cultures
Max Weber - Economy & Society
Ken Blanchard - The One Minute Manager & Leadership
South Park - Gnomes