Lots of Agile transformations failed to deliver the expected/desired outcomes. And Scrum Masters were one of the first group of people to blame. Although I agree the industry created some certified but inexperienced Scrum Masters that couldn't perform, I think there were also factors out of Scrum Masters' control that significantly prevented their ability to perform.
Let's take a closer look at those factors:
Undervaluing the Scrum Master role and recruiting inexperienced Scrum Masters
The first question that pops up is why in the first place organisations would recruit inexperienced Scrum Masters. The Scrum Master role is a crucial leadership role in my opinion. Some organisations selected their Scrum Masters from their existing teams to just tick the box and some recruited dedicated but mostly inexperienced Scrum masters who fell short of fulfilling expectations.
Scrum Masters were not positioned to drive continuous improvement
Scrum Masters are often not given accountability or the support to go beyond their teams and drive change in the organisation. There has been a massive gap between how the Scrum Master role is defined in the Scrum Guide and how it is positioned by organisations. Scrum Masters with no authority and no psychological safety just got squeezed between what they think they should be doing and what their managers expect them to do. Scrum Master was seen as a Team level role and was confined to facilitating events, managing dependencies (not real impediments), administering Jira and so on. Besides, driving a change in an organisation requires different elements. Think about how an organisation makes decisions. Change decisions that require investment or affect multiple team/departments need a change management approach/mechanism that can include Scrum Masters in it.
Scrum Masters being an influencer role rather than a decision maker role
In conjunction with the items above, I'd like to fork that into a similar problem which is Scrum Masters not being a decision maker. Why is this important? When you look at any organisation, people are busy and they have little or no room for investing in improvements. Their focus is completely different, usually delivery. That is one of the biggest problems of a Scrum Master, finding people's time or motivating them for change. However this is slow, not always easy, and most of the time pushed back. Middle managers who retain at least some decision making power:
May not feel safe with the changes that are suggested and are happier with the status quo
May have a totally different agenda and focus
May not want to take risks and possibly fail
May see the change as someone else's job
So Scrum Masters without authority have to deal with a number of those people in the organisation and their managers to drive change. How easy that would be? (I've been there many many times).
Scrum Masters trapped in their own silos
Lot of organisations tried to map their existing roles and functions to a so-called Agile organisation without changing the reporting lines or the hierarchy. Some created virtual x-functional teams, some touched the organisation but failed to change the management layer or the management practises/culture. How did this affect Scrum Masters? Due to problem 1 and 2 above, Scrum Masters failed to influence organizational changes or drive them. Changing an organisation requires very top-level support and key decisions. Scrum Masters not only couldn't influence those but were also trapped in their own silos or failed to change the silo culture in their organisations. People reporting to different managers with different objectives and priorities were in continuous conflict. The hardest changes are horizontal, i.e changes that require two or multiple functions to take actions. Sometimes merging those under the same manager would make things much easier of course but it may not always be the case and Scrum Masters have to deal with those cross departmental impediments/conflicts. That is a good potential for a Scrum Master to add value to the organisation, however they may be confined or constrained by the organisation depending on the culture of the organisation and managers' openness and support. Some managers may not want their people to go and interfere with other functions, some feel that as a threat or a risk.
Scrum Masters' Managers attitude/skills
In an organisation you can observe differences in different Scrum Masters due to how their managers position them and their managers' management skills/attitudes. This is a very important factor, among others, that would significantly affect a Scrum Masters performance. A manager being open and willing to change, being open to feedback, being able to control their ego, creating a safe environment, helping on the impediments, supporting an experimentation culture, trusting people, knowing how to delegate, not having a control mindset, etc., can help Scrum Masters to perform. Good Scrum Masters would love to drive change and be motivated with it if supported by their managers. One can ask, isn't it a Scrum Master's job to coach their manager and influence them to align with more modern management practises? The answer is yes but this is most of the time very hard and slow. Scrum Masters should never give up but if they are not so experienced this could burn them out.
Summary
So to summarize, the industry is sometimes just finger pointing at Scrum Masters for Agile transformations failures. In this blog post I wanted to give you some insights and factors on how Scrum Master performance can heavily be affected by external factors. Depending on the Scrum Master's skills and experience, these factors can be tackled to an extent but mostly they are very hard and require lots of proactive leadership support and organisational level support.
About me
To connect, here is my LinkedIn and my company web site:
Alper Gurbuz
https://www.linkedin.com/in/alpergurbuz/