Recently, we hosted an “ask-me-anything” session with Jimmy Janlén. Jimmy is an Agile coach & trainer based in Stockholm. Before starting his own company “Agile With Jimmy”, he worked at Crisp and supported organizations like Spotify, Casumo, KRY in their quest to increase agility. Jimmy is the author of the book “96 Visualization Examples — a toolkit for the Agile coach”. And has created tons of supporting materials like card decks, papers, videos, blog posts, and much more!
During the one hour meetup, we covered a wide variety of questions, for example:
- What are the facts & myths about the Spotify model?
- What experiences from your time at Spotify do you cherish the most?
- If you start in a new organization as an Agile Coach, how & where do you start?
- What made you decide to become an entrepreneur, and what is the added value of having your own company compared to being part of an organization?
- Imagine it’s the year 2050, Agile has fully lived up to its promise, what have we started doing differently to make this success possible?
- Where did the idea to create the 96 Visualization Example come from? Any favorites you often use yourselves?
Check the video to learn how these questions were answered:
In this blog post, we focus on one specific question that was raised:
“How to measure the success of an Agile Coach or Scrum Master?”
Spoiler-alert: the magic silver-bullet answer didn’t emerge. We did have a very interesting conversation. In short: it’s very difficult to measure because even if a team or organization is successful, are they successful because of your contribution, or despite it?
Peter Wynands, a member of our Patreon community puts it as: “Indeed this is a tricky question. The success (or failures) of an Agile Coach or Scrum Master is not that straightforward to measure, and any causality with the success of a team you coach is difficult to prove. One obvious question that comes to mind is how pleased the client is with the coaching, or you could even measure it with some kind of NPS (Net Promotor Score). But I like the quote of Nanny McPhee: When you need me, but don’t want me, then I must stay. When you want me, but no longer need me, then I have to go.”
Nils Dimmers suggests to rate yourself against the feedback you get from your surroundings: “If the team and management are very happy with the development of the team and how the Scrum Master supports that, you’re probably doing a good job”. And according to Guillaume Delépine it’s unfortunate that sometimes you only realize the impact of an Agile Coach or Scrum Master when they’re going. It can take weeks or months before you can see things starting to collapse or improve. Something they’ve been involved with, in a good (or not so good) way.
If you receive the question by management on how they can measure your success as an Agile Coach or Scrum Master, Jimmy and Franziska Wiebel offer to explore the following question together:
“If, 6 months from now, you would decide to (not) extend my contract, what would you base your decision on? What is it, you would like to see changed?”
The ideas from Jimmy and the members of our community are tremendously valuable. In the remainder of this blog post, I’ll include my personal findings as well. I’ll focus mostly on how to measure the success of a Scrum Master because that’s closest to my own experiences. Although I guess the terms Scrum Master or Agile Coach are easily interchangeable in this context.
The Purpose Of A Scrum Master
Before we dig deeper into the metrics by which you can measure the success of a Scrum Master, let’s start by clarifying their purpose first. In short, the purpose of Scrum Masters is to create transparency around the ability of teams to deliver valuable outcomes to their stakeholders, and the impediments that get in the way.
As we describe in our paper “Scrum — A Framework To Reduce Risk And Deliver Value Sooner”, it’s up to the Scrum Master to include the perspective of empirical process control and the quality by which transparency, inspection, and adaptation are taking shape in and around the Scrum Team. The Scrum Master is there to make the elements of the Scrum framework come to life in the team and the broader organization. Rather than drawing attention to themselves, Scrum Masters help others be as effective as possible. They do not manage or lead the team by telling them what to do or how to do it. The only area where Scrum Masters should take a strong position is when decisions are made that adversely impact the empirical process or the degree to which members of the team feel safe to take interpersonal risks.
Indicators Of Success
This description of the purpose of a Scrum Master hopefully makes sense, but I can imagine it’s still very abstract. What does it actually mean? What is it that you can see, hear, and notice about a successful Scrum Master? To make it more specific and tangible, we’ll share some indicators of success. These are examples of our book the “Zombie Scrum Survival Guide”, we wrote together with Johannes Schartau. If a Scrum Master is successful, what you’ll probably see happening is:
- Scrum Teams are continuously focused on how effective their work is — that is, how much value it delivers to their stakeholders and the organization they work for. To do so, they have a close and frequent collaboration with their actual stakeholders. Meaning the people that have an actual stake in the product.
- Scrum Teams work in a Sprint-based rhythm where every iteration results in a new version of the product, that can be potentially released. It’s in such a state that it can be deployed with the proverbial press of a button. Whether or not to release is up to the Product Owner. Either way, the work the team put into getting everything release-ready wasn’t wasted.
- Scrum Teams use the Sprint Retrospective to reflect on their ability to create a high-quality, releasable version of their product for stakeholders. They use objective data to support this reflection, ranging from cycle time to bug count. Scrum Masters support this reflection by keeping the focus on delivering valuable outcomes and helping teams to navigate the inevitable conflicts that surface while doing so.
- Scrum Teams have full autonomy to make decisions over the product and how, when, and by whom work for that product is done. They are aware of the impediments that make it hard to ship fast, to build what stakeholders need, and to improve continuously. They possess the self-organizing and self-managing capabilities to resolve these impediments and to drive change across the organization.
Measure The Smell Of The Place
Another way to get an indication of the success of a Scrum Master is to measure the “smell of the place”. This concept was coined by Sumantra Goshal, professor at the London Business School. It’s a way to think about how your work environment shapes your behavior. Organizations often have different “smells” to them, and all those smells work together to drive how people behave. This leads to one interesting opportunity for change: don’t change the people, but change the environment.
In a talk at the World Economic Forum, Ghoshal described how some organizations smell like a hot, gritty day in the concrete jungle of a big city that is overrun by zombies (we call this “Zombie Land”). They drain your energy and make you lazy and sluggish. But in other organizations, the atmosphere is more like a green forest at the start of Spring that energizes you and boosts your creativity (we call this “Autonomy City”).
A powerful tool to measure if you’re team is stuck in Zombie Land or flourishes in Autonomy City, is the Scrum Culture Index. It helps create transparency on how your work environment shapes your behavior as individuals and as a team. And it turns this into a (simple) metric that you can track, and see if you’re heading in the right direction.
If a Scrum Team is stuck in Zombie Land, this doesn’t necessarily mean the Scrum Master is doing a bad job. Changing the environment of the Scrum Team is tremendously difficult. It means changing the governance, procedures, processes, and culture of an organization. It’s about challenging the status quo and the mindset of “this is simply how we do things around here…”. But a Scrum Master can set the desired change in motion. (S)he can slowly change the smell within the team. Together with other Scrum Masters and the supporting organization, they can improve the smell of the entire organization. If you’re interested to create a Scrum Culture Index with your team, check this do-it-yourself workshop we designed.
Other Indicators Of Success
So far, besides the Scrum Culture Index, we shared five indicators to measure the success of a Scrum Master. Put differently, these indicators are related to “build what stakeholders need”, “ship it fast”, “improve continuously”, and “self-organize to resolve impediments”. These are some of the topics of our Scrum Team Survey. With this survey, we do measure other topics as well. Topics that I personally consider the most important. Improvement in these areas is something I perceive as a huge success! I’m talking about improving psychological safety, team happiness, team morale, trust, and the way the team deals with conflict. Topics that are all connected, intertwined, and influenced by one another.
Let’s start with psychological safety. Psychological safety is one of the most important contributors to successful teams. Amy Edmondson defines psychological safety as “a shared belief about the consequences of interpersonal risk-taking”. It’s not at odds with having tough conversations, it is what allows teams to have tough conversations. Psychological safety is not about always agreeing with each other, or about giving compliments all the time. It is much more about encouraging people to speak up when they have worries, uncertainties, or doubts.
Something I used to measure was team happiness. Do people enjoy being part of the team, are they happy to work together, do they like the Scrum events? In short, the focus is on happiness. And if everyone in the team is happy, this should be an important indicator of the success of a Scrum Master, right? Well, maybe not. According to my colleague Christiaan Verwijs, the happiness metric is sub-optimal at best. In the article “Agile Teams: Don’t Use Happiness Metrics, Measure Team Morale”, he explains the pitfalls of measuring only happiness, and why measuring team morale is more valuable. As Christiaan puts it: “I strongly believe that in a cohesive, well-running team, people are willing to go the extra mile even if it makes them (a bit) unhappy for the duration of the task. They will accept this for the greater good.”
Morale does include happiness, but more subtle: Teams with high morale are generally teams with happy individuals. Teams with many unhappy members will probably also have low morale. But not always. So rather than throwing happiness away as a metric, be aware of the narrow view it offers you on how the team is doing.
The empirical process of Scrum only works when there is a basic level of trust. The nature of complex environments in which many Scrum Teams operate is that mistakes will be made and are a necessary result of experimentation. The Scrum Master needs to help create an environment of high trust, so the team members dare to take the necessary risk to experiment and validate assumptions on how to build the product. A Scrum Master can use the five core values of Scrum to encourage trust: openness, courage, focus, commitment, respect.
The final indicator of a successful Scrum Master is when the team members are able to have a constructive disagreement. Conflict is a natural aspect of people working together in a complex domain. Although you can resolve individual conflicts, you can’t resolve conflict entirely. And why should you? Conflict is a good thing when it is navigated correctly. Successful teams have learned how to navigate conflicts in such a way that disagreements don’t escalate.
Closing
I don’t believe there’s a metric you can use to measure the success of a Scrum Master. I do think you can use indicators that give you a sense of how successful the team is. Indicators we shared in this blog post are about the ability of the team to build what stakeholders need, to ship increments fast, to improve continuously, and to self-organize to resolve impediments. Other indicators are team morale, psychological safety, team happiness, trust, and the way the team deals with conflict. The question remains: even if all of these indications are positive, is this because, or despite the role of a Scrum Master? In my personal experiences, I like to believe it’s because of my contribution as a Scrum Master :-)