Current State and Future Prospects of Scrum and Agile Development in Japan

(Japan)

March 8, 2023

Since I became a Professional Scrum Trainer in November, I have been asked a lot about the current state, and the future perspectives, for Scrum and agile development in Japan, where I’ve worked for the last six years. This interest from people around me encouraged me to do some research and thinking on the subject, which I have summarized in this article.（日本語版はこちら）

The state of Scrum and agile development

In 1986, professors Takeuchi and Nonaka published the famous white paper entitled, "The New New Product Development Game" (¹).The paper presents successful examples of new product development projects by Japanese companies and lays out some of the characteristics these projects had in common. Those characteristics, including "self-organizing teams" and "overlapping development phases" among others, are very much aligned with the principles underlying Scrum and agile development. So much so that many consider the paper to be part of the roots of Scrum. More recently, examples of successful Scrum adoptions in Japan have continued to emerge (² ³ ⁴ ⁵). The local communities and events also grow in size every year.

Yet, Scrum and the principles of agile development are not as common in Japan as they are in other parts of the world. It doesn’t matter whether you look at the available research on the progress of DX (⁶), case studies, the job market, the number of certification holders, public training or conferences; the evidence tells the same story. If you compare these numbers with countries like the US, Germany, France or India, you will come to the same conclusion. None of these are great indicators in isolation, but they all have one point in common: they indicate that Scrum and agile development is just not happening in Japan to the extent that it is elsewhere. Why is that?

Adoption of different methods and practices in Japan ("DX White-Paper Executive Summary" Information-Technology Promotion Agency p11)

Japanese culture

Japanese businesses have a strong desire to avoid risk and minimize unexpected events. Living in Japan, I am partly amused, partly puzzled by the amount of warnings and rules I encounter everyday that are designed to reduce even the smallest risks. There are instructions for getting on an elevator, warning signs for the toilet flush, and if you dare touch your smartphone while on a treadmill at the gym, someone will probably come and ask you to stop because you’re putting yourself in danger.

The Japanese also value and take pride in careful planning and flawless execution. This may explain why Japanese trains are always on time, why daily life is generally convenient, and even some of the success that many Japanese manufacturers had over the last few decades. But these mindsets don’t help when the nature of the problem you’re dealing with is such that you need the ability to experiment, fail safely, and learn.

At the same time, other aspects of Japanese culture turn out to be real assets.

In Japan, everyone learns the value of teamwork and the joy of performing and competing as a team from a young age. The Japanese people are also polite. They are respectful in conversations and are good listeners. I think it is fair to say that they are generally better “team players”. This is an advantage because complex product development usually requires close, trust-based collaboration between people with different skills and perspectives.

They are also reliable and they care about meeting the commitments that they make. This is important because agile teams are self-managed. They don’t rely so much on pressure from the top to keep each other accountable and deliver. This pressure instead must come from within. Teams that have the right degree of peer-pressure to keep each other accountable are less likely to suffer from collective under performance, and to fall back to top-down control later.

When Scrum or an agile mindset already has “a foot in the door” of a Japanese team or organization, I have found that these traits truly fuel agility, and support performance.

Lastly, I believe that Japan, like every other culture, has specificities that are neither clearly good nor clearly bad when it comes to embracing agility, but that need to be understood and accounted for when leading change. For instance, there is a business culture of consensus-based decisions and nemawashi (⁷). Creating consensus is obviously time-consuming and companies that over-do it will suffer from lower performance. On the other hand, in Japan, once you win the heart of your key stakeholders (employees, managers or customers alike), decisions become easier to implement and are more likely to stick.

I experienced that first hand when I was consulting for a big Japanese insurance company. My colleagues had spent months discussing the Why and the How of standing up an agile team to work on a new important market opportunity. One day, the decision was made to go for it. A couple weeks later, the Scrum team was staffed with competent people from Sales, Sales Planning and IT, and we even had a team room with brand new furniture.

All in all, culture is certainly playing a role, but I don’t think it fully explains why Japan lags behind the rest of the world with Scrum and agile adoption, and I believe other factors are at play.

Software engineering in Japan

Outside Japan, the adoption of Scrum and the underlying principles of agile development mostly happened in two phases:

Phase 1: software engineers and their software developing organizations embrace agility in order to deal with the complex nature of the work they do, respond to rapidly changing customer expectations, and leverage the flexibility that the technologies they use provide.
Phase 2: people from different industries are inspired by, and learn from what happens in the software world. People from different professions get pulled into increasingly cross-functional teams. Agility becomes an org-wide priority.

I believe that Japan is in a sense stuck at Phase 1. As Tim Romero, Head of Google for Startups Japan puts it (⁸), "Japan’s software industry in the 80s and 90s remained much like it was in the mainframe area. The software had to be just good enough for the client to sign off on it, and since they were largely captive clients unable to look outside their keiretsu group for support, that was a very low bar indeed". In other words, for most people in Japan, software development was and remained an exercise in box checking. Software development did not get the same kind of focus it got elsewhere, and software engineers did not get the same kind of respect they earned in the West.

Therefore my sense is that for agility to break through, Japan’s best chance may not be the typical “software → everything else” path. Rather, I believe that the initial leadership and role modeling is likely to come from somewhere else. I could see it coming from very different fields such as manufacturing, or marketing for instance. Policy makers and other thought leaders may also play a role in shaping it.

What happens next?

Agility is an essential attribute of successful organizations. It’s not the only attribute that matters, and not all organizations need agility to the same extent. But if an organization is doing something even remotely complex, where technology is involved and/or customer needs evolve over time, it will benefit greatly from the ability to make quick decisions, to release early and often, to learn, and to adapt. This, to me, obviously applies to Japanese businesses, and I am optimistic that it is only a matter of leadership, momentum and timing before agility starts to snowball.

When I share my relative optimism around me, I hear mostly two counter-arguments. The first one is that the Japanese people are too risk-averse to ever dare embrace agility. The second is that the Japanese people are unable (or at least incredibly slow) to change. I find both of these arguments unconvincing…

Japan’s risk-aversion a deal-breaker for agility?

When Japanese people go to Kaiten-Sushi restaurants, they order one or two plates, then wait a couple minutes, then the plates come, they eat, then they consider what to do next. They may order more of the same, order something different, or ask for the check. Compared to traditional restaurants, this approach minimizes both the risk of waste and the risk of unhappy customers.

Conveyor belt at a Kaiten-Sushi restaurant in Japan

When we develop something the agile way, we do essentially the same. We make small investments, and we keep our options open. This is how risk is controlled in an agile environment and it is pretty effective.

Therefore, I don’t think we have much of a dilemma between avoiding risk and embracing agility. I think the problems we have are problems of misunderstandings surrounding agile development, and old habits. And I believe these problems can be overcome.

Is Japan slow to change?

In the 1850s, Japan suddenly (⁹) came to realize it was technologically under-developed compared to Western nations. In the few decades that followed, Japan went through a period of dramatic change and innovation called the Meiji restoration, changing pretty much everything (the legal system, the educational system, the government, etc.), boosting its economy (¹⁰), and reshaping its culture along the way.

One of Commodore Oliver Hazard Perry's so-called "Black Ships" (Massachusetts Institute of Technology)

After World War two, Japan was devastated, occupied, and the Japanese people suffered from starvation. But, once again, this palpable crisis triggered adaptation. In a few decades only, Japan got back up during a period known as the Japanese economic miracle (¹¹). It became a powerful economy and generated many innovations the world still benefits from today.

Very few countries have managed to reinvent themselves so dramatically in such short periods of time.

One of my clients in Japan, a big company in a highly regulated industry, started adopting Scrum five years ago. Over a period of a few months only, they selected people from different parts of the organization to become Scrum Masters. Many of them had never practiced Scrum or agile development before. Most of them were in their 40s. Yet, many of them have now turned into awesome Scrum Masters and role models of servant leadership and the agile mindset.

Whether I look at Japanese history, or simply around me at the amazing people I work with and the many changes they’ve been able to embrace in their careers, I refuse to conclude that Japan is simply “slow to change”.

Yet, the above examples suggest a pattern by which strong disruptive forces are necessary to create both a sufficiently compelling case, and a momentum for change. In other words, Japan may need some kind of “wake up call” before agile ways of working have a chance of becoming mainstream. And we, agile practitioners and everyone who cares about the future of work and product development in Japan, have a role to play in making sure it happens sooner rather than later.

How can we help?

In this article, we have explored several factors that may explain why agility in general, and Scrum adoption in particular, remain marginal in Japan. However, I made the case that none of these factors were deal-breakers. Instead, I argued that Japan had great assets, and that with a good mix of leadership, momentum and timing, Japan would also learn to embrace agility.

If I am correct, Japan will need experimenters from different fields, who can then turn into inspiring leaders for others around them. It will also need teachers and coaches.

As the only Japanese speaking Professional Scrum Trainer at this point, I believe I can contribute to shaping this better future for work, and product development in Japan. My contribution will be limited, but I have decided to focus on a few things I am passionate about:

Providing quality learning opportunities, including a lot of pro-bono
Getting better at making a compelling case for agility and Scrum, for various audiences
Helping grow future leaders and creating space for them

But this is only a fraction of what needs to happen and I would like to offer a few questions for all of you to ponder, since you’ve made it this far…

In what ways would you like to get involved in authoring the next chapter in Japan's amazing growth story?
What industries or professions do you see being elevated by Scrum and agile ways of working?
How can we support whatever you're committed to happening sooner rather than later for Japan?
Who do you know that might care about Japan's future and how can we engage with them on these questions?

If you feel like you also have a role to play in this, or simply if this article resonated with you, let’s please connect (Twitter, LinkedIn, www.agorax.jp), and grow the Japanese community together!