Branching is a Dysfunction

Hi Ryan,

your link doesn't work for me. Try this one instead: <a href="http://blog.chadalbrecht.com/post/2012/04/29/Is-Branching-a-Dysfunction…">http://blog.chadalbrecht.com/post/2012/04/29/Is-Branching-a-Dysfunction…;.

I think that Branching is not a dysfunction in itself. On the contrary, with large products and a magnitude of teams I believe it to be essential. One customer of me has for example 10 teams, all of which only work with the "Mainline" on a really complex product (hardware involved, four architecture layers, several clients and so on). No branching. It's a hell of chaos, nothing ever being in a "Done" state and having never a shippable increment. This customer needs a huge pile of luck and a "stabilization phase" ranging from 3 to 6 weeks to allow the tests passing green at least once.

As with every tool, it depends on how you use it. With the above customer for example, we are building up a branching infrastructure, that is intended to create shippable increments every day. This works as follows: Every team has it's own branch. Once a feature is "implemented", the team merges it with the development-branch, which gets the most recent version from the mainline every day. If the feature works there (and the corresponding tests are green), it must be merged with the regression system (which is the mainline). Only what can be found there (and working of course!) is considered "Done" and allowed for the Sprint Review. If something fails on that system, it is reverted to the last working version.
It's not yet established in this way, but we believe the merging effort to be smaller than today. Besides, we will finally be doing Scrum - I don't believe you are doing Scrum if you don't deliver a product increment.

Shame on me: Here is the correct link: <a href="http://blog.chadalbrecht.com/post/2012/04/29/Is-Branching-a-Dysfunction…">http://blog.chadalbrecht.com/post/2012/04/29/Is-Branching-a-Dysfunction…;

By the way: I can't edit my post here.

Dominik,
I was in the same place as you for a few years. What finally swung me to the belief that that branching is dysfunctional is being able to compare the behavior on large sets (10+) of Scrum teams. What you will likely find is the “hell of chaos” you feel ongoing is amplified when the teams begin the merge. You branch to create isolation, when you have isolation people behave as if they don’t have to collaborate with other teams. In this isolation and individual Scrum Team can work independently for a few days and all is good. Then when the merge begins the teams find that all that progress (and yes it’s a lot) comes to a screeching halt as they try to realign with the other team’s knowledge and assumptions.

My first experiments involved working with the teams to do more frequent forward integrations. This helps a little. The main problem is that if everyone is doing the same, there is no visibility into work not in the mainline, thus leading to the “big merge” scenario we all know. The other problem is that the behavior is really hard to maintain. Teams feel that they can “get more done” if they delay the merge just a little longer. For many teams, this happens very slowly, a half-day here another day there, until they are waiting a week to do a forward integration. Once this happens, many teams catch it in a retrospective and commit to more frequent forward integrations. After a couple of these cycles, the team gets frustrated and looks for other “easier” solutions. Where they all have ended up is working in the mainline.

After seeing this behavior repeatedly, I wondered “Can a large set of teams work successfully in the mainline together?” What I found was “Yes!” Using the “product is always shippable” idea borrowed from Continuous Delivery, I have found that much of the frustration and waste goes away. I detailed this in my blog post.

http://blog.chadalbrecht.com/post.aspx?id=a719f1cf-c211-4b22-89b0-c3106…

I’d be curious to see where your teams end up. Good Luck!

Chad,

I liked your article

Having been a Sr. Java consultant in the biz for 13 years(observing many contexts) before I did the Scrum Coaching thing, I would contend, in an Agile implementation, that branching is definitely a dysfunction. I can't speak for non-Agile implementations, but my guess is that it is dysfunctional too, for some of the reasons non-Agile approaches are!

Having said that, do you find that well structured feature teams will tend to focus (though not exclusively) on a slice of the product/knowledge in order to protect themselves from a large amount of "knowledge merging" ??

I mean, they almost always have to integrate at some point, but do you find them trying to minimize the integrations to avoid even the "knowledge merging" ?

For instance, assuming a retail web site.

One team focuses on product search.
One team focuses on product display.
One team focuses on cart and check out.
One team focuses on related/companion products.
One team focuses on account profiles.

Do you see this happen too?

I do agree with Chad - branching is a dysfunction.

A friend and former colleague of mine Paul Hammant coined the concept of branching by abstraction.
http://paulhammant.com/blog/branch_by_abstraction.html

Instead of using sources branches you should be branching with abstractions. This is when all the good coding practices come in play like Design Patterns, OCP, DIP, IoC, ... Build the branches into your application and make them easy to change by configuration or other approaches.

Very powerful concept.

Charles,
Good distinction between Agile vs. non-Agile teams on branching. I think if your requirements truly do not change and you can use Design By Contract, then branching may not be dysfunctional.

Yes, I have seen many teams try to "specialize." While this creates domain experts and short term productivity, it also creates knowledge silos and mid to long-term risk. What I advise as a coach is that teams think of themselves as "feature teams." That is, they may work on any area of the code as long as it pertains to the "feature" they are currently implementing. This gives every team a more holistic view of the entire code base. More progressive teams seem to latch onto and embrace it, while more traditional teams seem to shy away. I have not found a good technique for getting more traditional teams to embrace this other than mixing more progressive team members into the team.

Thoughts?

Ralph,
I love Hammant's BBA ideas! They work very well in a single branch, multi-team environment!

I don't think branching is inherently dysfunctional. I do think the greater the time and distance from the shipping line of code the greater the smell. The longer any effort goes without verifying assumptions both with customers and with the rest of the team, the greater the dysfunction. In that way, branching might be thought of as a slippery slope.

I'd present Git, Github, Forks and Pull Requests as example #1 of how branching can support a very productive collaboration effort.

I don't discount Chad's insights though. I've heard similar statements and seen branching be as much and more painful than the original symptoms it was meant to alleviate.

Together with feature toggles, well factored code, and branch by abstraction, I think moderately lived branches can be a useful tool.

Ralph,
This is the first I've read of Branch by Abstraction and I love the ideas as well. While the techniques aren't new, I do like how he's explicitly describing them as an alternative to branching. With that in mind, I might be getting closer to considering branching a dysfunction (shared branches that is).

> Git, Github, Forks and Pull Requests as example #1 of how branching can support a very productive collaboration effort.

They can in the right context. I see a lot of teams using Git (or more specifically, DVCS) in a situation that doesn't require it. A small team that is entirely co-located, and doesn't have a ton of DVCS experience doesn't need DVCS, in my view. CVCS(centralized, like SVN) is probably the better choice in that situation. I've actually seen this happen. The team in question just used Git because they assumed it was the "next Subversion", when in reality, it is a different solution for a different problem space.

I guess I should also add that the team in question had a zillion problems learning how to use the DVCS with all of its pushing/pulling/local/remote repository concepts. In my view, they had selected a tool with much more complexity than they needed. Come to think of It, I saw this happen with 2 different teams.

Found a good relevant link, from an Agile Manifesto author himself:

http://martinfowler.com/bliki/FeatureBranch.html

Comments/feedback welcome.

Team members sharing a codebase don't always collaborate when making changes. If they did then merging those changes would be unnecessary.

Version control systems, by their very nature, will partially compensate for dysfunctions around team collaboration. This doesn't make them a bad thing, but it can easily lead to their misuse. Time and again I've seen DVCS used as a crutch to offset poor agile discipline. Branching is merely an illustration of this problem in extremis. Perhaps at this level a bad smell does indeed become an anti-pattern.