Hierarchies remove scaling properties in Agile Software projects

There is a lot of interest in scaling Agile Software Development. And that is a good thing. Software projects of all sizes benefit from what we have learned over the years about Agile Software Development.

Many frameworks have been developed to help us implement Agile at scale. We have: SAFe, DAD, Large-scale Scrum, etc. I am also aware of other models for scaled Agile development in specific industries, and those efforts go beyond what the frameworks above discuss or tackle.

However, scaling as a problem is neither a software nor an Agile topic. Humanity has been scaling its activities for millennia, and very successfully at that. The Pyramids in Egypt, the Panama Canal in central America, the immense railways all over the world, the Airbus A380, etc.

All of these scaling efforts share some commonalities with software and among each other, but they are also very different. I’d like to focus on one particular aspect of scaling that has a huge impact on software development: communication.

The key to scaling software development

We’ve all heard countless accounts of projects gone wrong because of lack (inadequate, or just plain bad) communication. And typically, these problems grow with the size of the team. Communication is a major challenge in scaling any human endeavor, and especially one – like software – that so heavily depends on successful communication patterns.

In my own work in scaling software development I’ve focused on communication networks. In fact, I believe that scaling software development is first an exercise in understanding communication networks. Without understanding the existing and necessary communication networks in large projects we will not be able to help those project adapt. In many projects, a different approach is used: hierarchical management with strict (and non-adaptable) communication paths. This approach effectively reduces the adaptability and resilience in software projects.

Scaling software development is first and foremost an exercise in understanding communication networks.

Even if hierarchies can successfully scale projects where communication needs are known in advance (like building a railway network for example), hierarchies are very ineffective at handling adaptive communication needs. Hierarchies slow communication down to a manageable speed (manageable for those at the top), and reduce the amount of information transferred upwards (managers filter what is important – according to their own view).

In a software project those properties of hierarchy-bound communication networks restrict valuable information from reaching stakeholders. As a consequence one can say that hierarchies remove scaling properties from software development. Hierarchical communication networks restrict information reach without concern for those who would benefit from that information because the goal is to “streamline” communication so that it adheres to the hierarchy.

In software development, one must constantly map, develop and re-invent the communication networks to allow for the right information to reach the relevant stakeholders at all times. Hence, the role of project management in scaled agile projects is to curate communication networks: map, intervene, document, and experiment with communication networks by involving the stakeholders.

Scaling agile software development is – in its essential form – a work of developing and evolving communication networks.

A special thank you note to Esko Kilpi and Clay Shirky for the inspiration for this post through their writings on organizational patterns and value networks in organizations.

Picture credit: John Hammink, follow him on twitter

6 thoughts on “Hierarchies remove scaling properties in Agile Software projects

  1. Nice post.

    If you listen to Dave Snowden’s work, he will tell you that the more beurocratic the hierarchy, the denser the social network. Dense social networks make for effective communication.

    From my experience, no one communicates via the hierarchy. The only time the hierarchy is used, is to resolve a dispute. Effective managers know it is best to resolve disputes themselves rather than involve their boss. Once changes are agreed that need more senior levels of ratification, both sides escalate it up the hierarchy.

    The more subtle and insidious impact of hierarchies is the way they distort the design of things. i.e. Conway’s law.

    My preference is to assign responsibility to people for delivery. i.e. The outcome or the goal, and leave it up to them to work out their communication paths.

    Most consultancies train their staff to create a stakeholder map. This maps the formal routes of authority. Effective managers drink coffee and gossip which allows them to navigate the social graph.

    My conclusion is that the formal hierarchy is irrelevant to scaling agile, but it does have an impact on design. Of more importance is the level of respect for the hierarchy.

    • Existing hierarchy can hinder performance due to enforced restriction on delegation, information sharing, code owning, etc. More than that, hierarchical mindset presupposes people for some stuff (silo mentality, elitism, information hiding) and not to do some other stuff (free information sharing, common context building and goal setting, collective artifact ownership, etc).

  2. Hi Chris,

    Firstly, I still need to read your book The Committment. Hopefully you’re the same person. It may have supplied the clarity I’m seeking.

    In which way does respect for hierarchy effect the scaling? e.g. does ignoring, for want of a better word, hierarchy work better. If so why bother with them?



  3. “Hence, the role of project management in scaled agile projects is to curate communication networks: map, intervene, document, and experiment with communication networks by involving the stakeholders.”

    Hmm… what to do with managers, who think that their role is to exercise command & control duties in military style?

  4. If we consider the design phase (all planning before building anything) of the pyramids, Panama Canal, or A380, how many people were involved in those?

    Does it make sense to try to scale up a design activity? Or would it be more beneficial to program (i.e. design) lots of small good systems that interact with each other? Greetings to Mr. Conway.

Comments are closed.