On the positive benefits of lateral flow

Productivity and stability are the usual excuses for turning creative human beings into mechanical adjuncts to production processes. - Donella H. Meadows, Thinking in Systems: A Primer

Highly valuing stable product teams is a symptom of organisational dysfunction.

Coming to this realisation earlier this year was something of a surprise to me. I've had good experiences working in stable product teams, where the members get to know their product, domain, and one another really well and the work just seems to flow.

It shouldn't have been that big a surprise though. Whenever I've talked about how much I value a stable product team there has been this nagging doubt. Often, I've been voicing this principle in the context of interviewing for a new job. If I value stability so much, why am I even at that interview? Well, putting aside financial concerns, it's usually because I've found that a change does me good.

Change reenergises me. The learning dial gets turned up. New product, new system, new people. I am usually resistant to this, as people often are, but once I'm over that threshold I remember how reinvigorating it usually is, and I don't think I'm alone in that.

For people to benefit from this, they'd need to be changing teams once in a while. In a big enough org, this would mean that team changes would have to be happening a lot. Surely, given what we know of the importance of stable teams, this would be terrible?

Well, what we know may be wrong. In an excellent article from 2017, Doc Norton explains how the Forming / Storming / Norming / Performing model devised by Bruce Tuckman that led to the valuing of stable teams, was not backed up by subsequent studies.

Read the article, it's worth it, but the key takeaway from it for me was that the stable team model works because it reduces context switching in organisations that utilise resources to 100%.

Utilising resources to 100% in product development is a bad idea.

It will slow you down. It will increase anxiety in team members, potentially leading to burn-out, and almost certainly leading to churn. And if you're prioritising stable teams in this context, that churn will likely be out of the organisation, especially for members with highly transferrable skills like engineers and analysts.

We need to be careful not to fall into the trap though of confusing having highly transferable skills with being entirely fungible. Domain knowledge, system knowledge, and relationships within a team are all still valuable. But they shouldn't be sacrosanct, and any (perceived or real) loss in them can both be mitigated through good practices and amply compensated for by the benefits that allowing a bit more lateral flow across the org will bring.

In many ways, allowing greater freedom of movement, with the control in the hands of the individuals and teams, acknowledges that non-fungibility much better than either tightly drawing borders around a team or a top-down reallocation process both of which tend to reduce people to numbers.

The cases against

"Loss" of Domain Knowledge

Mitigation

To be an effective product developer you need to know the product. Gaining domain knowledge, really understanding a domain, takes time. There's no better teacher than experience, but there many ways to optimise learning environments.

The most obvious, and boring, one is documentation. Up-to-date, clear, concise documentation pitched at the right level. It's easier said than done, but putting in the effort yields real rewards in almost any context, and it's extremely valuable in onboarding. Write ADRs, runbooks, weeknotes, etc. etc. and make all of it accessible.

Another is the application of Domain-Driven Design. Again, easier said than done, especially for an in-flight system, but directly representing the terminology and rules of the business in the software in a highly readable manner is going to help foster domain understanding in newcomers and have them able to implement requirements much faster than if they have to translate back-and-forth between business and system concepts.

Benefits

If an engineer or analyst moves between teams in an organisation, their domain knowledge is not so much being lost as transferred.

When someone comes into your team, they are bringing knowledge from another domain with them. Given you're in the same organisation, chances are it isn't such a hard boundary to cross - there will likely be shared concepts. Possibly there was even direct interaction between the products. Possibly there should be direct interaction between the products, but the idea was never conceived because the boundaries were too impermeable and so nobody recognised the opportunity. Cross-pollination to generate new ideas is made a little more likely.

The reverse is also true - "losing" someone to another team opens the same kinds of opportunity.

Given that some of the same concepts are likely to exist between products, someone with experience of working with them might also know some of the gotchas and how they can be dealt with. Conversely, a fresh pair of eyes can also be transformative - asking questions that those of us who have worked with the same ideas day-in-day-out no longer think to.

"Loss" of System Knowledge

Mitigation

The importance of documentation has already been mentioned, and it applies equally here. It's still important to pitch the documentation at the right level - you do see people want to document individual classes and methods on a wiki, and generate diagrams of classes and their interactions. You probably don't want to do that, and instead concentrate on making the code speak for itself more (that said, since switching to data engineering I've seen a lot of value in documentation, especially visualisations, auto-generated from data pipeline source code).

A lot of mitigation of this is applying good engineering practices.

The aforementioned DDD helps because it influences how a codebase is structured, either naïvely because it just makes sense to group domain types and functions and separate them from more technical concerns or because a pattern like hexagonal architecture / ports and adapters is consciously followed.

XP practices like TDD/BDD and CI/CD when followed well should lead to code which is easier to change and readably specified, both of which will be very helpful to a newcomer.

Pairing and ensemble programming also help to spread knowledge (and ownership) and are great for onboarding.

Loose standards or recommendations around tech stacks across the org can also help here, but care should be taken not to lose the benefits of self-organisation. Make these too rigid and innovation can suffer, and diversity in technical implementation can also be good for security, reducing blast areas. It might also remove some of the motivation around team changing for engineers looking to broaden their technical horizons.

Benefits

You could pretty much take the earlier section on the benefits of Domain Knowledge transferring and apply s/domain/system/gi...

If somebody leaves your team for one where the system directly interacts with yours, the knowledge they can take with them can help to make that interaction smoother, and vice versa.

The fresh pair of eyes equally applies - opportunities for improvements and technical collaborations with other teams might be more likely to be seen by a newcomer from another team.

It's also potentially helpful in spreading practices around an org. If your org is going through some kind of transformation(s) that would benefit from more people understanding some key concepts, then having more information flowing alongside people between teams should help it spread, and prevent a team being stuck with just its own interpretation of those ideas.

"Loss" of relationships

Mitigation

So-called soft skills are often underrated, and this seems like one of the hardest of these three aspects to deal with. Trust, and camaraderie in general, is not always quickly built, and it feels great. Meetings are less of a chore, it facilitates reaching consensus on decisions, and makes dealing with the hard times easier.

Don't make the team boundaries too permeable. There's a balance to be struck. If nobody ever gets the chance to make any personal connections, that's not going to be good for retention either, so don't turn it into a free-for-all with entire teams switching it up every month.

Pay careful and appropriate attention to team spirit. There are lots of ways to do this, and no single right one. On one team I was leading once I took inspiration from the idea of the Fika, and we set aside an hour each week for it. Team bonding "reset" days can also be productive. Whatever you do, the trick is not to make it too forced and not to make people uncomfortable. Don't use ice-breakers that get too personal, don't encroach on people's personal time, don't make it about drinking. Psychological safety is paramount.

Don't be too insular. Teams that are more open in the first place are more likely to find people who want to join them. It's great when teams periodically showcase their work, whether it's the product work itself or engineers sharing their learnings. I've also been lucky enough to be on a team where the nominally-team-internal slack channel was open to all, and we had a nice enough team spirit that people from other teams just came and hung out with us.

Keep in touch. Although you can't invite all your alumni to team events, especially if team hopping becomes more common, there may be occasions where it's possible. This is easier when colocated as it can be almost completely natural to have a coffee or lunch with a former colleague, but requires a bit more effort when working remotely.

Benefits

The relationship isn't really lost - one end of it is now in another place. There might be a lower frequency of contact, but it's still an open channel to another team that might not have otherwise existed. Again, this is an opportunity rather than a threat; cross-pollination of ideas, and easing of communication are made more likely. Networks can have powerful effects.

Also, it feels good to make new friends! And it's good to feel good, and have people who feel good around you.

It's also good for personal development, especially for juniors, to be exposed to a wide range of viewpoints and ideas, and to make more connections. If people don't feel like they're getting this, they will make it happen by going elsewhere.

How do you do this?

1. Don't run teams at 100% utilisation

How you achieve this depends on your org. It might have to start from the top. It might involve OKRs, if that's how you roll.

In a self-organising team, there should be some scope for the team to build in some slack, though realistically there is always going to be external stakeholder pressure. Educate them - tell them the benefits of cutting a bit of slack, give them something to read about it. There are good articles out there about this, find one you think will help the people who need to read it!

Don't do scrum. Seriously. Don't do that thing of working out the velocity of a previous sprint, estimating story points, using this highly misleading data to work out what you can fit in the next two weeks based on headcount, then padding out the time left over to 100% with small (typically tech debt) tasks from the backlog. It's absurd on many levels, and a complete flow killer. It's done with good intentions, I'm sure, but we all know where those often lead.

2. Just do it

It's easy to criticise it as a circular answer (and a cliché), but in my experience, the best way of making sure things get done is to do them.

Don't water down the concept. Don't form a cross-team committee to agree on what it really means, and how to implement it, in your org. That's almost the surest way to prevent it from happening in a meaningful and effective way.

Do start small. Depending on your org structure and how self-organising you can be, it could be an agreement between two teams, or an experiment run within a department or domain. I wouldn't rule out starting big, if that is possible within your org, but it is more likely you will get there by making continual small steps and setting an example.

Don't wait until everything that could facilitate it is in place, or you can be paralysed waiting for the perfect moment that will never come. Hiring somebody is a good way of finding out where the real holes are in your onboarding process, which you then fill in. The same principle applies here - uncover your requirements by doing, not speculating. Experiment, gather data, iterate + share your results.

Do explicitly reflect on this process. This can be on a cross-team level, but equally importantly within a team - it's both how well the individual teams are handling this and how they are connected that will make it work at scale. Teams should share with one another what they're doing, what works and doesn't, to help one another out, but only really share "up" if policies and support from those levels would help. A functioning anarchy where every group and individual is involved and has agency will always be healthier, more effective and have a better sense of shared purpose than a top-down hierarchy can ever provide.

Don't give in to geopolitical resistance! Product teams and matrix org domains can be highly territorial, deliberately making migration as difficult as possible, and quite possibly citing Tuckman in their defense. Find the more open-minded product owners and engineering managers to convince that the experiment is worthwhile, remembering that they are individuals who will need convincing in different ways.

If you can break down territorialism and make onboarding easier, then it potentially opens up a way to deal with another mild absurdity you will occasionally encounter - when numbers of people aren't allocated according to overarching need. All teams have plenty of work for their members to do, but the importance of the output of each team will ebb and flow over time. It often happens that a team has a disproportionately large number of members because at an earlier time it needed it and it simply remained that way (because teams need to be stable, remember!) while other teams can be struggling despite being more mission-critical at that time. Not enough product owners admit this, but it can be actively harmful to have too many engineers and too much work-in-progress.

Final notes

Disorderly, mixed-up borders are sources of diversity and creativity - Donella H. Meadows, Thinking in Systems: A Primer

Cultures are enriched and strengthened by migration, as are individuals whether they are the immigrant or the welcoming host. An isolated population is not going to be a healthy one in the long term, even if in the near term it might seem happy and productive.

Long lived, stable product teams are ultimately a myth. They always break down at some point, frequently with a mass exodus as disillusioned talent looks elsewhere for development inspiring others to ask the same questions (and sometimes directly taking them along). Such booms and busts can also be exploited to reenergise teams and organisations, but it would be a very risky strategy to follow deliberately.

Building hard borders and restricting movement is an optimisation for the short term, and to me that's incompatible with product thinking. An organisation where a good level of lateral flow is occurring will naturally be healthier as the constituent sub-organisations will have to be in a good, or improving, state to permit it.