How can growing a startup engineering team slow down production?

The situation

Imagine this situation. A successful startup company has secured funding to grow. They have a great engineering team who have made an amazing product which it's customers love, but the market is changing, new features are needed and the company wants to position itself to take advantage of new opportunities.

The engineering team at a startup may have divided responsibilities, but they can still be considered to be a single team.
- Everyone on the team knows the whole system
- Everyone knows the details of the features and technology.
- Corners have been cut and the reliability isn't rock solid - but this isn't a problem as everyone knows everything and the team is very agile so they can pull those all-nighters to fix those problems when they occur.

There is no architecture or abstract designs, and no documentation or plan detailing how different components of the system interact because at the single team scale there is no need - this stuff will just slow the process of getting our new features to the customers.

It is tempting to say that the team all trusts each other but at this scale trust is not really needed. Everyone understands, reviews and has experience using everyone elses solutions so they know things are right without any trusting required. With 7/8 people in the team it is also not difficult to come to a consensus because everyone is approaching problems from the same base concerns.

Also there is no problem with the corners being cut. This just means that the team is closely connected to the business needs and they haven't overengineered from a reliability and scalability point of view. As they are a small agile team they can up the quality and engineering level when required.

I could draw this team like this:

Essentially the team has responsibilities and is able to change the way it does things (it's practices) to meet the responsibilities. There is nowhere to hide and no responsibilities fall through cracks in the system because there is only one team!

But, back to my imaginary situation, this company is successful and they need to scale up. More features, and more reliability are required. The workload is easily more than this single engineering team can handle so we need to grow it. The founder gives the engineering lead a bigger budget and tells them to go hire some more devs so the company can keep growing.

What happens

At some point someone is going to mention microservices. Splitting the monolith will also be talked about. The company starts hiring and the team is grown. Developers are split into several teams, with senior and junior members, possibly team backlogs, scrums, etc.

This is where some problems start, because it's rare that the responsibilities are clearly divided between the teams and rarer still that these new teams become free to invent their own practices.

The new teams are fundamentally different to the original team that was created. The original team was free to work through the responsibilities it has and develop practices to meet those responsibilities. Those practices could change rapidly to eliminate inefficiencies. This made the original team really good at things like taking advantage of new technologies. It was also incentivised to avoid creating unnecessary and overcomplicated solutions because it would be stuck with dealing them.

In short the original team was:
- closely coupled to the business need (it's responsibilities)
- free to use it's talents to meet the need (it's practices)
- Had clear ownership of it's product

The new teams may have been asked to work on only one part of the product. They may have been assigned and given ownership of this part or (worse) the idea is that they visit this part then move on. This means the team is further down the food chain when it comes to business functionality. They must also use the practices already established by the company. If they didn't it would be chaos!

The new teams:
- Must use established practices
- Must input into some kind of bureaucracy to change/invent new practices
- Can only see part of the functional picture

Practices get documented and enforced

Now that there are new teams in the mix they need to know how things are done. Specific practices are documented and process established. The company wiki has pages like:

  • This is how you deploy something
  • This is how you must setup the CI/CD pipeline
  • Here are the security guidelines
  • Here is a template every microservice must follow

Changing any practice is harder since it is followed by many more people. Small value improvements to practices stop happening. Conversation start to go something like:
- Developer: Let's add an automated performance test to the pipeline, it makes sense for my product A
- Lead: Yes but it doesn't make sense for products B/C
- Developer: I don't know about that I only work on product A
- Lead: Well we can't have product A following a different process. That would be impossible to manage.

Other things like leveraging new technologies and pivoting functionality become harder due to the organisational challenges and development rate slows.

In this structure there is enough frustration to go around:

Role What they are thinking
Developer How do I change or improve this practice?
Developer How can I invent a new practice to cover this new situation?
Developer The practice I am being asked to follow is just making things harder
Lead Developer I am not sure what I am responsible for
Senior Developer There are too many voices
Senior Developer I am stressed trying to keep tabs on everything
Senior Developer Why are the practices not being followed?

Without establishing ownership, clear lines of responsibility and a method for incremental improvement at all levels the organisation is in danger of hitting a feature paralysis.

So what's going to happen?

Democracy and large scale consensus is too slow so what's going to happen?

  • Authority structures are going to develop - Senior developers will be in the clique and make the design choices
  • Senior developers will make the wrong choices - they don't know all the details as it's too much for one person to keep in their head
  • Status quo will always be favoured due to the huge effort of making a change
  • New tech will be harder to leverage
  • Where opinions are sought solutions will tend towards the lowest common denominator as that's what can be agreed on
  • Developer frustration will be high as they are forced to put up with inappropriately applied practices and convincing people to change is hard
  • Junior develops won't increase skill sets as they have to deal with the grunt work this structure inevitably produces
  • Without knowing who is responsible some things will be done twice
  • Things will start falling through the cracks because everyone thought someone else was responsible
  • All this leads to less features delivered to customers.

TLDR

Small startup teams excell at leveraging technology and being efficient. As companies grow developer productivity suffers when the central responsibilities of security, efficiency, resilience etc. have been abstracted away from the teams. The only way they can be met in this organisational structure is if they are imposed centrally. This creates a lot more work and slows the developers down.

This causes frustration in developers. There is a popular saying which applies here, "Why get a dog and bark yourself". This translates to why hire (and pay) more developers and not let them build and design their CD pipelines, alerting, logging.

Also, if a junior developer is disconnected from the responsibility for creating, and working with their own CI/CD pipelines, managing security etc. then what prospect is there for them to develop?

Of course I have ideas advocating particular solutions to this which will be the subject of a future blog.

Published Date