A brief history of management

Software engineering management is an interesting business. The vast majority of people in there are essentially ex-engineers who never formally studied business or management. The knowledge transference happens in an ironically archaic way by the word of mouth and best practice sharing.

Overtime, I have cobbled together some ideas of how we actually ended up where are with our contemporary management practices. And so, I thought that maybe it’s a good idea to share the story. Maybe I can make it entertaining; albeit in a bit cynical way.

The ugly old days

We don’t really have much of written account of management practices in the old days. It suffices to assume though that it involved a lot of basically punching people into faces until they do what they were told.

The earliest practices that would be considered “management” were probably the process of taxation of occupied territories. Which again, most likely, involved maintaining a careful balance of punching people in the face just enough for them to part with their belongings and not cause them to start punching you back. Or, well, die of starvation.

The old days were ugly, and we don’t punch people in faces as much any more. But, I think it is important to appreciate the fact that the roots of the management are in coercing people into doing something they didn’t really want to.

The industrial revolution

With the advent of mechanized means of production, the societal structure changed dramatically. Now, in addition to the lords and the peasants, we have those who own the means of production, and those who work there. And with that we could say that the management and the working classes were established.

I think, it is a mistake though to think of those as managers and workers in the modern sense of the word. They where more like oompa-oompas and pain administrators. Because the choice for those workers was to either die of starvation on the streets, or work in atrocious conditions and die of heavy metals poisoning while working at a factory.

The managers job here is mostly to maintain a steady supply of poor bastards and keep up the discipline in attempts to prevent people from rioting too much.

Fun fact. This is where our obsession with the office hours originates from. As well as the school education system, which is basically designed to prepare you to work at a factory until you either die, or become unusable.

You know those managers who want you to travel to the office and maintain stable office hours for no particular reason other than them believing that this is the right way to work? Well, they still pretty much live here mentally, and they probably see you essentially as oompa-loompas.

The second industrial age

By the second industrial age we normally mean the era that starts around the first world war. That is when the military-industrial complex and large industrial factories were born. And this is where the actual managerial and working classes truly come to play.

This is the era when the factory management started to show interest in their workers well being across the globe. I like to think that it downed on them that happy workers work better. But, most likely, it was the result of the fact that manufacturing work became so complex that beating workers into a stupor was counter productive.

Along with the change in the employee/employer dynamics, this is where we see the first attempts at improving the production processes at the global scale. Which gave the birth to two ideas that are still very much at play to this day.

The first one was the idea of the org structure and specialisation. No more running around the shop doing a little bit of everything. Now there is a ladder and a military style ranking system. There are people who think and there are people who do.

And the second one is the relentless processes optimisation. Because much of the factory work is repetitive, managers were literally standing next to workers with a stopwatch and benchmarking the expected performance. And in addition to a job description we now have the KPIs. Welcome to waking up every morning to labour under capitalism.

Believe it or not, because they took measurements, they called it the “scientific management method”. Which gave the birth to the “if you can’t measure it, you can’t improve it” mentality that is very much present throughout most of the industries to this day. There is still no end of managers who want to run a “data driven” company and treat people as disposable units.

The raise of the creative class

It took two world wars, countless conflicts in asia, and probably close to a quarter of billion people dead for militant imperialism to fall out of fashion. And at that point people started question whether working their entire life as a cog in a giant machine is a life worth living.

You see, not everything that is important in life can be measured. Actually, if you stop for a second and listen to those old greece philosophers who died 2000 years ago. You’d know that value is inherently subjective and hence unmeasurable. And so people started asking questions.

The world was, and still is, a very much class based society, but now there was a new class of people: the intellectuals and creatives. People who create knowledge and experiences for a living. And those people were way too privileged to be managed the old ways. And so, the management adopted.

Instead of coercing people into doing what they’re want them to do by imposing various structures, managers started to focus more and more on “gardening” people. They would tend to their needs and wants, and empower them to make their own decisions. All in the name of maximising the shareholders profits, obviously.

Because the first economies of scale started to kick in, the management quickly figured out that investment in people and diversity is the most profitable. Because people are the ones who create new products. And cornering the market with a new product is a much better deal than competing in a saturated market.

And this is where we start seeing the shift towards the servant leadership styles, and appearance of managers who are not the subject matter experts. Empathy and people skills became some of the most desired qualities in a manager.

The stone age of software

Coming on this legacy, we can see how the early days of the software industry in the 80s, 90s were panning out. Most of the software was developed by that privileged ex-academia group of people who were treated as special—hence the term “specialist”; jk. Everything around them would still work on the old factory mindset, KPIs and all.

Because most of those early days software companies were overcapitalised, they could tank the unpredictability of their few creative class workers and otherwise manage the processes as waterfall projects. And hence the new breed of managers was born, the project managers.

Now we’re starting to see this weird blend of managers who don’t really know what is that exactly their specialists do and yet have the incessant need to manage. They would bikeshed by assigning JDs and KPIs as they would at a factory floor, and the creatives would sign basically anything that will let them have a play day every day. And as long as value still produced both sides agree to pretend that it’s all going their way.

As the profits and the size of these organisations grew, they did the only thing they knew how to do. They turned into feature factories. Org structures, seniority ladder, and everything. Welcome to the corporate america.

A significant part of this setup was that making software in 80s/90s involved a whole lot of non-programming overheads, such as building infrastructure, marketing, physical copies manufacturing, distribution, etc. Which were a subject to the standard methods of planning and execution. Hence the feature factory actually made sense.

The age of startups

As the industry progressed during the 90s, two things happened. Firstly, thanks to the internet infrastructure the barrier of the new business entry fell to the floor level. And secondly, the corporations became too large and bureaucratised to innovate effectively.

As the result, corporations turned into vacuum cleaners of innovation. Instead on innovating themselves, they would let somebody else to take the risks and then acquire those startups that made something useful. Which, in turn, attracted all sorts of risk taking types and created the whole startups industry. At which point most corporate management practices went out of the window in the game of fail quick or sell to google.

How can we live as an industry with a 90% failure rate? Same as the scratch ticket addicts do, you ignore the facts and keep believing that you’re special. No amount of facts and reasoning will convince an average startup leadership that blindly following agile practices is bad for their health, and that PO driven scrum pushes their teams to make suboptimal decisions on a daily basis.

In all seriousness though, the significance of this period is that because startups are tight strapped for cash, they could not afford to run the same management machinery as the corporates do. And once again, the management practices had to adopt. People started to experiment with custom roles, flat org-structures, and borrowing heavily from the LEAN manufacturing practices. Because when you’re a small 10 people shop there is no point in overcomplicating things.

As the result, this is where we see the raise of the product management and scrum masters as the decisions makers; as opposed to the more traditional setups where business people are calling all the shots. Which had a huge impact on how we manage software engineering teams those days.

The problem was that in attempts to get products on the market as quickly as possible, they over corrected the change towards the product function. Although they would structure teams as cross-functional, they wouldn’t really act as cross functional teams in this period. Most product management was essentially the old project management in disguise, agile practices were implemented superficially, and processes were still attempts to command-and-control. The amount of suffering that inflicted on engineering teams is still reverberating throughout the community.

It’s not so much that agile is bad in it’s own. The problem was that in this period the agile/lean practices were applied within the same old factory mindset of command-and-control. One cannot implement enablement practices while at the same time assume that workers are dumb and irresponsible. It’s either one or another. Sadly it is still very much a prevalent mindset in many technology companies even in this day and age. We are not out of the woods yet.

The open-source land

Speaking of having no access to professional management. Do you know what would be a project manager’s worst nightmare? Open-source. Thousands of people working in a weird anarchic frenzy and delivering the best quality and most complex software on the planet, completely remote and without a single meeting or a manager involved. No offices, no org-structures, no KPIs.

The actual realities of open-source work are a bit more complex than that though.

Firstly, there are at least two generations of open-source now. The original founding generation was essentially a rebellion against their corporate overlords. And then there is the generation of software engineers who grew up with open-source practices as a norm, and so they expect similar setups at their day jobs.

Secondly, the no management in open-source is a bit of a myth. There are layers of ad-hoc community management. They abandoned KPIs and the annual performance reviews as stupid ideas, but it doesn’t mean they don’t plan and coordinate their work.

And lets not forget that open-source is essentially overcapitalised in terms of access to the human talent. Which means that one doesn’t really need to run a tight ship and manage all that well to make progress. If an idea is good and hyped up enough, there will be no end of willing volunteers ready to commit endless human-hours into moving the project forward.

Whether we like it or not, open-source community had a huge impact on work environment expectations that engineers have those days. This is where remote work and hate for meetings and pointless HR paperwork comes from. And once again, management had to adopt. In many good companies those days, DX, community engagement, and reliability have just as much value as customer or stakeholder value. DevRel management cropped up as it’s own critical role in a technology company.

Here and now

Today’s software engineering management landscape is a very interesting one. It has layers upon layers of ideas from the previous generations, often some of those completely contradict each other. And on top of that there is an incessant need to keep improving and finding every edge that can help improve the outcomes.

I think it’s easier to think of this landscape as a gradient that goes from completely ad-hoc practices to heavily structured corporate environments. And most companies are actually on a transition path from one end to another. Startups -> scaleups -> enterprise.

There are few truisms that we have figured out on our journey here though:

• No function should have the upper hand. Truly collaborative environments are the ones that produce results most consistently
• People don’t need as much management as we thought they do. Autonomy and enablement are the keys to performance
• There is no way of predicting the market, and an iterative approach to delivery is the most effective one
• Organisational structures create pain one way or another, but lack of structure creates even more pain and suffering
• In creative professions KPIs and annual performance reviews is a colossal waste of everyone time and energy

Our problem right now is not so much that we don’t know what works. Our problem right now is two folds. Firstly, management is a slow moving profession, and all those new understandings implementation often falls on the shoulders of people who learned the ropes in the old days; some of those practices create an outright identity crisis in those managers. And secondly, we are quite oversaturated with management ideas at this point and that creates a mess of confusion.

If there is one trend in the contemporary management worth mentioning it will the the fact that we are slowly moving away from the idea of management as we used to know it—the big boss model—, and shifting towards, well, coaching. Teams are filled with grown ups and they are more than capable of making their own decisions; that is where the key to the next level of performance lies. And, unlike a manager, a coach doesn’t make decisions, they train the team to make good decisions for themselves.

The future

An interesting cultural quirk of humans is that every generation has a tendency to think that they are the pinnacle of morality and best practice, and everyone before them were basically unwashed savages.

If we follow this line of thinking then we arrive to the inevitable conclusion that in a generation or two, it is us who will look like unwashed savages who couldn’t put two and two together. And I’m kind of weirdly excited about that. It gives me hope.

But, there is another way to think about it. Management at its essence is all about managing complexity. Unfortunately, our barely evolved monkey brains are made of meat and not all that great at dealing with complexity. We have all sorts of biases, glitches, and emotions; eww. And do you know what is great at dealing with a lot of unwieldable complexity? That’s right: computers.

I for one welcome our new computer overlords.

But then again, I’m just an ex-developer who never had any MBA; so, what do I really know?