Ghost Engineers

There is a new viral tweet circulating, which states that 9.5% of all engineers—are ghost engineers. Their performance is <0.1x of a median engineer, they do virtually no work, and they might work multiple jobs, thus collecting 2x-5x of an average engineering salary. Usually, I’d ignore such statements, in the end, there is a lot of BS circulating in the World Wide Web. But this research seems to come from Stanford, and it has been picked up by news outlets such as BusinessInsider, 404 Media, and the Washington Post (which is behind a paywall, so I’m not linking it), to name a few. Surely, something is going on.

Ad Hominem

Before I dive into the research, I need to address something else. I don’t want this blog post to become an Ad Hominem argument, an argument directed towards the person rather than their position or stance. But nevertheless, I do want to address some concerns about the author.

Firstly, while there is a short paper explaining the methodology, it has not been peer-reviewed. The only credible mark of this research is the Stanford name on it. Academia has always been viewed as a source of credibility, but let us not forget that despite the fact that many good things came out of the academia, it has also served the interests of both individuals and corporations, so before this research has been peer-reviewed, we ought to view it as what it is—an infographic from a Stanford MBA.

Secondly, the author. I tried to find as much information about him as possible. But aside from the fact that he had/is having an e-commerce business, I couldn’t find any relation between the author and software engineering in general. And while it is not forbidden to give advice or express your opinion about things you do not understand, I also think it harms credibility when a person who has no experience in a particular field, tries to measure productivity in said field. It’s as if I would say that doctors should be measured by the amount of pills they prescribe per day, and will provide a nice infographic to support my claim. I can do it, but any reputable doctor will know to dismiss my opinion as it’s clear that my knowledge in medicine is limited to Googling my own symptoms.

With ad hominem out of the way, let us dive into the research.

The research

Findings

First, let’s talk about the findings. According to the author, they looked at private git commits, and fed the data to a panel of 10 experts who evaluated the quality of the code. How scalable was the code that was committed, how slow it was, and, yes, how complicated it was to write it. And the results were shocking!

On average, 9%, or roughly 1 out of 10, of engineers are Ghost Engineers. Ghost, in this context, means that they produce less than 0.1x of a median engineer, and do virtually no work at all. This number goes down to 6% for office workers, and climbs up to 14% for fully remote workers.

In the next slide, the author goes to demonstrate that from a sample of 5,832 slackers ghost engineers, ~58% make less than 3 commits a month, while the other 42% make trivial changes like deleting a line or changing a character.

And the next 3 slides, and the most important in my opinion (I will share further in the article why), talk about the wasted $$ these ghosts create. In the top tech companies such as IBM, Microsoft, Amazon, Google, and others — ghost engineers are costing a whooping $500B. Thus, laying them off, will free these precious $$ money for… hyper growth? I don’t know.. Anyway.

Methodology

Looking at the published paper, we can get some more information about the methodology used. The research focused on Java as Java “is versatile and widely used across various domains, including mobile, web, enterprise, and data applications.” They used 10 Java experts consisting of:

• 3 Managers with 11 to 13 years of experience, team size of 1–50 people
• 2 Executive with 15 and 24 years of experience, team size of 1-10 and 201–500 people, respectively
• 3 Senior engineers with 16, 20 and 25 YOE, team size of 11-50, 51-200, and 1-10, respectively
• 1 Director with 23 years of experience, team size of 11–50 people
• 1 Vice President, with 24 YOE, team size of 51–200 people

Please tell me you can see that flaws in this, please!

The researchers then sampled, wait for it, 70 commits (some teams push that many commits in an hour, roughly a representative number), and asked the 10 experts to evaluate the commits by asking various questions. I won’t list the entire questionnaire, but here are some examples of the 7 questions asked:

• How many hours would it take you to just write the code in this commit assuming you could fully focus on this task?
• What is the experience level of the author?
• How maintainable is this commit?

Then, they created an AI model (of course they did), that matches the output of the experts. That’s it. That’s the research.

My analysis

I am a software engineer with 14+ years of experience. Throughout my career I’ve been an interviewer, a tech lead, a mentor, and progressed from junior script kiddie to leading my own complex designs and implementations. Moreover, I’m a tech entrepreneur building, running and scaling my own online business.

When I talk about productivity, I, and people with my levels of knowledge and expertise, know that lines of code do not equal productivity. Sometimes, you have weeks of discussion that lead to few line changes. Consider an example where your contribution is commenting out a feature flag that held a global roll-out of a particular feature to your entire customer base.

Sure, from the commit itself, you have deleted one line (or flipped a boolean variable). But from the business perspective, it probably involved days, if not weeks of testing; discussions with PMs; alignment with other teams—for example DevOps engineers to support the increased traffic/CPU/memory; and for some engineers it might even involve speaking directly with customers.

And it’s ironic that they chose Java for their research. Considering the fact that Java null reference has been crowned as “The Billion-dollar mistake”, by the creator of the null reference himself, hence commits such as if x != null, could theoretically save millions to the companies running Java for their services. Yet, this “ghost engineer productivity” AI model would qualify such commits as “low value/low quality/super simple”.

The other flaw of the research is the panel selection. Among the 10 selected experts, only 3 are actual engineers. I do not disqualify managers/leadership as being unable to write/understand code, but do you really think that a VP with 24 years of experience, or an executive that manages 200+ people, have been writing code in their day-to-day job, for the past decade? Java is 29 years old, and assuming you are not born as a VP, this person probably wrote some java 20 years ago, before he was promoted to management and then leadership. I think it’s fine to include some managers/leadership in your experts panel, but don’t you think that for a panel that measures software engineering productivity you would want the ratio of hands-on employees to be, I don’t know, higher than 30%? Unless…

My speculation

…you try to appeal to management/leadership. Look, I don’t know the guy, and I don’t know what’s his motivation. Maybe he is trying to make the world a better place by pointing at inefficiencies in software engineering. And there are inefficiencies, no doubt in that, like in any other profession. Is 1 out of 10 engineers are not doing anything? Could be. I don’t know.

But this entire research seems like a sales pitch to appeal to the leadership to get rid of software engineers. The papers seem to focus on creating AI model that consists of 70% “business peoples’” opinion, while the infographic seems to focus on the monetary loss due to ghost engineers.

I’m sorry, I’m going to be blunt here: most people in leadership position know shit about what software engineers are doing. Sure, there are VPs or CTOs who were technical in their career, but most of them are either business people who are focused on streamlining the business side of operation, or they were engineers very long time ago, and they would think that React is a name of a spell from Harry Potter. They all are obsessed with the word productivity, but they are unable to measure it correctly. Therefor they employ companies like McKinsey, a global management consulting company, who published papers about developer productivity. Or they are looking for the next Stanford MBA guy to sell them on some AI, because everything is AI nowadays, on how to measure developers productivity. The deliberate use of business jargon like 0.1x engineer throughout the infographic is made to appeal to business people who like numbers. What does it even mean to be a 0.1x engineer? What is the baseline? Who is 1x engineer?

I worked with engineers who saved their companies millions of dollars per month. They were analyzing spending pattern across cloud providers, talking with different stakeholders, and eventually delegating their work to their peers who would go on and change a config value to create less resources on AWS. Such changes would result in decreasing the monthly bill to the cloud provider.

I, myself, spent weeks working with other people to optimize the interviewing process and on-boarding more interviewers, an action that would increase the amount of quality candidates while decreasing the monetary spending on interviewers who interviewed in pairs, thus costing the company two times more per interview. I spent weeks working on a design review that was so detail oriented that the development of the feature was almost on autopilot, allowing two teams to run in parallel. Heck, I mentored and on-boarded other engineers.

All these things resulted in 0 lines of code written or committed. Luckily, I worked with managers who appreciated people like us, who understood that the output of a software engineer is not the lines of code they produce. And in fact, this research seems to support my claim. You see, managers who are obsessed with productivity, seem to inflict this obsession on their subordinates. Hence, you see that an office environment has a lower percentage, in fact 2 times lower, of ghost engineers than fully remote positions. In the office, it’s harder for you to sit and think about a problem, especially if your manager expects you to write code. This is briefly mentioned in the infographics in this tweet:

Comparison between remote and office engineers.

On average, engineers working from the office perform better, but “5x” engineers are more common remotely.

And unintentionally, this research actually confirms what it tries to confirm, but from a different angle. Productivity raises when you are not spending time in an abusive office environment, or participating in meaningless corporate rituals. An environment that gives you peace of mind and ability to focus and think, which is your home, tend to have higher rate of “5x” engineers, unlike the uninviting and chaotic setting of the open office, with potentially abusive manager who breaths down your neck.

But until managers/leadership will understand this, it seems like the worst tech market since 2008, is yet full of surprises and a grim future where our productivity will be measured by AI models created by business people who never interacted with software engineering in their life.