How Software is Destroying Jobs and Improving our Lives

Each time a new technology comes along, alarmists decry how technology is taking our jobs, changing our culture, and making the world a worse place. In retrospect, these fears have rarely materialized, and a more prosperous society for all has eventually developed instead. For example, AC electricity isn’t dangerous (despite Thomas Edison using it to kill an elephant!), microwaves don’t take all nutrition out of food, cell phones don’t cause cancer, and the internet isn’t just for porn.

This time, some alarmism may be warranted. I welcome the future of innovation, however, I expect it to be different from the past for three reasons:
1. The Rate of Technological Improvement is Increasing
2. Software is Eating the World
3. Most People Are Not Capable of Thinking Like a Computer

1. The Rate of Technological Improvement is Increasing

As Isaac Newton said in 1675:

If I have seen further it is by standing on the shoulders of giants.

While calculus may have been a groundbreaking discovery in Newton’s time, it is now available to nearly all High School students. Some universities require their students to be able to swim in order to graduate, MIT requires their students to pass calculus.

When I was in high school, computer programming was something you did as a hobby if you were interested and motivated. There was a computer lab available, but it was not expected that students have a computer at home. I still remember handwriting some of my college admission essays. Now, children grow up with Code-a-Pillars and their own personal tablets.

You can think of technological growth as being almost as powerful as compounding interest:

Imagine what would happen if life expectancy started increasing by more than 1-year each year? Ray Kurzweil coined the term “the singularity” for when technology improves so fast that all of time before it is left in the pre-historic dust. If he’s right, the most important thing you can do with your life may just be to survive until then. Unfortunately, it’s hard to predict whether that’s 30 years or 30 decades away.

2. Software is Eating the World

As Marc Andreessen wrote in 2011, “Software is Eating the World”. Software is better at performing mundane tasks, and as we get better at writing code (and the machine learning that powers more and more of it) the tasks get less and less mundane. Software can work 24/7, is never late, doesn’t make one-off mistakes, doesn’t get distracted on the job, and has a marginal cost of almost zero.

Kodak had 120,000 employees at its peak, while Instagram had just 13 employees when it sold for $1 billion to Facebook in 2012. We now make our restaurant reservations online, read books on kindles (or just listen to them via audiobook), purchase travel by ourselves, and pay for everything with credit cards that don’t require a trip to the bank.

Many see these past changes as dramatic, but they are actually just the beginning. Soon, waves of low skilled jobs will be automated:

1. There are 1.7 million truckers who currently earn an average of $45k/year(some estimates are much higher). When self-driving technology gets better, these drivers will find that their former employers won’t even let them drive the trucks for free. The machines will drive 24 hour shifts, never get in accidents, and optimize perfectly for traffic and fuel efficiency. 750,000 Uber drivers in the US may face a similar fate. 10 years ago, the idea of self-driving cars was a futuristic concept out of “The Jetsons”. Today, we openly question whether our newborns will ever be allowed to drive.
2. There are 3.4 million cashiers in the US who are already having their livelihood threatened by self-checkout and now newly launched no-checkout stores like Amazon Go.
3. There are 2.6 million waiters and waitresses whose labor is going to be substantially reduced by orders and payments on smartphones/tablets. Shake Shack already lets anyone order on their phone, and the service is so popular they only accept digital orders in their newest Manhattan location.

Pesky fact-checkers will be quick to point out that not all of these people will lose their jobs and some industries will even expand as lower prices lead customers to take more rides, buy more goods, dine out more often, etc. However, market wages for unskilled labor will inevitably face a sharp decline as more of these workers compete for fewer available jobs.

You may see this list and think “I have a college degree, so I will be fine.” Perhaps. Menial tasks are the easiest ones to automate because they’re simplest, but they also have the lowest dollar value. As our machine learning algorithms and processing power improves, higher end jobs will be the obvious next target.

Would you rather have your radiology chart read by your local human who has studied tens of thousands of charts in their lifetime (and most of their experience is with the more common diseases) or a flawless computer who has studied tens of millions of charts under the direction of the world’s best radiologists? The computer never misses a detail, has perfect vision, is never sleepy, and doesn’t forget anything it was ever taught about an obscure disease.

There will always be a role for humans at the very top of their professional field to guide computers and society at large, but by definition few people can be in the top of their field. Where will the rest go?

3. Most People are Not Capable of Thinking Like a Computer

With big increases in demand for software engineers (and a decrease in demand for everyone else), won’t the market ensure that more people learn to code? At the margin, some certainly will. The proliferation of coding boot-camps is a great example where many people are able to increase their salary by a factor of 2–3x in just a few months. This is great and should be strongly encouraged by those who are a good fit for it, but in practice these bootcamps will only meet a small fraction of the demand.

Thankfully, schools are starting to incorporate programming concepts into their curriculum at a younger age. However, writing code is something that is fundamentally hard to teach. The most important skill a programmer can learn is the ability to teach themselves. Writing code involves understanding how logical pieces works at the most basic level, and keeping that knowledge in your head as you assemble building blocks together. Our monkey-brains didn’t evolve to write code, and it requires a very particular subset of human intelligence to perform:

1. Strong logic skills. The best barometer (and what some coding bootcamps use) is probably high school level math. This is not because you need to compute things in your head, but because you need to build simple concepts on top of each-other in a similar logical way.
2. Very intense focus. Writing code is not something you can generally do by writing a little here and a little there from time to time. You need to build in logical components that fit together and work 100% of the time. Any tiny mistake may cause your program to misbehave in ways that can be difficult to detect and fix. For this reason, it is very common for skilled software engineers to be introverts who enjoy spending endless hours in a windowless room trying to make their program work.
3. A hacker mindset. When programs crash, they don’t tell you how to fix themselves. Despite newer programming languages being written with easier syntax and more helpful error messages, they are fundamentally machine systems and not human ones. Any experienced programmer can tell you the story of the time they spent many hours banging their head against a wall trying to fix a bug only to find that it came down to one errant character. You cannot truly claim to understand how something works until you can break it apart and recreate it. PSA: hacking is not some nefarious hoodie-wearing act, but instead the process of hacking something apart and putting it back together.
4. The ability to teach yourself. The field is so incredibly vast, that it would be impractical to try and learn everything. A software engineer is constantly learning what they don’t know and how it’s applicable to the problem they’re solving. They must possess a DIY mindset that is completely foreign to some. A common experience from software engineers is to be disgusted by the code they wrote in any new language even as recently as a year ago.

Even with strong incentives, few people are capable of having all four of these essential skills. The best proof of this is the well-documented “engineer shortage” that Silicon Valley has been experiencing for two decades and counting. Compensation has increased, and movies like The Social Networkhave helped society see computer science majors as more than just Nerds.

Enrollment in computer science courses has not increased substantially despite very strong economic demand, two recessions (dot-com bust in 1999 and the great recession in 2008) where the earning power of software engineers far outperformed other graduates, and a cultural shift towards embracing computers in every facet of our lives. While demand (and wages) for software engineers will only continue to increase, the difficulty of writing code and the subset of people for whom it makes sense will limit supply from being able to grow massively to meet demand.

Won’t everyone just learn to code?

One of the basic rules of economics is that people respond to incentives. If in the future software engineering salaries have exploded while other opportunities are shrinking, won’t people be incentivized to learn? Just as not everyone could be proficient as an artist, an athlete, or a chess player, it makes sense that not everyone can be a software engineer.

Those with the most valuable programming skills will continue to find themselves very employable. As the share of the economy that tech powers continues to grow, so will the compensation talented software engineers are able to attract. Over time, companies will massively increase software engineer compensation as they realize how underpaid this vital group currently is. Just look at how much revenue these tech companies made per employee last year!:

Many workers will shift to new valuable positions that support software-enabled businesses: data scientists, social media managers, QA/test teams, front end developers, conversion rate optimizers, paid ad specialists, SEOs, sales engineers, copywriters, etc. But when it comes to software engineers, the so-called best 10x engineers are so much better than others that the best companies can’t afford to carry much deadweight. These salary gains will go towards engineers with mastery in distributed systems, machine learning, cryptography, compilers, and perhaps devops. While bootcamp rails developers may still see their wages increase, the increase will be much smaller.

Though many existing workers may be unable to adapt, their children will probably fare much better. This cycle will be slow and imperfect, but eventually the supply of engineers should come closer to meeting demand.