The great polarisation

Cost disease, corporate strategies and breaking through technological stagnation

[Part of an ongoing series - one, two, three, four, five, six]


The world changes surprisingly fast sometimes. For example, a century ago, 22,000 people died in the construction of the Panama Canal, which was an acceptable casualty rate for something so important. It dropped a lot for the empire state building, which only cost five lives and was considered massive success. The world trade centre in the 70s brought the bill in at sixty. There is substantial variation, some due to actual performance and some due to just speed, but the trend line here is a tad murky but clear.

If you look at the changes over time, it sure looks like we're getting better at preventing the type of crazy deaths we used to see before. Whether due to better policies or better enforcement, it seems true that most types of jobs that one would consider dangerous (mining, construction, transportation) have gotten safer.

This is undeniably a good thing. But it's also emblematic of the way in which our stated goals sometimes conflict with what we are willing to give up. Patrick Collison keeps a list of things we used to build ridiculously fast. Many of the things essentially show how coordination problems somehow used to not be as bad as it is today. Whether it's building the Eiffel Tower in 2 years or Apollo 8 being moon-ready in 4 months, or building the Empire State Building in 13 months, or building the world's longest suspension bridge in 4 years, it definitely seems like a time when stuff just got done.

It sure seems like we're living in a world that's bifurcated. The rewards to being the largest, or number one, is extremely high, while the rewards to being the median is ever declining. This can be seen in most areas of human endeavour including economics, politics and in businesses.

When you look at the trends of the last few decades, it’s difficult to miss the signs, overt and covert, of there being an increasing polarisation in the business world. Even in the last decade, firms have been seeing extremely high profit rates primarily driven by the lack of competition. But this hides the underlying fragmentation of a previously solid market, which is likely to be the beginning of the next phase of exploration vs exploitation. To take a few examples:

  • Entertainment: The top musicians make more money today than ever before, and attract the lions share of profits of the industry, even as the industry contracts. The top writers retain their king status. The Hollywood studios make fewer movies, at larger budgets, which aren’t independent. But, there is the concomitant rise of the independent artist, who has a pretty good living making music, the rise of the independent studio who makes the $80mn budget critically acclaimed movie for adults, and the proliferation of the television medium by destroying the existing channel. What this shows is not the great polarisation within the industry, where winners take all, as a cautionary tale, but rather a fundamental reorganisation of the industry into a slightly more multi-polar world where there are larger number of players, with more money at their disposal, who are able to create things. The polarisation is coming with the emergence of multiple systems which differentiate the sources from which we get the critically acclaimed works, and those aimed at popular consumption. While Borders has been killed ruthlessly, indie bookstores have been thriving.

  • Finance: The biggest banks have gotten bigger, and the largest hedge funds have gotten much much larger, showing a similar trend towards the winner take all market effects. But this hides the fact that banks today are no longer the dominant financial players in the way they were in the past. We have a plethora of other options — venture capital funds, private equity, sovereign wealth — all of which together have created an ecosystem in which the flow of money is much more egalitarian than ever before. Once you look also at the newer systems that are emerging, around payments, transactions and settlements, then even the steady paychecks that banks used to cash are no longer so steady. Once again we’re seeing a polarisation where the number of players are increasing, the heterogeneity of the players belie the fact that in any one individual vertical we assume we’re seeing a winner-take-all effect.

  • Technology: The technology companies in the world are seeing massive growth, and massive scale advantages, taking over entire sectors with their might. Whether this is Uber in transportation, or the FAGA mafia in general tech, there are only a handful of firms that guide the evolution of products, industries and standards. The innovative endeavours, even when they occur, are explicitly or implicitly showcased as an outsourced talent and research agency for these behemoths.

  • Movies: In the movies, if you’re not the blockbuster, you’re toast. The largest movies of the whole of history are from the last two years, and so are some of the biggest disasters of the last 30 years. This shows that part of what has happened is that everyone seems to know what everyone else is seeing, and therefore the network effect has caused there to be a much larger polarisation in the movie industry. This means that the sources of movie production has stopped being the six major studies in Hollywood, but includes the six major studios in Hollywood, the indie studios and completely new entrants like Netflix.

The trend, as it exacerbates, will cause us to deconstruct the boundaries between various sectors. While the arrival of new technologies, including machine learning and artificial intelligence, will cause some rupture in the basic economics that underlay this phenomenon, it’s worth looking at this landscape to understand the true power of the market behemoths that exist today. It’s, as is so often claimed, truly unlike what’s happened before.

Positive feedback loops create the power laws we see know nature, with usually simpler interactions and lower complexity, and negative feedback loops create outcomes alone a bell curve, highly complex and a result of a substantial number of independently moving parts adding up together. Perhaps there’s a threshold of the number of factors and types thereof where the end behaviour changes from one to the other.

We often try to understand the dynamics behind economic events — such as the discussion around top 1% seeing their net worth rise while everyone else sees stagnation, or how the rise of new technologies creates instantaneous rise of monopolies due to network effects. The orthodox method of economic analysis to understand these trends give us widely disparate answers:

  • Wages for most of the folk was suppressed from late 70s onwards due to emphasis on “technology” as the key factor of production, instead of labour. “Technology” as a factor of production is a fudge that exists solely to help express this relationship though, and has no sensible definition outside.

  • Suppression of the wages also happened because of globalisation, as it brought cheaper competition and suppressed labour’s bargaining power.

  • Increase of bond market returns in the last few years happened due to the “safe haven” effect of US treasuries, and most of the rally in the world equity markets happens as demand is picking up.

  • Capital allocation has moved between sectors and investment styles according to the fashion at the time, and led by the paucity of investment options in ‘normal’ products.

  • and so on…

The situation however is slightly more complex than it appears. Some industries like computers and telco have seen such drastic reduction in prices that it seems like a genie was at work. And other industries like healthcare seem like it was cursed by that exact same genie. It's not very clear as to what makes one stand out vs another.

The decline of costs - lands of plenty

Technology has given us an intuition that doesn't bear out a large part of things in our life. Moore's Law has taught us that computers decrease in price consistently, year after year.

Same for all forms of communication. The costs have fallen off a cliff in the past few decades. Used to be that calling someone across an ocean, or even across a landmass, was something to be really considered. But now, it barely takes a second of forethought.

The share of household expenditure on communications doesn't seem to change, showing that even as the costs have fallen we have increased our consumption.

If this is close to indicative, what have we learnt? That certain goods have fallen precipitously in price over the past few decades, and that even as they fell, we ended up using and buying them more. It's in effect the Red Queen race - running as fast as you can to stand still. The sector has got monumentally more productive only to see their revenue share stagnate.

Is this a unique occurrence? Does this also happen in other sectors?

A look at transportation shows that the same trend holds true. They too are running the Red Queen race. Even as we yell at the airport security theater we forget that the cost to undergo the humiliation is a fraction of what it was before.

The cost of transportation has declined so steeply that you wouldn't want to ski down its slope.

What about food? It also seems to have followed a similar trajectory, and grown at just about rate of inflation. Most are cheaper for sure, except for a few items like grapefruit and peppers, and white fish, the prices have all fallen. We live in a golden age of lower prices again, as long as you're not solely living on grapefruit and bread.

Is this only a problem in some countries? Doesn’t seem like it!

In 1982, spending on food by individuals and families represented 8.3% of disposable personal income. By 1992, spending on food had fallen to 7% of income, and by 2002 to 5.9%, and by 2011 (last year available) to only 5.7%.

What else, if we look at toys, same news. TV, same trend. Clothing, cheaper than ever. Cars and home furnishings, same story. At the same time however, there is another part of the story. Healthcare, Education, Housing, Childcare - they take up an ever increasing portion of a household's expenditure. There are some squiggles depending on when you start to draw the line, but apart from that there has definitely been a bifurcation.

So some things have definitely gotten cheaper over the decades, and some things have definitely gotten more expensive. And somewhere in that dance, the disposable income has most definitely gotten squeezed to almost nothing.

Industries that have bucked the trend and gotten more expensive

So what's common about the things which have gotten more expensive? Well, one thing that immediately comes to mind is that they are all areas which have seen market failures due to government subsidisation and power of signalling. Let's have a glance at two of the worst offenders - healthcare and education.


While there has been substantial improvements in our ability to find and treat later stage and terminal diseases, by and large we are at a standstill, or at best minor linear improvements. Life expectancy has definitely improved in the past few decades, even if it has hit a bit of a standstill recently.

As we spend more and more, we can also see that the years lived with disability or disease has increased. This has definitely contributed to the small and tangible increase in the life expectancy too. Note that this is on a log scale, which means we have to keep spending exponentially more in order to keep getting a small linear increase in years lived.

While this is indisputably due to better healthcare in some ways, it's pretty clear that the difference did not come about due to some fundamental new discovery, but rather painful and incremental improvements coming from other, potentially outside, innovations. Just like one of the biggest improvements in life expectancy came from the great advances of sanitation, i.e., washing your hands if you were a surgeon, there are plenty of ways by which changes in the environment could have affected the slight increase in life expectancy.

This seems like a reasonable place to be. After all, without a new paradigm emerging regularly, there's no reason to believe that we would be able to bend that curve upwards like a hockey stick. However, there's a wrinkle, which is that this has come at the expense of an extraordinary increase in the per capita healthcare spending. Of course, US is an outlier, but it's not like the other countries have seen substantially flatter curves either. They have done better, but take the red line of the US out, and we would still be left with a mystery to unpack.

What's the conclusion? We do seem to be in a world of diminishing returns here, with minor improvements being made in actual health as the healthcare costs balloon.


Overall, the public expenditure on education seems to be relatively flat over the past few years, even as there seems to have been increased (and discontinuous) spending on higher education resources.

The flip side is that the proportion of expenditure that's shouldered by the households is dramatically higher as the education levels move off of the "baseline", which is secondary education, and move into the tertiary stage. In other words, if you want your child to go to college, vocational or otherwise, you need to shell out the cash for it.

One of the key questions here is whether this increased expenditure actually has any effect.

One of the studies that looked at average reading performance in PISA and average spending per student in the secondary school stage at least showed that there was essentially no relationship between spending more money per student and getting better outcomes, at least if your aim was to see if they learnt how to read better.

In one way this makes sense. In many ways the spending you do is for the teachers and the facilities and the administration. It's not for the curriculum. So once you have some baseline, then you might pay more for school in some places than others, but the curriculum itself doesn't vary by much. You don't really see 12 year olds happily solving calculus in one place while seeing them being taught multiplication tables by rote memorisation in another.

But one question is why the costs would have increased in the first place. Maybe we are paying teachers a lot more? No, doesn't seem like it. Maybe we are hiring four assistants for every teacher because they hate doing the annoying admin work? No, that doesn't seem to be the case either.

In fact, it looks like the growth is mainly because schools have now expanded from 'centers of learning' to 'centers of children enablement'.

And it's those additional areas which schools now have, such as music and drama and sports and so on, that have created the added bloat. Looked at it like this, it looks like a bundling problem. More people are being forced to shell out money to subsidise the growth of the added services, even as an unbundling might be more economically efficient.

In one of the more interesting studies, it seems like universities in Western Europe came into place to replace Monasteries as places where you would send young men (mostly) to learn about the world. Every time in history you read about a monk or a priest making some scientific discoveries it's worth noting that it happens because if you were the curious sort, there really wasn't many other places you could go to satiate yourselves.

But our original analysis of the cost change shows very clearly that College Tuition, in this case more correlated with Tertiary education, seems to have gotten out of control. As the costs have increased, so have the ways in which students have started to take out loans in order to be able to afford college.

It's not like the students have a choice. If they want to get a decent job, they have to go to college. But colleges are becoming more expensive. So they have to take out loans. And once you take out loans, you have to go for specific types of jobs if you want a hope in hell of paying back those loans. And so the circle continues.


It is kind of a truism that we spend an inordinate amount of money in housing. On the one hand, the house prices are a much higher percentage of earnings than almost ever before, and thus the affordability seems to be taking a hit. But there's a wrinkle, which is that mortgage payments as a percent of take-home-pay is close to all time averages. Why would it be? This though is because interest rates are at an all time low.

And when you look at what's driving the price increases, it seems like most of the increase is being driven by the value of land vs the value of buildings. If you look carefully enough though, you can see the giant housing bubble that burst by 2008. It was a time of slightly higher interest rates though, which made the mortgage payments spike even higher!

One added note is also that we are most definitely changing the size of houses too. When you look at how the cost of the average house has changed from 1950 to 2010, you see that the house prices have gone up 3.6x, while the price/ sqft has only gone up 1.5x. That means that the much vaunted efficiency through better materials and construction has shown up, but we’re squandering it through poor site selection, land prices, and house sizes.

Why have these sectors seen such increases in cost?

Now, the 'cost disease' has been discussed in several places for years now. But the mystery remains that the costs have indeed increased. Through some combination of incentives that changed across the industries, their costs have increased dramatically even as those 'buying' the products complain about why this is the case.

It might just be because we no longer have the same ability to get value from the organisations that do give us value. Perhaps we have all just lost any bargaining power we had with the forces that give us these things - through a combination of regulatory overhead, increased power in the form of larger companies, and decreased purchasing power that those buying things have.

Part of the argument seems to be that nowadays you need to halt construction if you even find a rather manky mole, in case there's environmental impact for your project. It's the classic 'regulations make life hell' argument. It could also be that evil corporate overlords have discussed this in their regular evil-corporate-overlord meetings to increase prices and make life tougher for everyone. While the argument sounds facile, since clearly there's no evil-corporate-overlord meeting, it's not immediately clear if there is actually no implicit price increase collusion.

Regulations have increased steadily since the 1980s

A cross country analysis of regulations show us though that there's a part with linear relationship in the middle, but just as common sense would indicate, high levels of regulation and very low levels of regulations both lead to lower economic growth. As one example, the growth in the infrastructure sectors is coming from the more deregulated sectors. It might not be the sole reason, but there definitely does seem to be an argument in favour of it having a stifling impact on growth.

A look at the spend by federal regulatory activity and staffing shows the immense growth in regulatory activity over the last few decades. It seems like an awful coincidence that the business profits grew at the same pace.

There seems to have been an incredible effort on all parts to find and reduce corner cases, which is what leads to the explosion in regulations required. The rules on how to do something grows much faster when you try to solve for every single eventuality. It is much more difficult to make rules for marginal cases than for average cases, and the more effort we put towards the corners the greater the effort needed.

For instance, people are rightly worried about voter fraud. After all, we want elections to be free and fair. But also, it's a system that rewards a binary win for candidates, which means the incentive to cheat is high! But the level of fraud is very low. What gives? Maybe it's civic responsibility, though it seems odd that this is the only place that comes to fore. And every election you do find a few examples of it being done too. It's that voter fraud just really tough to pull off in large quantities.

So we have a choice, do we spend millions and billions chasing down every single fraud, or do we do make a certain level of effort and not focus on frauds < X, where X is an arbitrary but small enough number that we don't care, and move on with our lives? The more effort we put in to reduce X, the more the costs!

Corporate profits have been steadily increasing

The corporate profits have been on a tear at the same time as the regulatory bonanza. This has been growing proportionally alongside the GDP. Even as the corporate profits have increased in recent years as a percentage of GDP, the income-tax receipts have decreased in tandem.

The tax collection reduction is indeed a conundrum, but it seems mostly due to a couple forces - 1) more international revenues from increased globalisation means less tax receipts in the US, 2) there has been a democratisation of the efforts to create and slip through tax loopholes.

So how should we think about the fact that even as regulations hiked up, the companies seem to have made hay? It's unclear. Of course quantity of regulations is not the same as quality of regulations, so it might very well be that we were just closing loopholes and levelling the playing field. It is kind of striking though that this point doesn't seem to be made by anyone on any side of the political spectrum. It's always possible that there's a split between the companies that got hit as the regulations increased, and the extra growth that other industries and geographies got which helped push the curve up.

A look at the work done by Bain, looking at company profit pool evolution over the past two decades, indicates that the majority of the global profit pool growth has come from US and Developed parts of Europe.

The largest companies have seen much higher ROE and faster growth

Moreover, there's a clear growth in the return on equity that US public companies have seen, mostly driven up by the large public companies. The most recent figures also indicate that this still stays true, and most of the market now moves in tandem with the largest few companies. The smaller companies seem to have found their growth through layering on debt, while the largest companies seem immune to this, by and large.

So, to recap, there's been a massive increase in regulations, but simultaneously the largest companies saw a massive increase in corporate profits. And one part of the story of why this happened is that the profits shifted to sectors which are capital light and idea heavy. But is that all?

Industry concentration has increased in most sectors

One other funny thing that happened is that the industry concentration (in the US) has been rising since 1980, after a decline from 1960 to 1980. A coincidence that happened alongside the increase in regulations and the increase in corporate profits? Someone steeped in the education that correlation does not mean causation would surely balk at the identification of a casual link here, but ... come on! A thing that increases the staying power of larger companies somehow manages to coincide with companies getting larger and more profitable? Yep, could be coincidence.

In most industries, there has been an increase in the HHI, a well known way of measuring and understanding market power and dominance of the largest companies in a sector.

But it's of course likely that the largest companies became large purely through their intense and successful competitive efforts, rather than acting as a way to increase sales weighted average markups. In that light there was a golden period of competition when the markups declined, from 1960s to 1980, and it's all turned around since then.

Maybe the right question is now why we are seeing an increase in the monopoly power today, but rather what made the 60s and 70s an anachronism. The US did change its merger guidelines to make it friendlier and more 'hands off', but surely this has to do with something other than capital structures?

Purchasing power shares of labour has gotten K-shaped

Now, maybe what's going on is that people are just not getting their share of the corporate profits. So if the workers just aren't getting their share, then maybe the prices are going up as a result of basic supply and demand economics.

Well, that's a rather damning slope. Is this one of those cases where the averages hide what's going on?

Hmmm, that's advantage to the 1% hypothesis. The decline of the labour share is much better when you look at this cohort.

They also seem to own a much higher percentage of stocks. Coincidence?

Might this mean that the rise in corporate profits is highly useful to those who are also the holders of stocks? Possibly there is a correlation here, and tells us that the top percentiles of the income distribution, if you ask them, are not the the ones who will complain vociferously about the cost disease.

But maybe this is only relevant to stocks. Let's have a look at another asset. Looking at property, the question is whether there's a location premium as well? After all, unlike stocks, property that you own usually stays in one place. Funnily enough (though maybe only to me) it is one of those assets where adding further functionality like mobility makes the price and depreciation lower. Having argued with my better half about the necessity to buy a boat instead of a house unsuccessfully, the stark reality of this is rather clear.

Looking at the UK this phenomenon becomes quite clear. If you want to live in London, then the median price is higher, and the price increase in the last decade has been higher, and the distribution is wider. It ticks every single box.

This indicates that there is a significant location premium at least for this asset class, even if we can’t directly see the ownership income distribution. But looking at the per capita incomes of the different regions, the correlation is clear.

So if we look at property as another asset class, clearly there's a cost inflation here as well. Those at the top of the income distribution might find this cost inflation easier to bear, or maybe even find this highly useful because they actually own the properties, but the median trend is pretty clear.

The exact same distribution also seems true in the US, with big differences between the coasts and inland, between urban and suburban and rural areas. So why don't the people in London move out to North East? Simply put, that's where the jobs are. This might move around more now that remote work is more acceptable, but it's not nearly a fait accompli as it's made out to be.

Is this a question of educated vs uneducated? Looking at the salaries of doctors, lawyers and bankers, it definitely doesn't look like it. There are bumps and rises, but it’s not something you look at as a phenomenal rise, but more like what a decent GDP growth clip would bring.

Now, one area this breaks down is software! Software engineers have been on an absolute tear over the past two decades. But it's worth noting that the entire industry is only a few decades old, so some allowances must be made. After all doctors and lawyers and bankers have been around for centuries! Even Peter Thiel talks about when he went to school, real engineering meant building things, not software.

There's therefore the alternate question of whether the software engineer phenomenon is the exception that proves the rule or just an exception. When you look at aggregate behaviour of a variable, picking out the outlier has to be done very carefully. Otherwise you're just cherry picking the data. Either the outlier is an outlier for random reasons that lay outside the model, or the outlier is exactly what the model would predict.

One way might be to look at other industries in their first few decades of life, to see if an increasing trend is emblematic of an industry that's ripening vs some uppity labour folks getting their due!

Corporate strategies

So far, so macro. We've been looking at things from a fair distance above and seeing if the squiggles on the ground below are somewhat neat and allow us to distinguish it into relevant categories.

Having some distance is good. It stops us from getting overwhelmed by the sheer crazy diversity of an entire economy. Just thinking about the insane chain that exists to make the smallest thing, say a pencil or a button or a shoelace, makes the mind boggle. With that level of complexity you need some distance.

But there's also another layer that's important. While it's all very good to think of our individual bargaining power and the Sisyphean task of understanding price trends in healthcare or whatever, it's useful to look at the persons that help form the actual ecosystem.

How do the companies figure in this complex web? They're after all the entities that end up getting blown hither and thither by the vagaries of the market and governmental power.

So when faced with these circumstances, companies had to figure out a strategy that would help them increase their profits. For any company the decision was, in the fullness of ambition and insight, to follow a simplified path:

Globalisation as cost arbitrage, and opening new markets

The first two became easier to do in the 1980s and 1990s. You could rely on global supply chains getting more and more sophisticated, and there was an increasing realisation that brains were not the sole provenance of those companies in the West.

And hey presto, globalisation! Not only could you grow by finding more markets and selling Coca Cola in the far east, you could also substantially reduce your employee costs by moving it somewhere cheaper. As an ex-consultant, the archives are filled with case studies of millions saved and efficiency gained (and consequently profits increased) as companies found ways to move larger and larger bulk of their employees to an offshore location.

M&A activity to generate economies of scale and scope

One prediction would be, if the framework is right, that you would've seen waves of M&A happen as management teams looked to take the easy way out and increase their share prices.

And that's exactly what a McKinsey study showed, that M&A had grown 5x since the 1980s.

The corporate world has experienced spikes of mergers and acquisitions in the past, but nothing like the recent wave of deal-making. In 1990, there were 11,500 M&A deals whose combined value was equivalent to 2 percent of world GDP. Since 2008, there have been some 30,000 deals a year totaling roughly 3 percent of world GDP

The incumbents keep succeeding

There is also the very reasonable and Econ 101 reason of economies of scale leading to market domination. For most industries where there is larger levels of fixed investment, the power here is very strong. E.g., in the industries of energy, or utilities, or automotive, the benefits of being a larger incumbent is substantial. The only new car companies to come out in the past few decades have been through heavy levels of protectionism enabling the creation of better products and larger markets (Japan and Korea both did it to great effect, including providing a captive domestic market to iterate over), or through the sheer wilfulness of a crazy millionaire (Musk).

Companies have tried to succeed in this world through increased globalisation, both to increase markets and do cost arbitrage, M&A to increase their size advantage, and hoping to take advantage of innovation and automation to entrench their advantage.

But does this mean that we have a ton of companies forming and trying to take advantage of this unique and wonderful confluence of events? It sure doesn't seem like it! In fact, what seems to be happening is that there seem to be a substantial portion of tax incentives that seem to go into value-added as a percentage, and this is disproportionately enjoyed by the larger incumbents.

For a substantial proportion of jobs, there also seems to be rather onerous requirements regarding licensing and certification that needs to be completed, and this creates further barriers in front of wannabe entrepreneurs.

There is also a clear implication in that this has reduced the number of companies that are starting up in pretty much each sector. Not only has it fallen within each industry looked at here, but it has also fallen across most educational levels.

It does seem like a bit of a conundrum.

Of the top 10 pharma companies in the world, 8 are more than a 100 years old. Of the top 10 food companies, the youngest was started in 1935. In Banking, apart from China, JPM started its lineage two centuries ago, Bank of America formed by merging companies in 1928, HSBC started a century and half ago, BNP Paribas and Credit Agricole both started a century and half ago as well. In Insurance, again, the largest companies are overwhelmingly centenarians. There is a pattern of the largest companies being focused on marketing and distribution, while the innovators act as externalised R&D, which then get acquired.

And technology is following the same pattern. Amongst the largest technology companies, they have all been incredibly acquisitive, and the power continues to grow. Funnily enough, it plays out even in the sub-sectors within technology. With the first wave of computing power, IBM evolved as the behemoth. With the wave of personal computing going mainstream, Microsoft arose as the giant to reckon with, with Apple not far behind. The internet gave rise to Google and Amazon. The smartphone gave rise to Facebook, and the resurgence of Apple. And the horsemen surge higher, and remain resplendent in their power.

Will there be another wave? Yes, even though we don’t know when it’s likely to be, or what is likely to lead it, it's impossible to try to innovate in any meaningful sense without that as a yardstick. The broader pattern however seems to play out with every technological paradigm.

The stagnation of a particular technology or a particular sector seems to be broken up only through the invention of the next wave. But with scientific stagnation across multiple sectors reducing the space for the next wave, technological stagnation is the inevitable end point. And without a constant new supply of S curves being created, the impetus that drives further growth gets nullified.

Market failures

So it seems like we live in a world where the regulations are increasing rather rapidly, corporate profits are also increasing (albeit not equally), the largest companies have reaped disproportionate benefits in terms of increasing market concentration (again, for a variety of reasons) and most of the benefits of this has happily gone to the top echelons of income earners, who are often also the capital owners!

So why are the expensive things so expensive, while the cheap things are so cheap? And why are the companies providing the cheap things so successful at making record profits, and those providing the expensive things seemingly not as successful? The pharma companies are doing well, don't get me wrong, but none nearly as well as Facebook.

In any case, the areas where we see the trends of lower prices not applying are the ones where there are market failures of one sort or another. Not all market failures are the same, some can be brought about by over-regulation and stifling of innovation, while others can come about by the exact opposite, under-regulation and letting companies compete away any hope of profits (hello Uber!).

We might have to work backwards to figure out why it is that certain costs are increasing much faster than we give it credence for. On the one hand, apart from silicon valley and its hordes of technology millionaires, there seems to be almost no real startup activity happening elsewhere. Even there it’s been declining. On the other hand, the presence of silicon valley stands as a rather giant "oh really" sign against the whole thesis.

What would account for the difference? There really seems to be only one possible explanation, the parsimonious one that the emergence of technology as a sector helped push along the first major wave of innovation, while the emergence of technology as a horizontal enabler made the rest of the industries stand rather still.

A short detour to Silicon Valley

Silicon Valley has always held at its core two seemingly contradictory ideas. One is that the essence of innovation comes from the outsider, the precociously brilliant 22-year-old who asks the question why things are done this way, and is able to cobble together an answer that surprises everyone. That rewrites entire industries, through the sheer power of ignorance of the status quo they are able to push forward a new narrative and see a new vision. The second narrative, and dare I say the more popular one at least as seen by funding fervour, is to find those who have proven themselves before and fund them again. This is where you see serial entrepreneurs and ex-executives from large companies getting huge rounds of funding thrown at them, in order to disrupt the very industries they were part of often for decades.

So which is true? Is innovation a worship of a new way of seeing something, born through ignorance of the enormousness of a task, and only brought to light through a clean-sheet reimagining of an archaic industry? Or is it a bet on reinvention through intense familiarity, one only brought about through the sort of expertise that takes decades to build up?

Around the time there was a sea change brewing in the economy. The post-war (World War II) created a new form of financing, primarily led by the government, that aimed to foster new ventures and thereby recreate America.

Stanford University, at the time, stood serenely as a premier college but without the funding fervour or the science strength we associate with it today. As the new university called Stanford tried to build its reputation out on the West coast, this seemed a fantastic new way to grow and increase in importance. It was led by a charismatic chap named Frederick Terman who made a decision to make this come about.

The system that Terman put in place at Stanford relied on four pillars:

  • first, reach out to military prospective customers to better understand their needs, then offer to craft them a prototype in Stanford’s research laboratories—this generated substantial revenue for the university and strengthened its trusted relationship with key military figures;

  • second, if the prototype satisfies the customer, encourage one of your students to found a company and manufacture the product at a larger scale—this inspired an entrepreneurial spirit among the students and contributed to stimulating their hard work in the university’s laboratories;

  • third, make sure a member of the Stanford faculty (if not Terman himself) becomes a board member or consults with that newly founded company—this contributed to training Stanford scholars in business and turned them into better teachers and researchers;

  • fourth, provide office space in the Stanford Technology Park, which was made possible by the fact that the university was the primary land owner in Palo Alto—this ensured that the upstart company stayed close and helped the nascent entrepreneurial ecosystem reach a higher density.

And thus the silicon valley ecosystem was born, far away from the Santa Clara valley and reshaping a university into a crucible of economic growth. But even then, if you had a bright idea and wanted to pursue it, you had to either convince the military or find a rich patron.

The venture capital ecosystem exists to ensure that bets with asymmetric payoffs are made in sufficiently large numbers that the cumulative rates of returns are high. What does it mean? Suppose you had $100 and you wanted to invest it. You're looking for the best return. There's the safe bet where you'll get a tiny extra sum of money with minimal risks, but that's not enough. You'd like more.

So you look at the people who are starting companies and realise that if you owned just a part of their company then you could do very well. But lots of companies also fail. So how do you square these two facts? Simple, you treat your ignorance as taken, and invest in several companies so that even if a few fail, on average you'll still get a great return.

That thought process is the essence of venture capital.

And it is with that thought that people backed companies trying to muscle aside the IBM behemoth. Not necessarily because they all could, but because if one did, that would create an inordinate amount of value. The interesting question became how one company managed to basically own compute for however long it took, with the castle only tumbling once they decided to make it happen themselves.

The emergence of silicon valley was brought about by a confluence of events that combine both technology, the chips themselves but also the hardware and the software, and the business savvy to put them together.

Technological stagnation

In 2005, Pentagon physicist Jonathan Huebner published a study in which he said the rate of technological innovation reached a peak a century ago and has been declining ever since. It created a fair bit of furore, in demolishing what essentially has become a shibboleth of belief in most technological and economic circles. He looked at when the major innovations and advancements happened and mapped them out, showing a declining pattern.

Why should linear growth require, or emerge from, exponential inputs? Because the marginal output has diminishing returns for most processes. Because getting performance growth in anything requires doing more work. Doing more work requires scrabbling atop existing stacks. So when there’s more work that’s been done before, plus there’s lots more work that’s been done, then a new researcher has to spend substantially more time to do something innovative.

In physics, the last period of incredible excitement was in 2012 when the God particle, the Higgs Boson, was measured. But it was predicted in the 1960s. The biggest problems that theoretical physicists have been trying to solve, the quantization of gravity, dark matter and energy, and more, they have been known for close to a century. Innovation itself is slowing down.

There's a fundamental difference between technologies that help you organize things better and increase efficiency, vs ones that provide fundamentally new capabilities. Most have aspects of both, but in differing degrees. The ones that increase efficiency end up creating artificial ceilings and increase the polarisation within the system. Like the famed Fenrir in Norse mythology, when the economy runs out of new innovations to grasp on to and fully explore, it ends up looking for ways to get value from within the system. And that’s a recipe for any system to get more polarised.

However, there is still the argument that the pursuit of efficiency is exactly what creates enough innovative fervour within the sector and leads to the creation of an entirely new branch. Because none of us can know where the fruits lie of a particular research endeavour, but it’s still worth doing something new, because that’s the only way to uncover parts of the world that’s dark.

For example, we all have supercomputers in our pockets. And yet, do we do anything supercomputer-esque? The machine learning that underpins the camera, that allows capturing images to come out exquisite on our phones, it’s amazing! And yet, what’s next? Where is the fundamental new creation that allows humans to do things we could have never done before? To go beyond even the theoretical capabilities, but actively create something new.

We have had the ability to deliver our thoughts instantaneously across space in a way that would seem like magic to our ancestors. But twitter doesn’t seem like the apotheosis of such magic. The form and format of a development should not be represented solely by what exists thus far, but it’s still a shock.

On the opposite end of the spectrum, humanity went to space in the 1960s. And that generation of extreme focus, dedication and intellect that allowed us to set foot on alien soil, the implications practically that led essentially devolve to Teflon. That seems massively underwhelming.

Comparison table

Progress is blind. We can try to focus on building things that are of value, but often we don’t know what we don’t know. Sometimes we stumble upon penicillin, and sometimes we stumble upon Teflon, and sometimes we stumble upon Twitter. Perhaps the only other conclusion is that it is tempting to look back, find exceptions, and identify all the reasons why we were always destined to have an exponential economic growth curve, how progress is inevitable, and how we might not know where the next big bridge will come from, but come it will because it has always come before.

You could subscribe to Ray Kurzweil and believe that this essentially means that we would be all part of the Singularity within a few decades, or be pessimistic that all the low hanging fruit have been plucked. It’s unclear where the truth lies amongst those poles. Or if those are indeed the only two poles.

Looking at Nobel prizes, in everything except Peace, the age of the recipient is trending up. But it’s not a strictly linear increase over time either. It’s more complicated than that. If you look at the age at which the prize winning work was done, it too has gone up by a decade, but it shows a smile effect - you can be too young, but you can also be too old. It’s markedly difficult to win for work done after the age of 50.

So what the gap indicates is not just that it’s the older scientists who are creating breakthroughs, but that scientists need to get older to be appreciated for what they created. The road to proof is longer!

Progress in a field is a function of becoming an expert in the field, which gets more difficult as the field itself expands, of ensuring you have a good grasp of all the work that’s being done in the field, which requires familiarising with all the work your peers are doing at the same time, and of having the right innovative hypothesis. You need all three. The first gets tougher as the field gets more mature, as there’s just more to learn. The average age at which scientists have been doing ground breaking discoveries has gradually increased, from 37 to 47 years. All great discoveries are made on the shoulders of giants, but climbing atop the giants takes time, and effort.

The second gets tougher as the field gets bigger and there’s an increasing number of people in it, it just takes a longer period of time to familiarise yourself with all the literature and be in a position to do the work. Great discoveries half a century ago were made in small groups, or even individually. When Einstein came up with the theories of relativity he was doing physics part time, as a patent clerk, working through and around his day job. And yet today, thousands of scientists band together to solve problems of similar magnitude, they have to work harder than ever. Research group sizes have increased dramatically. And it’s not just that teamwork is important, it’s that the amount of work needed to stay atop the exponential increase in the number of scientists, researchers and so on; it’s incredibly difficult.

And third, which is the least age-dependent perhaps, is to get that spark of inspiration that allows you to break past ossified ideas and to try something new. Inspiration, once properly prepared, strikes as it does. Even assuming it’s a random confluence of events that creates the magic, at least this part of the process should remain relatively untouched. Spending $40 billion on a particle accelerator is not called the road to progress, not because it wouldn't find anything worthwhile. It absolutely might. But it’s because you can only do so many of them. And when the number of experiments are getting smaller while the payoff per experiment is keeping roughly constant, then the progress has to be getting slower too.

Until there’s a new paradigm, a new branch whereby one could work and find fruitful results, the work is relatively ordinary. Within an existing paradigm, as low hanging fruits are eaten, the work involved is much more akin to identification of tools, collaborating on theories, expounding on corner cases of existing theories, and pushing research papers forward. There is some degree to which pre-existing knowledge is required before enough people have the understanding and foresight to create true innovations. At the same time once a field gets advanced enough, it requires exponentially more effort to just get up to date and stay up to date, which reduces the ability to become innovative.

A “smile curve” is the natural outcome. It has to be, and it’s seen everywhere.

Is a startup company most innovative when it’s two people in a garage? Or is it when it’s a hundred thousand? It’s somewhere in the middle. Once there’s enough economic security that there’s room to innovate, but before there’s bureaucracy and coordination costs that reduces the search space.

There’s an argument about the amount of expenditure that happens today on Research and Development. Of how the larger companies, especially technology firms, employ thousands and spend billions in R&D. But what it misses is that not all R&D is the same. Those figures capture the scientists looking to find the cure for cancer, but it also captures the Google engineer trying to add 0.01% more efficiency to its ad distribution model. The gross overall numbers can so easily paint a misleading picture.

And is this something that's limited to science? It doesn’t seem that way. It holds true for most forms of human achievement. In high jump, for example, the ability to take a leap backwards was a shift that transformed the way the sport is played. But once done, within the new structure, progress is once again incremental.

It’s an illusion. Not all growth rates scale with the number of constituent components, but growth numbers still might. In some fields, like arts, perhaps there can only be a few at the top. Only one Mozart, one Shakespeare, one Picasso. But even so, if talent itself were all that’s needed, and not the explosive ability to create entirely new branches of art, doesn’t it feel we should see much much more, at least since there are so many more people striving towards it.

There are always resource constraints behind any process. Perhaps we need to fundamentally reassess how technologies evolve. It’s not about expecting trends to continue forever, but rather the constant rediscovery of new frontiers, each single one probably on an S-curve, but cumulatively creating the apparent view of exponential growth. Diffusion of anything, a technology, a social feature, or a meme, they all spread exponentially once they’re past a critical point, and lead into an S shaped growth curve.

There are a thousand curves plotted that show stylised versions of how these things happen, which all look like different versions of an S curve, with a slow beginning, a rapid explosive growth phase, and a maturity. All physical systems, and most other ones too, eventually hit negative feedback cycles. Either through a form of combinatorial innovation where innovators try to combine parts from multiple fields, or where Metcalfe’s Law shows how the number of interconnections within a network increases exponentially, the new ideas keep emerging. A set of multiple possible sigmoid curves distributed over time could show that the existence and maturity of one curve is what often creates the baseline that enables the growth of the next one.

It’s the growth in semiconductors that led to the growth of the PC that led to the growth of the smartphone. Regardless of the focus on how that progress is distributed, the fact that newer S curves are discovered and disseminated leads to the view, if you step well back so the perturbations in individual curves start to smooth out, that looks a lot like the exponential curve we all like.

And what’s the result? The technology firms have taken over the economic reality. They overwhelmingly control the public narrative, allure for top talent, and the economic power. Just looking at the largest companies makes it clear where the value comes from and where the value is created. It’s no longer those who have large capital assets, invested and depreciated over longer time frames. It’s the ‘intangibles’ brigade, led by tech. Despite only spending around $20B a year on R&D, a fraction considering their cumulative market cap is $5 Trillion and still growing, they’re stratospheric.

The number of public companies is smaller than it was in the 1970s and 80s too. The companies that do exist, as we saw above, are much larger, and over the last twenty years, also older. The ratio of investment to assets is lower, and cash holdings is at a record height. The earnings are highly concentrated, in that the top firms earn more than the rest combined. Looked at from any angle, ‘Big’ is in.

The traditional ideas about capital, innovation and growth are becoming largely irrelevant to this new economy. Sure, ideas still matter, but the problem with ideas is that they lend themselves all too well to a highly unequal distribution in the end. Polarisation isn’t a failure of the system, but the end result.

Breaking through the wall

It looks like a large part of the history, across corporate strategies, regulatory burden, cost disease in certain sectors and the income distribution all lead to higher polarisation. This isn’t as simple as a one-cause disease, it’s a hydra. A large number of the organisations and emergent-behaviours we deal with on a daily basis are polarised, and increasingly so in most instances. The polarisation we see is actually the inevitable end result of a system we've created and maintained.

For instance, manufacturing could be highly beneficial for the masses because it could employ masses to build things in factories. But that's also true of Amazon and Uber, who also employ millions in their warehouses and as drivers. What's different? Is it that the jobs that were done in factories is much closer to value creation? Surely not, since without warehouse workers there's no Amazon. Without drivers there's no Uber (at least as of now).

It could be that the workers before could ask for more wages before, due to unionisation, while now of course now this is highly discouraged. But even if they were unionised, would the wages be high enough that it gives them the ability to purchase healthcare at 10x the price, or go to college at 5x the tuition cost? That doesn't seem true either.

One thing that's different is that the workers do have less choice today. If everything was indeed cheaper, would being a warehouse worker and an Uber driver be seen as jobs at the bottom of the barrel? Probably not, which means it has as much to do with wage constriction as cost increase.

And as for technology, software in particular, as it becomes ubiquitous it creates two immediate and major impacts on the other industries it interfaces with:

  1. It allows higher levels of automation for any process it helps manage

  2. It enables speedier and higher throughput communication

Most of the history of the rise in technology over the past two decades have followed one of these two methods. And both are directly related to ways in which we perceive company strategy shifts in the previous section.

As it helps increase automation, naturally this creates a vacuum in the 'previously employed but now unemployed' category. What's the way out of it? To ensure that those who got unemployed get reabsorbed into another part of the economy (the stone carriers who became quarry miners), or they, with others, become part of another S curve.

So what are some factors that could affect it? We’ll have to explore:

  1. Industries are more ossified and harder to break into

    1. So there's limited competitive pressure on prices in several industries

    2. Without new breakthroughs, this means prices will go up, or automation forces cost reduction

    3. Regulations make it harder to start things up

  2. Dominant technological trend is towards automation and knowledge management

    1. This leads to hollowing out of the middle as humans get replaced with machines

    2. Increased hyperspecialisation creates dissatisfaction amongst employees, and easy replaceability for the employers

    3. Bargaining power also gets hollowed out except for firms that are growing - others have only one option, to cut costs

And we’ll explore some ways to bridge the gap later on, but it might require some combination of the following:

  1. The median wage for jobs need to get pushed up - there's a supply-demand dynamic where the bargaining power for labour needs to re-emerge

    1. Better unionisation to ask for more wages

    2. Better regulation to control price increases

    3. Bargaining power of companies need to be tamped

      1. There needs to be more companies coming up

      2. There needs to be a reduction in company size and market concentration

  2. The costs for some of these basic necessities need to be reduced, or at least grow at a more sensible rate

    1. Regulatory burdens need to be reduced

    2. Competition needs to be re-introduced

      1. There needs to be more companies coming up

      2. There needs to be a reduction in company size and market concentration

    3. The underlying system needs to be made less complex since it adds a high coordination tax

Breaking through this stagnation to create a growth scenario requires an understanding of what actually creates growth. And a cursory understanding of exponentials helps us realise that growth only comes from increasing inputs.

A model I created to assess, simplistically, how growth could be spurred through technological improvements but constrained by number of people required, showed some interesting outcomes.

  • Number of people needed to make progress grows as amount of information needed to keep atop the world grows

  • This creates increased communication and coordination friction

  • The friction can be reduced through better communications technologies, but of course not perfectly since it doesn't impact the "capability per brain"

  • Technology improvements grow at a steady rate!

Put all of this together, and it shows that the growth rates we're likely to see tends to slow down as long as we're working inside any individual paradigm, and we need to connect a whole bunch of dots to break free to create the next one.

But in the end, there's only one option, emergence of a new S curve. One that can employ large numbers of people without need for high degrees of credentialing, at good wages, and which is scalable to become applicable to a large enough fraction of the human population.

There's no shortcuts to this, but while we're waiting for it it might be a good idea to widen the base who can help make this come about.