Nobody Knows Anything

on living with ignorance

I have not published much of what I wrote the last month. This wasn’t due to lack of writing, but because I got extremely busy with a forthcoming major project. Some of you already know, and this will get launched properly soon! But meanwhile.

I.

As I’m beginning to work on some knotty problem, I start to think about something that I think about fairly often as it is, which is how nobody knows anything. In almost any of these kinds of endeavours with respect to human life, if you think you actually know something about this, you probably don’t.

Almost every piece of advice that you hear is basically entirely filled with survivor bias. What that means is that you probably can’t take any of them at face value and you probably can’t follow any of them without effectively just trying to follow someone else’s beaten path.

Now, this is not, you know, strictly speaking, always true for everybody because there is a large number of paths that have high beta. Basically, you can follow someone else’s path to become a doctor or to become a lawyer or basically do any kind of profession. In fact, a large part of success in life is effectively when you are able to take something that is difficult for many people to do, but possible for you to do, and try to make that your success criteria.

But if you’re trying to do something different, if you’re trying to succeed in some sense on your own terms, then there is almost no chance that you are going to be able to take someone else’s advice and follow along. Or at the very least, not without it trudging into such problem that at which point might not have bothered in the first place.

It’s a conundrum that’s not really talked about much at all, especially in an era that is as filled with advice as our own. For example, most of the interesting questions are basically like this. They’re not questions that anybody actually knows how to answer because the knowledge of how to answer is what makes questions become less valuable over time.

Whether it’s questions about the future or whether it’s questions about family or whether it’s questions about what one should even do ... They’re all like this.

Quoting myself from before.

Jeff Bezos, perhaps the best manager of the century, said a key to his success was “Disagree and commit”. You must have had teams you worked in to do projects during your time here. And if they were anything like mine, there would have been a lot of arguments.

Bezos says that at some point, when arguing back and forth is no longer useful, you should just say “I disagree with you, but I commit to doing it your way”. And then the chips fall where they may. No recriminations, no “I told you so”.

It’s great advice, right? Jeff Bezos created one of the most impressive companies of all time, presumably by following through on his advice. It’s about as valuable as advice gets. It’s constantly referenced in speeches, in books, in articles. A true piece of management history.

Now, Steve Jobs, also no slouch when it comes to building iconic companies, was asked this question too; how does he manage conflicts?

He said something different. Which is that his job, and the job of senior management, is to make the absolute best decisions they can for the company. And, he said, being humans we naturally are less willing to fight to get to the truth, or do our best work if we don’t think what’s being done is right. And knowing what we’re doing is right is crucial. In other words, he thinks “disagree and commit” is both intellectually dishonest and impossible to make work.

So which one do we believe? Which one do you follow?

…

II.

The fact is though that even though we don’t know much about the right ways to make decisions or the right things to do for success, and yet we live in wondrous civilisation. We have whole cities and medicines and semiconductors and a dizzying array of choices for anything our hearts desire. How?

There’s this quote I love by Alfred North Whitehead.

Civilization advances by extending the number of important operations which we can perform without thinking of them.

To live in civilisation however is to live in ignorance, of exactly the type we’re stuck in from listening to advice. Almost everything you do, or care about, or rely upon, is built by an incomprehensible array of people and efforts often going back centuries. And we can’t know its full extent even if we wanted to.

There’s an old essay, called I, Pencil. If you haven’t read it already, you should do so immediately. It’s an autobiographical story from the perspective of a simple lead pencil, and explains just how staggeringly complicated it is.

I, Pencil, simple though I appear to be, merit your wonder and awe, a claim I shall attempt to prove. In fact, if you can understand me—no, that’s too much to ask of anyone—if you can become aware of the miraculousness which I symbolize, you can help save the freedom mankind is so unhappily losing. I have a profound lesson to teach. And I can teach this lesson better than can an automobile or an airplane or a mechanical dishwasher because—well, because I am seemingly so simple.

Simple? Yet, not a single person on the face of this earth knows how to make me. This sounds fantastic, doesn’t it? Especially when it is realized that there are about one and one-half billion of my kind produced in the U. S. A. each year.

From the cedar wood that gives birth to the large quantities of machinery to cut it down and mills to process it and ship it to where they can be cut and processed, and the lead itself which is a marvel of material science, and all the work that went into discovering it and mass producing it starting from mere clay, and so on and on.

Everything we see or use or is a tip of some vastly incomprehensible process to which we might have a tiny bit of input, and if we’re very lucky an opportunity to shape, but the overall movement is not understandable by anyone.

(This is also why we can’t just ship someone or even many someones with many things to Mars and hope it would just work. Yet on earth things seem to work as if by magic!)

This magic is basically because we have given up the control that would come with understanding how to build a thing to an immensely complex decentralised computation machinery, which is the market.

This isn’t just a modern affliction. Frederic Bastiat asked his readers to consider how quickly Paris would starve if provisions stopped flowing into the city.

On entering Paris, which I had come to visit, I said to myself—here are a million human beings who would all die in a short time if provisions of every kind ceased to flow toward this great metropolis. Imagination is baffled when it tries to appreciate the vast multiplicity of commodities that must enter tomorrow through the barriers in order to preserve the inhabitants from falling a prey to the convulsions of famine, rebellion and pillage. And yet all sleep at this moment, and their peaceful slumbers are not disturbed for a single instant by the prospect of such a frightful catastrophe. On the other hand, eighty departments have been laboring today, without concert, without any mutual understanding, for the provisioning of Paris. How does each succeeding day bring what is wanted, nothing more, nothing less, to so gigantic a market? What, then, is the ingenious and secret power that governs the astonishing regularity of movements so complicated, a regularity in which everybody has implicit faith, although happiness and life itself are at stake?

Or as Hayek mused about a new use of tin being discovered, and how it would trickle through the economy until the “correct” calculations are done and everyone adjusts their use and need to rationalise for the new price and source of demand.

The most significant fact about this system is the economy of knowledge with which it operates, or how little the individual participants need to know in order to be able to take the right action. In abbreviated form, by a kind of symbol, only the most essential information is passed on and passed on only to those concerned. It is more than a metaphor to describe the price system as a kind of machinery for registering change, or a system of telecommunications which enables individual producers to watch merely the movement of a few pointers, as an engineer might watch the hands of a few dials, in order to adjust their activities to changes of which they may never know more than is reflected in the price movement.

As things get more complex, more hierarchies got introduced, and we started getting specialised firms and whole industries which would comprise parts of the overall system. Nobody can understand which ones would come or which ones would emerge, we can only see from within the superintelligent structures we’re a part of, even as they help build what our collective will demands.

III.

We live surrounded by various superintelligences. Immensely complex, decentralised, computation machinery whose overall functioning is not understandable by anyone. Markets, firms, states, laws, even collective norms, all are akin to this, all infinitely more powerful than any of us can fathom and inscrutable to almost all of us almost all the time. We live with their power and their excesses, for better and worse, and spend inordinate effort aligning them to principles we think good.

Ashby’s law of requisite variety, out of cybernetics, says a regulator can only control a system to the degree that the regulator’s own repertoire of responses is as varied as the disturbances the system can produce. All we can do is govern this from within, with simpler tools that let us measure parts of the system and make course corrections, even though it’s not a replacement for perfect understanding.

These superintelligences were painfully built up over years, decades, sometimes millennia, and they evolved as a result of what we want from them and our interactions with them. The history of civilisation is a history of how we went about controlling these superintelligences as they grew bigger and smarter.

In genuinely open-ended domains, which is most of them, nobody possesses reliable recipes. We succeed despite the knowledge problem. And the way we control our ignorance and govern these megafauna is through decentralised cognition. Continuous error correction, a focus on incentives, and an evolution towards worlds where nobody knows anything, but we seem to be able to make things go right.

Now though, we have new superintelligences being born, much like companies, from within the myriad AI labs. Each agent is akin to making every user a CEO, able to ask and command one. A capable, opaque agent that’s comprised of processes no human fully understands, pursuing set objectives in ways not easily specifiable, and requiring continuous governance from outside. They create a world where everyone has access to multiple “companies”, today quite small but maybe soon Fortune 500. To live in that world requires a different theory of corporate governance, especially since these “firms” might only exist for a short time.

And much like real life, they come in all shapes and sizes. It could be Novo Nordisk, or it could be Enron. It could be General Mills, or it could be Monsanto. Right now they’re not all that different, but the CEO only has so much control over them, and the firms themselves have signed onto some external codes which might or might not constrain them. The two ecologies are not really the same, they might be compatible, they might be competitive, the equilibrium is yet to be reached.

But the theories on how we deal with them shouldn’t change. Ignorance is not a temporary defect that we will eventually eliminate. It’s the permanent burden of increasing specialisation and living in a complex world. Building institutions to coordinate our partial knowledge and govern immense systems from within has been our greatest achievement. That’s how we already live beside, and within, superintelligences.