Last week we talked about the importance of experimentation to get to the future, but how do you know what experiments to run? This essay’s about that question.

I. What drives you to do anything

To do anything is an act of faith. To choose a future state that you believe in.

But how can we choose what to do?

In the old movie The Graduate, Dustin Hoffman was told to go into plastics. It was good advice for 1963. Since then plastics industry went on to grow so much that it effectively saturated the world by the 2000s, growth petering back down to global GDP levels. This is an extraordinary achievement.

And yet, is the advice good enough? What if he’d gone to semiconductors? Or gone to IBM instead, worked on the System 360, and later moved to Microsoft?

There are so many paths he could’ve taken, even assuming he didn’t have any well hidden talent for plastics.

So imagine you were him. How would you make this decision? Would you choose plastics? Oil? Semiconductors? Electronics? Banking? How are you supposed to make this decision?

The natural world solves this problem elegantly, as it does most problems. If you were a bee, and trying to figure out where to go to get honey, you make a mathematically near perfect choice. Bees, once they find a flower, tend to exploit it in proportion to the nectar it has. It comes back to the hive and does a waggle dance, which tells the other bees what it found with the duration, angle, direction from the sun and more. If it’s a lot, the hive jumps in and goes en masse to the flower. And as the nectar declines, the other bees see the waggle dance, and start to explore elsewhere. They exploit when they find a large trove, and explore at other times.

To do this well is difficult, and is the explore-exploit dilemma. We’re taught to use the time in school or university to explore, at least a little bit, and use almost all other available time to exploit. This dilemma is one of the most fundamental problems in decision sciences, intractable with an easy answer. Unlike the bees, the answer isn’t easy or straightforward. The rewards aren’t measurable in ounces of nectar. And the difficulties aren’t simple enough to be solved with a waggle dance.

Or, imagine you're looking to start a company. You go and tell your friend, and being a good friend they say, “did you know that the success rate is like 0.5% for venture backed companies?”

Which is true, but also unhelpful. That 0.5% fluctuates dramatically based on your background, knowledge, the idea you’re pursuing, the team you got around you, and more.

In this vein, where someone asks me about career thoughts, and I ask them what they like doing. I’ve had the conversation a hundred times in my life. The results are usually a slightly blank stare, followed by something usually derivative or generic.

Because when someone asks you what you like, that answer is usually not amenable to easy answers. And for most lines of work, you’re so unaccustomed to hearing that question that the very idea it would be important feels impossible to grok. The expectation that you can even plausibly have an answer stands in the way.

To answer these questions, you have to know how to balance exploration of possibilities with exploitation of something concrete.

The best tool we have is our curiosity. And it feels we’ve completely lost the place for curiosity as core to our search.

II. Whence curiosity

We’ve been obsessed by curiosity for quite a while. There was a rational analysis of curiosity, which proposed curiosity is driven by wanting to maximise ability to make appropriate responses in the future. The major theories they talk about seem to think of curiosity as a) novelty seeking, b) information gap, and c) learning progress.

It’s not particularly compelling, since it does try to derive an optimal solution of an agent in an n-stimuli optimisation problem, and relies on the agent predicting perfectly the changes her knowledge would have based on the stimuli.

There’s also the fact that we do have curiosity, guided by evolution. Infants learn by making up their own activities after all, and are intensely curious. To the extent half of an early parent’s life is spent just making sure their curiosity doesn’t kill them. The study though was fun, using robot learners who explored the hypothesis that progress in learning generates intrinsic awards.

There’s also the more interesting academic work, done in the computer science domain. Here the analysis of the role of curiosity comes up often linked to the idea of whether rewards ought to be extrinsic or intrinsic. Extrinsic rewards, like money in the real world, is often seen as a powerful motivator in the real world. And it is. However it’s not common everywhere. In one examination of this in the world of computer vision, we see:

In many real-world scenarios, rewards extrinsic to the agent are extremely sparse, or absent altogether. In such cases, curiosity can serve as an intrinsic reward signal to enable the agent to explore its environment and learn skills that might be useful later in its life. … Our formulation scales to high-dimensional continuous state spaces like images, bypasses the difficulties of directly predicting pixels, and, critically, ignores the aspects of the environment that cannot affect the agent.

The idea is of course that the research works very well in VizDoom and Super Mario, and therefore should apply to real life.

A competing paper does show that this by itself is insufficient in the world they created to verify hypotheses using reinforcement learning.

Even in creating worlds, curiosity can serve as an intrinsic reward signal. The agent gets to explore its environment and learn skills, which might be useful later in life.

Embracing curiosity also supposedly eliminates the exploration-exploitation dilemma.

Through theory and simulations we prove that explore-exploit problems based on this can be solved by a simple rule that yields optimal solutions: when information is more valuable than rewards, be curious, otherwise seek rewards. We show that this rule performs well and robustly under naturalistic constraints.

We should start applying this to the AI paradigm.

III. Confusing worldviews

Let’s take a step back. Imagine you're an investor, near starting out. You're trying to figure out your method. So you start reading biographies and videos and textbooks.

And turns out the lessons are all over the place.

Let your runners run, or maybe weigh your investments equally and rebalance your portfolio. Focus on stocks that are cheap, the value stocks, unless you should focus on momentum and growth stocks. And for the companies you at, you should expand into other adjacent product lines, except when you should focus on your core competency. You should focus on your particular competitive advantage and specialise, except when you should vertically integrate to control your supply chain.

It seems the answer is, you should do what you need to do to win. And what you need to do to win is as variable as the world is complex.

I’d asked a few weeks ago about why there isn’t a philosophy of business. I got a fair few responses that spoke about the existing few schools of economics, mainly Coasean theories of org formation and papers on incentivisation, anthropology on how people work inside companies, and pleasingly even a few on the similarities to ecology, like the growth of slime mold and implications for business niches etc.

All of which was hugely helpful, but descriptive, whereas it seems the answer is simpler. The secret of business is to try things until you win.

When we do try, and let our curiosity run, we find incredible things.

Normal Joseph Woodland was a graduate student in Philadelphia and worked at a grocery store during his summer break. They had a problem, how to actually track products, a major problem. He started experimenting with all sorts of ways to encode information. Ink, performations, electric impulses. Eventually, he figured a series of lines of varying widths would be great to get scanned by a sensor, and called it Bull’s Eye. He patented it, but took another three years of tinkering, and that’s what ended up becoming the “bar code”. It took another decade for it to catch on, but the impulse to tinker, to follow his curiosity, and the leeway to try and figure this out started while working at a grocery store.
When a young engineer got frustrated with sharing documents while working on a large public project, he looked at the things lying around, and found an older tech called Hypertext. So he spent a summer writing a language to use it, and a browser to navigate through it, and created the World Wide Web. Tim Berners-Lee, a British computer scientist, while at CERN, ended up creating something utterly extraordinary because he spent time noodling around solving the problem of how to get scientists to share their documents with each other.

That’s the benefit.

IV. What’s next

And it’s changed. There was work done in 1991 that showed that recent books started showing curiosity in a negative light. In children’s books! It’s from a low n, reading and assessing 210 books, but as a data point it feels instructive.

For much of the important decisions we seem to need to make in life, there is no roadmap. There is no clear path to success. Which is what requires what Taleb calls a flaneur. A curiosity driven observer who learns skills of interest, rather than a fervent optimiser.

As is said: "Most of the breakthrough discoveries and remarkable inventions throughout history, from flints for starting a fire to self-driving cars, have something in common: They are the result of curiosity. The impulse to seek new information and experiences and explore novel possibilities is a basic human attribute.”

But, in a damning piece of self reported evidence:

In a recent survey I conducted of 520 chief learning officers and chief talent development officers, I found that they often shy away from encouraging curiosity because they believe the company would be harder to manage if people were allowed to explore their own interests.

Or

Research finds that although people list creativity as a goal, they frequently reject creative ideas when actually presented with them.

So can we really blame the world when it turns out that our systematic focus against fostering creativity has had highly predictable results?

Again, this is not universal, nor is it a death knell. Useful correctives exist, dotted around in some smaller startups and some larger organisations. But it is present, and it’s perverse, and it is pervasive.

V. The curiosity theory of everything

There’s another famous problem, also set in a casino. Imagine you walk into a room full of slot machines. How do you decide which one to play, since you obviously don’t know the best machine to place your bets on? Well. You try pulling levers at random, keep close attention and see what works and what doesn’t. This is called the multi-armed bandit problem.

It works when you’re dealing with slot machines. It’s outmatched once the machines start giving irregular responses, when the rewards are no longer easily calculable, when the number of machines themselves change over time.

Sadly, for machines and us, our lives aren’t so easily circumscribed. We can’t create simple solutions like the secretary problem to figure out when to stop looking for something new or choose the best of what we’ve seen.

Real life doesn’t come with a simple terrain or easy to grok rules. You don’t know if you’re playing slots or blackjack or underwater cave diving. The number of options at any point are too large to comprehend, and the various paths your choices could take you infinite.

What we have to help us make this decision, almost always, is our inner sense of what’s interesting and what you’re curious about. That’s the key to narrow down the options. That’s how you get to define the terrain you then set about exploring and experimenting.

Some of this is due to the fact that the modern world rewards differentiation, as Packy wrote so well last week, and differentiation comes from following at least something unintuitive to everyone else, but where you’re guided by something inside you. The reason it feels so off when I’ve said it to others is that it feels too much like new age woowoo philosophy. It’s not. It’s not about eat, pray, loving your way towards happiness, it’s about using your curiosity as a bellwether for where to explore.

As John Fio said in his conversation with Patrick on invest like the best:

The things that are going to be valuable are the things you can’t teach or copy.

Most things in life don’t come with a recipe to follow. It requires us to use some judgement and make a decision without knowing how it’ll turn out. The root of almost everything interesting you see around you stems from someone letting their curiosity off leash.

Which also, scarily, means that our ability to guide a unique path is pretty much entirely up to us. Most times we’re stuck in an explore-exploit dilemma. Should we do something that clearly maximises reward, especially in the short term, or should we try to jump off a cliff and build our wings on the way down.

Ask your favourite reinforcement learning algorithm and they will tell you that the reason it occasionally does random actions is to more effectively search the whole latent space. We can create rules - only explore for a particular percentage of time, or explore heavily upfront and then stop, and so on.

Writing here, this is an act of exploration. In a world with infinite choices, how could it be anything else!

Strange Loop Canon

The curiosity theory of everything