The Problem With Experts

On the need for a transitive property of trust, at scale

Ego, by midjourney

I - The Story

We recently got adventurous at home and decided to get our kitchen and living room redone. To get this crazy task accomplished we decided to hire someone far more qualified than us to do it. And being the smart sensible people we are, we auditioned five builders, carefully asked each one all our questions, evaluated their answers and hired the best one.

Guess what happened. He switched materials from the one we'd explicitly agreed on, hired an electrician who has never laid floors to do our floors one day before installation who of course screwed it up completely, delayed the project by around 2x the time, forgot to put parts of the wall before the appliances, and generally made our life hell. On top of this, his team dented our oven, broke several drawers, forgot to screw in electric sockets, put live wires around in a house with kids, and raised hell when asked about any of these¹.

And later, on us sending the contractor a list of things outstanding, he threatened to invade the house and break up what he’d built unless he got a payoff. (Parenthetically this also resulted in a crash course in how British police work as they were extremely polite and extremely unhelpful - essentially stating they can’t help or show up or in fact do anything until something had happened. Which kind of defeats the purpose of preventing something from happening.)

Anyway. This seemed a suboptimal experience, but despite its seemingly unusual characteristics it seems incredibly common. In our same street we have 3 other horror stories with builders like this, and in our friend circle several more!

All of this together made me wonder, how could this have been prevented?

II - A Bigger Problem

A key aspect of buying almost anything in modern life, whether that’s builders or education or financial services or medicine or pretty much literally anything else, is that you can’t quite know what you’ve got until you get it.

Things are a little better with products, since the control seems to be slightly more centralised and standardised, but with services we truly seem to be out on a ledge with hope as our guide. Because with services you not only get a thing, you get an expert along with it. In products we solve this through certifications of the product to assure quality. The FDA does this for instance in the US, and recently helped recall things from pet foods to calendars to probiotics to beef.

I do like that someone is looking out for me not getting salmonella.

But when it comes to services, where we have tried to duplicate this through certifications and reviews, they are still incredibly buggy. As Dan Luu wrote with the problem of finding good accountants.

You can try to get better quality by paying more, but once you get out of the very bottom end of the market, it's frequently unclear how to trade money for quality,

The same problem plagues doctors. For instance, my wife had an ulnar nerve palsy at one point that three specialists and [n = too damn high] GPs couldn't figure out. They would see her, handwave some tests and vitals, and send to someone else.

I, who among other things am famous for not being a doctor, had to read research and find a solution that didn't involve “come to the hospital for us to take your vitals and prescribe something half assed like rest, paracetamol and fluids”. And then browbeat the doctors with my diagnosis, argue against them dismissing me for my uppity nature, and have them run tests.

They’re happy dealing with the easy situations where prescribing paracetamol and rest actually works, or where prescribing the first antibiotic that comes to mind actually works, instead of doing the work that would tell you you were actually allergic to that one. Or ignoring a tear behind the patella because of a basketball injury that permanently damaged the person’s ability to be athletic. All real stories.

George has plenty to say about this topic.

To hire accountants, it’s largely the same. To hire lawyers, the same. The only way out seems to be either to leave it all to God, or to become an expert yourself.

The clear implication from the stories seems to be, I’m the expert, and I’ll do what I need to do. You just sit there, and I’ll get you what you need. Note, it’s what you need, not what you want, or even what you said you want.

No matter which type of service you think of, the pattern repeats. The experts are an enormous pain in the ass to deal with, and getting what you want is hard.

III - But ... why

The normal methods we try to solve this problem with are:

• Crowd wisdom i.e., Reviews by other users - highly gameable, as any Amazon purchaser would know

• Own wisdom i.e., Due diligence - also gameable, as I now personally know

• Contractual - incredibly hard to actually enforce

• Regulatory - incredibly bureaucratic and cumbersome

The problem is that the first two have holes large enough to not easily be pluggable, and the last two are almost impossible to enforce except with big outlays of cash and effort.

In daily lives we give up mostly to the regulatory sector, like with the products above, and say that things need certification and standards to be applicable in a particular context. And beyond that we just kind of give up and hope everyone does their best. Because its impossible to extend to cover all eventualities of anything even remotely complicated in execution.

But what’s interesting is the realisation that when push comes to shove, rules aren’t nearly the protection we want it to be, and norms are all we have to fall back upon.

And since the rules exist only to create a sort of outside boundary, what we’re left with are just norms. And usually just two norms to prevent anarchy.

• Of honesty - whereby you actually tell people before you take a decision, to get their input, or after something happened, so we can rectify it together

• Of conscientiousness - of wanting to do a job well, trying your best or otherwise discussing things rather than trying to cut every corner you can possibly find

Norms, however, are like rubber bands. You stretch them too far, they snap. You stretch them too frequently, they deform and eventually break. You give time between occasional stretches, they regain their strength. (I don't have a simile for micro-tears in the material, but feel free to substitute your own)

In bad cases, norms are shackles. They are vestigial remnants of the "ways we have done things in the past", and ripe for getting thrown away. In fact, a large variety of business and economics literature, not to mention blogs, like to identify these topics and self-importantly declare the benefits of clear thinking and progress.

And yet, they seem to be the glue that holds it together. The builders who we did get in the end did their work because they were conscientious. The doctor we actually do talk to happens to pass the boundary of trust, to actually listen and respond to what we’re asking. The accountant we actually use is amenable to discussion and questions. All reached after years of painful trial and error, and all highly fragile states.

But the fact that norms are being broken by a few starts us down a path where they’re broken by many, creating a general dissatisfaction with experts everywhere. There is just zero trust in the system.

IV - The Barriers To Character

So why isn’t there trust? Why do we have to hope and pray or promise furious vengeance to get something done?

The first bit is ego. The impossibly high horse that experts sometimes sit on, lest it turn out that others can also do parts of their job well.

The second bit is the complete absence of any basic level of thinking-ahead. Of planning. The sort that is easily taught and makes literally everyone better off. The barrier to creating trust is so goddamn low here it's unbelievable.

Mostly though this seems to coalesce in a situation where the expert feels threatened regarding their status. After all, if we plebeians encroach on that sacred territory, what else would we find!

In space after space after space we seem stuck with these problems. “Trust us” they say, while doing absolutely nothing to earn that trust. You can only really do it if you trust them to be honourable.

The right way to pick an expert is as much a check of their character rather than institutional trappings that prevent them from defecting. When you don’t know what you’ll get when you’re buying something, the best you can hope for is that you get something close to what you wanted.

To figure out how to solve it, I asked a different expert. And as GPT-3 told me:

We can't rely on experts to keep the norms because they're just as likely to be the norm breakers. In fact, the more expert they are, the more likely they are to cheat the norms. So, we need to create a way for the market to create norms to create the trust necessary for the expert market to work. We need to create a mechanism that allows the market to build trust around experts

It’s a fairly simplistic libertarian credo, but it ain’t half bad. It also added:

I think you either need to be an expert yourself, or have a community that is small enough to have credibility, and open enough to share that credibility.

This is why I love Google's search algorithm, by the way. It's a huge, complex system built on a ton of simple rules. No rules that are a "secret" though. You can) pretty easily find out what they are. And they are complex enough to not be able to be gamed. But simple enough to be understood.

As soon as you introduce secrecy, the game is on. The net result of all this is that it is extremely difficult to make a system where experts are picked in a way that is honest and trustworthy.

Despite its misplaced love for Google’s search algorithm, I tend to agree. We do need a community that’s small enough for folks to build credibility with each other, and yes it’s a hard system to create.

V - Conclusion

The commonality amongst the stories upfront wasn't that people are dumb, or that we got fleeced. It's that despite due diligence, being reasonably savvy consumers, and not penny pinching, we consistently get horrible service on a regular basis.

And we're not alone. I literally know zero people (net worths between 0 to 10 figures) who don't have this problem. Zero.

What's the solution here? Or rather, is there a solution? How do we actually build a trusted network here as GPT asks?

I think there are three possible answers.

One option is micro managing the experts, which seems counterproductive to literally everyone's sanity, but also the only current way² to solve the issue of people not doing things without putting any thought into it.

This, by the way, is also the kneejerk public sector response. Oh the executive branch did something dumb, or that seems dumb? Let's create rules around that. Let's create oversight committees. Oh the actions in Congress didn't address [pet issue]? Sure let's throw the entirety of the fourth estate and the whole of the online world into the mix to judge things ad-hoc.

The reason this is suboptimal is because this is like judging a codebase by critiquing each line of code as it's written. Ideally never to be done, or only to be done when you actually are good enough to know when not to interrupt. But alas, the only way to get a bit of prevention in the picture to cure the disease.

The second is to have some basic level of communication and management training. For instance, the crazy thing from our particular experience upfront was how preventable the problems were! All it required was a conversation or two, where (from history) there would’ve been a majority chance we would have listened to his suggestions anyway. But instead, as in the fables, a lie begat another lie begat another.

Because one way to bridge the gap of expertise and inevitable divergence in what is done vs what was needed is to actually communicate about that gap. To do a complex piece of work after all requires you to think about what needs to be done, the steps to do them, the ability to convey that to others (employees, customers, subordinates, vendors) and in general to do some basic level of management.

This holds true for renovating a house, as for helping someone think through a diagnosis, as for thinking through implications of tax selections, as for figuring out the impact of specific covenants in a legal agreement. All of which are real examples from last week I can’t believe I had to step in to answer.

The solution is to have the absolute basic aspects of common sense be inculcated in a job being done.

The third is what Adam might call a version of trust windfall. My parents, being from a different generation and ethos, like to solve their problems though a phone-a-friend routine and an implied model of transitive trust. Even today that’s their preferred mode of solving every problem, including the eminently solved problem of booking flights.

The drawback here is that the networks aren’t mobile, and the local knowledge you build up is highly fragile. And yet it seems the most actionable.

But this works because the norms of screwing over someone who’s known within your social circle is limited. For everyone else, living in a larger society, these norms are broken on a regular basis! Funnily enough, at scale it becomes crowdsourced reviews again, but its perhaps the closest we get to a personalised service.

Still, it feels remarkably awkward that our methods of identifying expertise or developing comfortable plans seem to be so flimsy that no money or power can buy it in sufficiently strong quantity.

I genuinely think that this is the equivalent of a trillion dollar bill that's lying around on the floor. It's like restaurants before someone started doing health checks to make sure there isn't poop in the hummus. “Trust me and shut up” is a silly business strategy that only seems to exist because until recently trusting folks blindly was the only option. But in the world where transparency of outcomes is a thing, and research is a thing, and the ability to get second opinions is a thing, I can only hope someone picks the bill up.

The fact that conscientiousness exists only as a norm means it’s also fragile to awkward nash equilibria for defection.

The benefit in the olden days was we had a community which could create and enforce networks of trust within us. In the modern world, of a scale that’s orders of magnitude higher, this is much harder to do.

1. When we asked about literally any part of it, a typical response was:

I’m really sorry but I’m growing really tired of every single thing we do being questioned.

The Klima Matt is absolutely fine and and has good reviews, it was laid by professional electricians and has been used by them many times before, it was not cheaper or anything like and I can tell you now I will not be paying for the replacement of it.

I have been lenient to everything you wanted of me but now this job is stalling because everything I’m doing is being questioned and second guessed, I have been doing this a long time and I have far more happy clients than unhappy.

2. But Rohit, if you do this for everyone that's a colossal waste of time and you'll end up spending all your time explaining the job to idiots!

Well yes, not to be an idle Kantian but not everyone needs the same reassurance and the conversation will vary. We're a long ways away from experts having knowledge that cannot be named, and all jobs are at least as much about trust as anything else. Especially in jobs where the probability of success is as much about how happy the person receiving the service is as anything else.

Is this applicable to everything? What if you need open heart surgery?

Even in that case I’d argue you’d do well to get multiple opinions, reading up on why you need the surgery and what for, discussing the approach and recovery in detail. You don’t actually need to guide the hand that holds the scalpel, but there’s huge chasms of errors to be avoided everywhere.

Are you saying everyone needs to have like an MBA and spreadsheets and charts to do things?

No, I'm saying telling your client or patient “this is what we should do” before doing it is a basic courtesy. I'm saying trust is earned and isn't blind. I'm saying acting like an entitled idiot isn't likely to make your own job easier, much less anyone else's. I’m saying thinking about what you need to do before you need to do it is almost always a decent idea.