Aligning Anthropic

The Department of War is angry at an AI lab

Last week was a bit crazy. In many ways, but specifically with AI. For those who were blissfully unaware, The Department of War picked a fight with Anthropic over the ways they were allowed to use the model. The fights, as is often the case with the administration, got nasty. Anthropic said no we won’t budge, DoW got angry, and threatened to cut them off and declare them a supply chain risk. A few hours after, OpenAI said they managed to get another deal, apparently a better deal, and one such that any other AI lab can also avail the same terms.

So naturally everyone is angry. Anthropic is angry because they were declared an SCR. DoW is angry because someone tried to force their hand. OpenAI is angry because everyone seems to call them opportunistic ghouls, more or less. The media, both independent and institutional, loves it because they get to play their favourite game of good guy-bad guy.

I really didn’t want to write about this. But it is important, contractual disputes are actually interesting, and sometimes that deserves an explanation.

The facts are roughly as following, Anthropic had an agreement via Palantir to work with the DoW. They’ve been doing it since mid 2024. They made an different, supposedly unsafe version of Claude to do this. Somehow over the last week, they got into a tiff with the DoW, supposedly over some red lines they had (no mass surveillance and no autonomous weapons) or rather who will get to say what those lines are and when they’re crossed. OpenAI signed a contract which had those same red lines and an enforcement mechanism.

Now, the claims are roughly as following, noting that nobody knows if they’re true. Anthropic asked questions about the Maduro raid where it was used, and the DoW got upset. DoW asked a hypothetical about how to do autonomous missile defense using Claude, and got a non-answer that they’d need to talk to the CEO and they’d ‘work it out’. Anthropic asked for their red lines to be enforced by enabling them to act as the party to approve it (you’d ask them if you had a question). DoW wanted language referring to “all lawful use”, basically saying if what they’re doing is legal you can’t tell them what to do, especially during operations, i.e., you can’t tell them to stop doing something in the middle of an op. OpenAI said sure, we agree to all lawful use, but note these specific laws and regulations, and we will control the deployment of our models, using our people, since we know what it can and cannot do, and help you guys out.

Every point above is a claim, and we have no real proof. People are desperately trying hermeneutics of the OpenAI position and blogs, but honestly it feels kind of silly since we simply don’t have the data to conclude they did a bad thing. Or, particularly silly, that they defected in a prisoner’s dilemma. What we do have, are concerns. Concerns like:

• Didn’t OpenAI just accede to “all lawful use” and therefore allow mass surveillance on Americans?

• How can you let a private company tell the DoD, you should ask us if you’re violating any of our red lines during an operation?

• Why did OpenAI sign an agreement so fast anyway, surely they just said yes when Anthropic said no?

• What do those red lines even mean?

• Also, Anthropic and OpenAI seemed to have the same ones, how can that be?

• Can’t the government or the DoW just make up its own laws as it does anyway? Who can stop them?

• How can you guarantee this means the DoW won’t cross any red lines?

• What do technical safeguards mean, how are they enforceable?

• Etc…

Many valid questions, but I refer you to the openai blog, dario’s written statement, and Sam’s AMA for various points of view on them. They do cycle between thinking of the government as Leviathan, an entity you cannot negotiate with, only appease, and thinking of the government as Loki, a trickster you need to subdue or overpower.

My interest though is broader than who said what to whom, or who’s virtuous and who’s not, as I think yours should be. It’s not to relitigate the facts, but think about the following:

1. What are the right safeguards to put in place when a piece of technology is deployed as a tool by the DoW?

2. How do we enforce any of it?

Let’s think about this for a moment. Imagine you are dealing with the government for a moment as an AI lab. They want to buy your AI, and you want to sell it. How would you safeguard it?

You know that plenty of things are legal, but not “good”. So what’s the choice here? You could of course just try not to deal with them at all. But once you decide to do it, there’s either you need contractual provisions you think they would adhere to and execution guardrails you can have some control over.

You also know that plenty of things are legal, but impossible. You cannot build a stairway to the moon regardless of the fact that it’s legal. Saying “I want GPT to build my defense strategy in Iran” would be such a thing to ask, you can ask it you won’t get good answers. The AI labs both want to say that.

So, you have to write some provisions into the agreements. Of course, the DoW can buy anything it likes, and you can add constraints on the stuff you’re selling, but they have to be clear. This is true of all contracts but of course with defense it’s even more important. For the same reason that it’s important in a hospital. To take a silly example, most models will rightfully have safeguards against violence or nudity, but imagine we also need them to treat burn victims. It can’t be a blanket no, you need to figure out some way to separate what’s allowed from what’s not, and before it gets deployed ideally so that you’re not doing this live when someone’s in the OR.

Which is to say that whatever they’re using, the lines have to be clear. Either some things are allowed, or they’re not. As little ambiguity as possible. The DoW would also want the power to determine courses of action, and can’t leave operational control in the hands of another. This is the now infamous scenario that someone apparently painted in discussions with Dario, if a missile was heading towards the US would they be ok to use Claude to defend.

Apparently Dario said they’d work it out, and also later said they can carve out a missile defense aspect from the contract, but you hopefully see the problem. You could easily come up with a dozen other scenarios, so do you just keeping coming up with them and then taking them off the contract because ‘that seems fine’?

The other “red line”, about mass surveillance, is similar. What does that mean? You ask a dozen people, as Zvi did, you get a dozen different responses. Going from a vague feeling to something that’s specific is really difficult.

Now the DoW’s position seems to be that let’s just do it according to the law. The law is clear enough, or at least clearer than a goal that we might share. Laws are an operationalisation of principles we hold dear.

But what if the law has loopholes? If we disagree with the law? You still have to find some ways to make that clear, but honestly you either draft a contract airtight enough to solve for those, or you have to believe that your counterparty will obey the law. You can draft “permissions-based” (enumerated) vs “restrictions-based” (negative list) provisions, if you’re clear enough. And it makes sense to have explicit contractual red lines, even if unenforceable mid-operation, since they create legal exposure and political cost for the government if violated. But they aren’t clear though, then no contract can save you, and saying “I will decide” will not necessarily break in your favour.

Terms like “reasonably requested” or “as appropriate” or “reasonable doubt” are standard legal terminology precisely because you can’t nail down every eventuality on every contract, these capture some combination of norms and prior history to gesture at the types of things that will be ok and types of things that won’t be.

Because the only thing that matters is whether you have any visibility into their actions in the first place. The Anthropic deployment was of a separate version of Claude, under a different ToS, deployed by someone else. Which means, they probably had limited visibility into what it was being used for. Which also means the only way to enforce any standards is to codify things quite a bit upfront - it’s like doing an on-premise installation vs saas.

OpenAI’s contract on the other hand seems to have been hand-in-hand with their own teams of FDEs and something they call a safety-stack (guessing cloud deployment of their own models and some checks therein, I don’t know). Which means they have much more operational visibility into the model usage, which also means they have the leeway to negotiate if the usage of it started to violate any of their ToS.

I have no real opinion here on which is better. Contracts are not inherently all-powerful, they’re only powerful insofar as they can have oversight. I do have an opinion that neither is inherently superior to the other, even if what we know about them is accurate, which might not be the case. One has more contractual protections and limited operational visibility, the other has lower contractual protections and higher operational visibility. The first one relies more on trust with the counterparty, the second one relies more on execution control. Both rely on the existing legal system.

This entire saga seems to me like it was a personality clash rather than a contractual dispute. A version might well be: Someone asked a question about Maduro raid. DoW got upset they’re being asked. They posed a hypothetical. Anthropic’s response was bad, confirming DoW’s prior assumption that they’re trying to control the deployment. Which is why even though they were so close to being effectively done with the agreement the Secretary of War decided to blow things up.

To reiterate, it’s really bad to call Anthropic a Supply Chain Risk. This is just not true. It is eroding yet another norm about what capricious governments could do at a time we should not be eroding it, we should be strengthening it. It is perfectly fine for Anthropic to have rules about how their AI ought to be used. It is perfectly reasonable for DoW to say nah that’s not going to cut it, I don’t want to ask for permission.

But what is true is that this should not be much of a surprise considering the constant rhetoric over the past few years has been that AI is a power like no other. It’s like nukes, but times a thousand. We need regulation. And when an industry repeatedly calls out for oversight, asking for someone to make the rules on how it should be used, you cannot be surprised when the Defense department take that seriously. You cannot be surprised when they make up their own interpretations of what ought to be done, because you were insufficiently prescriptive. They will listen to your articulation of any red lines and wonder, what do you mean you want to tell me how to use the mega-nuke-crazy-power that you yourself are saying you don’t know how to control?

The US has nationalised or regulated whole industries for simpler reasons. Telephone lines, rails, steel mill attempted seizure, these aren’t small things. And that’s not to mention the times the government has threatened to do this, from JFK to FDR.

So if you think AI is important we’re going to see more of this. You simply cannot call your technology a major national security risk in dire need of regulation and then not think the DoD would want unfettered access to it. They will not allow you, rightfully so in a democracy, to be the arbiters of what is right and wrong. This isn’t the same as you or me buying an iOS app and accepting the T&Cs.

But it’s also true that a corporation acting as a bulwark for democracy against the government is fundamentally weird, even if true. Democracy is incredibly annoying but really, what other choice do we have! What we don’t have is a reckoning with the power that is now reality.

I am extremely uncomfortable with the fact that we can just purchase commercially available data on almost everyone. I am also somewhat uncomfortable that the future of war is going to be autonomous though there are days where having Claude or GPT decide where to bomb seems better than an average 22 year old. I’m uncomfortable that in the pursuit of absolute security we have effectively given up our privacy, and all that remains are small shreds that only sit with a couple of large technology giants. I’m uncomfortable that the few shreds of privacy that did exist can now be reverse engineered away using pretty normal AI tech.

I also am not sure there’s a way out where we would ever have digital guarantees of privacy. I think our children will think that a quaint old notion. “What do you mean, I can of course just ask my AI to analyse a bunch of information and figure out who ratmonster2024 is.” The work that only NSA used to be able to do a couple decades ago is probably within the grasp of the average startup, if they cared. Genies don’t tend to go back into bottles, and this one has powerful forces keeping it out.

The future will bring these questions to bear, much faster than anyone might expect. The current world survives because a lot of analysis is effort-bounded. If that’s gone, a lot of things we previously assumed secure will also go away. This is coming, whether you want to or not. The best part of last week is that the issue became higher profile, again. But bringing attention to the issue is only the first part. Unless we know what we want to do with the attention, tribal politics is going to overwhelm it all.

---

I had a conversation with Azeem Azhar and an august panel last week. It was really really good, and you should check it out.

🔮 Where the human ends and AI begins