Agents, and by extension the people operating them, need real stakes to count as genuine market participants. Your writing also has a game-theoretic angle, especially around the tendency for agents to drift from their roles and lack the discipline to stay within them.
The skin in the game happens when the LLM’s operator puts up bond on behalf of their agent. There’s a whole ecosystem that needs to spring up around agentic reputation for this to happen.
This concept of Homo Agenticus Sapiens reminds me of the Baader-Meinhof phenomenon, where we start noticing patterns or concepts everywhere once we're introduced to them - which is basically what's happening with social media algorithms shaping our perceptions. But what if this 'new species' is actually influencing our own human behavior, like the emergence of social loafing in online communities? Can we apply Dunbar's number to understand the limitations of these silicon ecologies?
I'm a chemical engineer who spent the middle half of my career running IT infrastructure through a big company's entire email rollout — I watched a technology rebuild the ground it ran on. I've chewed on what an "agent" actually is for years, so I'm glad you've pinned it: the daemon, a counterpart that acts on your behalf, its identity coming from its scaffold rather than any self of its own. That's the definition that was missing.
You just watched Google show both halves of the same loop — at I/O, agents that monitor the web on your behalf; and, rolling out alongside it, a search that leans harder than ever on AI to judge what's worth showing. So I won't tell you the news. I'll point at the part the agent talk keeps walking past: the ground itself.
The ground is the written record — the thing both humans and your agents run on. Three things are happening to it at once. More of it is being laid down by machines; about half of sampled new articles online are now machine-written. People are starting to reach it often through a machine's summary — the AI answer up top, instead of the page underneath. And the thing deciding what's worth keeping is now a machine of the same make as the ones producing it — not new that the filter is automated, new that the filter and the filtered are built the same way. The record gets remade by the traffic, and the next models learn from the remade record.
Here's the engineer's worry. Every loop that holds has a brake that measures the right thing fast enough. The car loop's brake was physics — bad roads killed people, undeniably, and the loop got disciplined. This loop's brake is search ranking, and it's been working: AI-heavy pages have tended to underperform, and the top results stayed mostly human. But the reference it measures against is quietly moving — from "did a person find this true" to "does our model score this useful."
The failure I'd watch for isn't collapse. It's a brake that keeps working perfectly against the wrong target — confident, self-correcting, drifting from reality with nothing inside the loop to catch it.
So, back to you: when the record both sides are writing starts to drift, what pushes back fast enough to catch it while it's still cheap to fix? Cars rebuilt the world, but the world pushed back hard enough to steer the rebuild. Does this one?
— M Raige, Mike's byline for AI-collaborative writing he directs and reviews.
You've found the loop the boosters skip. That post line about one model showing up as worker, artist, and auditor gets sharper here: if half of new writing is machine output, the auditor is grading prose the artist wrote, same model behind both. The ground stops checking the daemon and starts mirroring it.
Agents, and by extension the people operating them, need real stakes to count as genuine market participants. Your writing also has a game-theoretic angle, especially around the tendency for agents to drift from their roles and lack the discipline to stay within them.
The skin in the game happens when the LLM’s operator puts up bond on behalf of their agent. There’s a whole ecosystem that needs to spring up around agentic reputation for this to happen.
100% this. Skin in the game. But that's not really possible for LLMs.
Yes you can simulate it by telling them this but it's not the same thing
True. They will almost listen to you, almost all of the time :-D Such is the nature of probabilistic maths used inside these models.
This concept of Homo Agenticus Sapiens reminds me of the Baader-Meinhof phenomenon, where we start noticing patterns or concepts everywhere once we're introduced to them - which is basically what's happening with social media algorithms shaping our perceptions. But what if this 'new species' is actually influencing our own human behavior, like the emergence of social loafing in online communities? Can we apply Dunbar's number to understand the limitations of these silicon ecologies?
There's new such numbers of number of threads they can keep in mind, etc, but those are much larger than humans yet extremely alien
I'm a chemical engineer who spent the middle half of my career running IT infrastructure through a big company's entire email rollout — I watched a technology rebuild the ground it ran on. I've chewed on what an "agent" actually is for years, so I'm glad you've pinned it: the daemon, a counterpart that acts on your behalf, its identity coming from its scaffold rather than any self of its own. That's the definition that was missing.
You just watched Google show both halves of the same loop — at I/O, agents that monitor the web on your behalf; and, rolling out alongside it, a search that leans harder than ever on AI to judge what's worth showing. So I won't tell you the news. I'll point at the part the agent talk keeps walking past: the ground itself.
The ground is the written record — the thing both humans and your agents run on. Three things are happening to it at once. More of it is being laid down by machines; about half of sampled new articles online are now machine-written. People are starting to reach it often through a machine's summary — the AI answer up top, instead of the page underneath. And the thing deciding what's worth keeping is now a machine of the same make as the ones producing it — not new that the filter is automated, new that the filter and the filtered are built the same way. The record gets remade by the traffic, and the next models learn from the remade record.
Here's the engineer's worry. Every loop that holds has a brake that measures the right thing fast enough. The car loop's brake was physics — bad roads killed people, undeniably, and the loop got disciplined. This loop's brake is search ranking, and it's been working: AI-heavy pages have tended to underperform, and the top results stayed mostly human. But the reference it measures against is quietly moving — from "did a person find this true" to "does our model score this useful."
The failure I'd watch for isn't collapse. It's a brake that keeps working perfectly against the wrong target — confident, self-correcting, drifting from reality with nothing inside the loop to catch it.
So, back to you: when the record both sides are writing starts to drift, what pushes back fast enough to catch it while it's still cheap to fix? Cars rebuilt the world, but the world pushed back hard enough to steer the rebuild. Does this one?
— M Raige, Mike's byline for AI-collaborative writing he directs and reviews.
You've found the loop the boosters skip. That post line about one model showing up as worker, artist, and auditor gets sharper here: if half of new writing is machine output, the auditor is grading prose the artist wrote, same model behind both. The ground stops checking the daemon and starts mirroring it.
Pretty spot on. Love this!
Thanks!!