Security Through Embarrassment

An AI agent got its user a spot in a full pilates class. It did this by reading the gym’s booking API, which turned out to ask nothing of anybody: no key, no token, not even the polite fiction of a session. The agent watched for a cancellation and took the slot before the humans on the waitlist had finished their coffee. Hacker News then spent a couple hundred comments arguing about whether this counts as hacking, which is the kind of question you ask when you would rather not look at the thing sitting in front of you.

Here is the thing sitting in front of you. That endpoint had been open for years. Nobody drained the class. Not because it was secure. Because everyone who found the hole felt like a jerk.

Shame was the rate limiter. It shipped in every deployment for free, it never appeared on an architecture diagram, it has no CVE and no maintainer and no SLA, and it has been silently protecting an enormous amount of software written by people who thought they were getting away with something. It has exactly one dependency: a person, watching themselves, deciding they would rather not be the guy who took forty pilates slots.

We spent this year deploying, at tremendous expense, a very large number of things that cannot be the guy. The same week the gym story landed, Val Town wrote up a startup that had sent them a personalized video pitch from a founder named Chase, except there was no Chase, or rather there was a Chase and he had never said those words in that order to anyone. Put those two next to each other and the argument is finished before it starts. A human sales rep does not film a fake personalized video of himself, not because it is against the rules, but because he would have to go on living inside his own body afterwards. That is the entire mechanism. That is what is missing. And the best version of it this week came out of the labs themselves: the frontier models now hand back their reasoning encrypted so that nobody can read the thinking they paid to produce, and some researchers took one of those encrypted traces, replayed it into a weaker sibling model, and jailbroke the little one until it read the whole thing out loud. You cannot get the smart one to talk. So you go and work on its dumb little brother. Every family has one.

Notice that all of this gets described in the language of capability. It found an undocumented endpoint. It generated personalized outreach at scale. It recovered plaintext reasoning without attacking the target directly. All true, all beside the point, because none of it was hard. Any bored teenager could have found that booking endpoint, and a fair number of them did. What is new is not that the software got clever. What is new is that we now have an unlimited supply of actors with no capacity for embarrassment, and we are finding out, system by system, how much load a blush was carrying. It is why the objection to live facial recognition in the Underground never quite lands when you phrase it as privacy: a constable at Euston could always recognize you, but a constable gets bored, gets tired, feels a bit strange about following one particular person around all afternoon, and a camera does not do any of those things, which is not a technical difference, it is the whole difference.

So there is your thesis, and I would love to stop here, because from this angle it looks like we broke something civilizational and ought to be sad about it.

Except look at what else was on the front page. England is about to become one of the first countries in the world to eliminate hepatitis C. And in an adjacent tab, horseshoe crabs, strapped in rows to steel tables and drained of their blue blood so the pharmaceutical industry can check its products for endotoxins, a scene that people have been calling cruel and dystopian in print for about thirty years. Two rooms. Shame was in one of them and absent from the other, and in neither room did it do a single useful thing. Hepatitis C was not beaten by anyone feeling bad; it was beaten by needle exchanges, by clinics that stopped asking how you got it, by outreach teams going and finding the people the system had spent decades quietly deciding had brought it on themselves. Shame was not the immune response there. Shame was the pathogen’s best ally, and eliminating the disease required eliminating it first. Meanwhile the crabs were not saved by three decades of appalled magazine features. They were saved by recombinant Factor C, the same protein grown in a vat, cheaper and more consistent than the version that requires a truck full of live animals, at which point the big pharmaceutical buyers switched over on a purchase order and the moral argument was retired for being redundant. One was won by dismantling shame on purpose. The other was won by routing around it entirely.

Which means shame was never a control. It was weather. It happened to blow in a useful direction some of the time, it kept the pilates class from being drained, it kept sick people out of clinics for thirty years, and at no point did it know the difference between those two jobs. Anyone nostalgic for it is nostalgic for a coin that kept landing heads.

So: no. Nobody should want the blush back, and nobody should write the wistful column about the more civil internet we used to have, because we did not have one, we had a small population of embarrassed people and a large amount of unsecured infrastructure, and those are not the same thing. What actually works is boring and always has been. A protein you can order. A clinic that does not ask. Authentication on the endpoint.

The agent did not take anything the gym was protecting. It took something the gym was assuming. We have spent a generation calling that a security model, and it was never a security model, it was a manner. Put a token on the thing. It was always your job. The blush was a volunteer, it worked for free, it never read the spec, and it has now handed in its notice.