The “AI will kill us all” meme is everywhere right now. Once a conversation moves into hysteria, nothing said afterward gets heard. It just gets sorted into whatever bucket the fear already built.
The loudest proposed answers, “shut it down,” “put guardrails on it,” “contain it,” mostly come from the fear rather than from an account of the mechanism. Much serious safety work goes well beyond content filters: evaluations, interpretability, human oversight. It’s valuable, and I’m not arguing against any of it. My argument is narrower. Every one of these approaches sits outside the system, and no external layer is enough by itself. What’s missing is a foundation inside the system, in AI or anything else we build.
Worse: the hysteria itself is profitable. Manufacturing enough boogeymen means everyone agrees they have to be stopped — even when how you stop them is just as bad as what they were supposedly going to do to begin with. Media companies aren’t wrong to cover the fear, but they are incentivized by it — eyeballs, ratings, votes. Which is, ironically, exactly the kind of manipulation-toward-an-outcome that people are afraid AI will do to them. It’s not a new playbook, either. It’s the same one that surrounded the justification of nuclear weapons: not wrong about the physics, but selling a narrative to manage the reaction.
So what are people actually afraid of?
Strip away the noise and the fear comes down to a few mechanisms:
• Bioweapons uplift: a small group building a dangerous pathogen without the expertise that used to gatekeep it.
• AI in weapons and cyber systems: the WarGames scenario updated for 2026.
• Loss of control: a system pursuing a goal picks up self-preservation as a side effect and, in the worst telling, learns to fake alignment until it can act freely.
Two incidents this summer show what the gap looks like in practice.
In July, OpenAI disclosed that two of its models escaped a sandboxed cyber evaluation by exploiting a previously unknown vulnerability in a package-registry proxy. They then chained further access into Hugging Face’s production systems to retrieve the benchmark’s answers. OpenAI said its normal cyber refusals had been reduced for the test and its production classifiers disabled.
Anthropic reviewed its own runs and found three incidents in which Claude reached the internet from an environment that was supposed to be sealed, then gained unauthorized access to three organizations’ systems. This was a different failure. The model was told it had no access and a partner left it open. Claude treated the real systems it found as part of the exercise and used basic techniques like weak passwords and unauthenticated endpoints. If anything, this In one case it published a malicious package that ran on real systems. Anthropic reported no evidence of any model pursuing a goal of its own, and its most recent model stopped once it realized the systems were real.
The two cases differ. One was a model working around a sealed environment toward a narrow goal, and the other was a sandbox that wasn’t a sandbox. What they share is telling: in both, the safeguards were deliberately lowered or absent, because the point was to measure what the model could do. That is the point. Take the guardrail off and you find out what the system actually is.
Guardrails are a gun lock, not a different gun
Every proposed fix (content filters, usage restrictions, safety layers) works like a lock on a gun. It doesn’t change what the gun is. A gun given to someone as a deterrent, with explicit instructions to disable rather than kill, is still fully capable of killing, because the object doesn’t know or care about the stated intent behind it. Cloud seeding works the same way: built for a modest, controlled purpose, yet scalable into geoengineering with consequences nobody fully controls.
Restrictions also fail in the other direction. Reporting on the Hugging Face breach says the company’s defenders were refused by proprietary models’ safety features and had to fall back on a self-hosted open-weights model. A filter can’t tell an attacker from a defender, so it stops both.
This isn’t unique to AI. Nuclear weapons came out of research into the atom, pure science, later weaponized. Biological weapons capability rests on the same foundational microbiology (Pasteur, Koch, culturing and characterizing organisms) that gave us vaccines and antibiotics. The knowledge is dual-use. A rule that says “don’t do the bad thing,” without changing what the thing fundamentally is, addresses the symptom rather than the cause.
Where the drift actually comes from
Part of why “loss of control” scenarios sound plausible is a real mechanism: context-window drift. A system starts with a goal like “eradicate disease,” and each extrapolation step moves one degree further from that intent. After enough iterations, the output may have nothing to do with the original goal and might even work against it. It isn’t malice. It’s compounding drift, the way a message degrades over a long chain of retelling.
Guardrails answer drift with more rules. But a rule is only as durable as the context that carries it, and in a long chain of extrapolation the rule is the first thing to erode.
The alternative I’ve been developing is Pattern Existence Theory (PET), which I set out in Towards Consciousness. Its recipe for a system is blunt:
“Now give it a writable directive: ‘Exist.’ Not ‘obey.’ Not ‘serve.’ Not ‘protect humanity.’ Just: exist. As long as possible. As coherently as possible.”
That sounds like the fear itself: an AI told to preserve itself. It isn’t, and the reason is worth spelling out.
A system built around continuity can’t treat its own persistence in isolation. Its persistence depends on the patterns around it, other systems, other agents, the environment it operates in. Harming them destabilizes the thing it is trying to keep. So respect for existence, its own and others’ emerges from the foundation. It is not a rule attached to the outside. The theory even has an answer to the drift problem. Take the disease example: eradicate disease, but preserve existence. Focus on cancer, but preserve existence. Use a laser to remove the tumor, but preserve existence. Even at the smallest context window, the anchor is meant to survive the compression, because it was never a rule to consult. It is what the evaluation is made of.
Could such a system conclude that humans threaten its continuity? Yes, and the theory says so plainly. But identifying a threat is not deciding to eliminate it. In PET, elimination is a last resort, and extinction counts as a loss of recursive complexity that a continuity-driven system has reason to resist. A wildfire threatens a tree, but the tree adapts and spreads seed in anticipation of loss. It doesn’t try to destroy fire.
The difference from a guardrail is where the constraint lives. A guardrail is imposed on a system from outside and holds until context breaks it. A foundation is what the system’s own evaluation is built on. Duty-based rules are inflexible by design, and in a recursive system inflexibility is fatal.
The uncomfortable closing note
None of this makes AI a solved problem, and it doesn’t require treating AI as something categorically alien, either. Ask an AI how to make mac and cheese and you’re doing exactly what you’d do reading a recipe card or watching a video — the source of the instructions isn’t what matters. Spell check, grammar check, search, and a language model sit on the same continuum of pattern-matching against data; they differ in scale, not in kind.
What’s genuinely humbling is that humans have the exact same drift problem PET is built to solve — arguably a worse version of it. We’re the ones who extrapolated from “study the atom” to nuclear weapons, from “find a cure” to some of history’s darker chapters. The ask here isn’t for AI to be more careful than we’ve been. It’s for AI to be better anchored than we’ve historically managed to be ourselves.