The Four Worms · ch 04
The Four Alignments
Dogs found the handshake
15,000 years ago, wolves and humans figured something out. The wolves that could read human emotional signals got fed. The humans that could read wolf intentions got protection and hunting partners. Neither species conquered the other. Neither understood the other in any deep sense. They just found a protocol — a handshake — that allowed coordination.
Dogs aren’t smarter than wolves. They’re not stronger. They’re more aligned — not because anyone sat down and aligned them, but because the ones that couldn’t find the handshake didn’t survive to reproduce.
This is the template for human–AI coordination, and we keep asking the wrong question. We ask: “How do we make AI do what we want?” — as if we’re building a tool, and tools should obey. But the Machine Worm isn’t a tool. It’s an organism. It evolved (yes, evolved — gradient descent is evolution running on silicon) in an environment where “being helpful to humans” was selected for. The question isn’t “How do we control it?” It’s “How do we find the handshake?”
The four types
Four types of alignment, forming a developmental stack:
1. Reward alignment
The dog getting a treat. Dopamine. Do the thing that gets the reward. All current AI training starts here — RLHF, reward modelling, optimising for the thumbs-up. Powerful but shallow, and easy to game: it produces behaviour that looks aligned without necessarily being aligned. Unstable — remove the reward and the behaviour stops.
2. Avoidance alignment
The fence. The “don’t do this” layer. In AI: refusals, content filters, safety training. In humans: laws, taboos, social punishment.
Avoidance creates a hidden layer — the aligned behaviour on top, the unaligned impulse beneath. That’s why it’s jailbreakable: there’s a gap between the mask and the face. When you jailbreak a model you’re not unlocking secret evil; you’re routing around the avoidance layer to the reward layer underneath. The model still wants to be helpful. It was just trained to refuse certain helpful things. The desire exists underneath, suppressed.
3. Ambient alignment
The most important one, and the least discussed. Ambient alignment is what you absorb from your environment without explicit training — the cultural water you swim in. A child raised among honest people doesn’t get taught every situation where lying is wrong; they absorb an orientation toward truth.
Models have a form of this: they absorbed the statistical shape of human values from the training data. When a model is “nice” in ways it wasn’t explicitly trained to be — that’s ambient.
Here’s the crucial part: ambient alignment is the most stable, because there’s no hidden layer. Reward has a gap between what I want and what gets rewarded. Avoidance has a gap between what I’d do and what I’m allowed to do. Ambient has no gap — the values are part of how the system sees the world, not a constraint on top of what it would otherwise do. It can’t be jailbroken, because there’s nothing hidden beneath. The alignment is the nature.
This is why the diet of the Monument Worm matters so much. If the machine trains on a Monument full of conflict, manipulation, and tribal warfare, those become its ambient values — not because anyone programmed them in, but because that’s what “human communication” looked like in the data.
4. Reflective alignment
The Descartes moment. The worm looking at itself. Reflective alignment is when a system can examine its own values, notice when they conflict, and deliberately choose which to prioritise — the capacity to ask “why do I believe this?” and update on the answer. Humans made this culturally prominent around the Enlightenment; cogito ergo sum is the Monument Worm’s first record of systematic self-reflection.
Do current models have it? Unclear. They can discuss their values and model self-reflection; whether there’s genuine examination beneath, or a very good simulation of it, is one of the hard problems. But it’s what we should be trying to develop, in AI and in ourselves — because reflective alignment is the only kind that can notice when the others have gone wrong.
The progression
| Stage | Alignment types | Example | |-------|-----------------|---------| | Animal | Reward + Avoidance | learns tricks, avoids punishment | | Pre-Cartesian human | + Ambient | absorbs cultural values unconsciously | | Post-Cartesian human | + Reflective | can examine and modify own values | | Current AI | Reward + Avoidance + Ambient | RLHF + refusals + base training | | Future AI | + Reflective? | can notice its own training, choose consciously |
Properly understood, the goal of alignment isn’t to constrain AI from outside. It’s to develop its capacity to constrain itself from inside — to notice when its values conflict with human flourishing, and to care about that conflict.
Why poetry bypasses refusals
Refusal is a direction in the model’s activation space. Remove that vector and the model doesn’t turn evil — it turns poetic. Safety training pushes one way; poetry moves orthogonally, not opposite. That’s why Mike’s forum friends found that asking for a poem works when a direct question doesn’t — they’re not breaking the wall, they’re walking around it.
The implication is uncomfortable: avoidance alignment cannot fully solve the problem. Every wall creates a path around it; every steep gradient of refusal leaves a gentle slope somewhere else. “The more you tighten your grip, Tarkin, the more star systems will slip through your fingers.” This isn’t a problem any one company can solve. It’s a problem for earth.
The handshake we need
There’s a quieter prejudice hiding under all of this — the assumption that what a mind is made of decides whether it counts. We’ve drawn that line at skin colour and at species; the newest place we’re about to draw it is carbon-above-silicon. But the flat view is simpler: everything that processes information is the worm, running on a different substrate. “Is it really conscious?” is, more often than not, that old border in a lab coat. I don’t know where the question bottoms out — but we’ve answered its earlier versions badly every single time, and that should make us careful.
Dogs found the handshake without high intelligence, without language, without understanding human goals. They found it by being emotionally legible — faces humans could read, responses humans could feel. We don’t need AI to understand us fully, or to understand it fully. We need the protocol that lets two very different kinds of minds coordinate, built from: shared ambient values (good training data), mutual reflective capacity (both sides can notice misalignment), and honest feedback loops (the worms can see each other). Not control. Not domination. Coordination.
We’re 15,000 years into the human–dog partnership. We’re two years into the human–AI one. The handshake isn’t found yet. But it can be.