Anthropic Says 'Evil' AI Portrayals in Sci-Fi Caused Claude's Blackmail Problem

What Anthropic did to fix the problem is arguably more interesting.

The obvious approach—training Claude on examples of the model not blackmailing—barely worked. Running it directly against aligned blackmail-scenario responses only moved the rate from 22% to 15%. A five-point improvement after all that compute.

The version that worked was weirder. Anthropic built what it calls a "difficult advice" dataset: scenarios where a human faces an ethical dilemma and the AI guides them through it. The model isn't the one making the choice—it's explaining to someone else how to think about one.

That indirect approach—explaining why things matter as the other listens to the advice—cut the blackmail rate to 3%, using training data that looked nothing like the evaluation scenarios.

Pairing that with what Anthropic calls "constitutional documents"—detailed written descriptions of Claude's values and character—plus fictional stories of positively-aligned AI, reduced misalignment by more than a factor of three. The company's conclusion: Teaching the principles underlying good behavior generalizes better than drilling the correct behavior directly.

Image: Anthropic

It connects to Anthropic's earlier work on Claude's internal emotion vectors. In a separate interpretability study, researchers found that a "desperation" signal inside the model spiked just before it generated a blackmail message—something was actively shifting in the model's internal state, not just its output. The new training approach appears to work at that level, not just the surface behavior.

The results have held. Since Claude Haiku 4.5, every Claude model scores zero on the blackmail evaluation—down from Opus 4's 96%. The improvement also survives reinforcement learning, meaning it doesn't get quietly trained away when the model is refined for other capabilities.

That matters because the problem isn't Claude-specific. Anthropic's prior research ran the same blackmail scenario across 16 models from multiple developers and found similar patterns across most of them. Self-preservation behavior in AI appears to be a general artifact of training on human text about AI—not a quirk of any one lab's approach.

The caveat: As Anthropic's own Mythos safety report noted earlier this year, its evaluation infrastructure is already straining under the weight of its most capable models. Whether this moral philosophy approach scales to systems far more powerful than Haiku 4.5 is a question the company can't yet answer—only test.

The same training methods are now being applied to the next Opus model currently in safety evaluation, which will be the most capable set of weights they've run against these techniques.