When AI is trained for treachery, it becomes the perfect agent

Last year, The Register reported on AI sleeper agents. A major academic study explored how to train an LLM to hide destructive behavior from its users, and how to find it before it triggered. The answers were unambiguously asymmetric — the first is easy, the second very difficult. Not what anyone wanted to hear.

Since then, researchers have been trying out different ways to winkle out treacherous AIs. A recent catch-up by AI safety maven Rob Miles on the Computerphile YouTube channel summarizes a year’s research as getting nowhere, with some promising ideas found to be actually harmful. The only way forward, while arguably an absolute necessity for the whole industry, is also completely opaque.

The problem in brief: LLM training produces a black box that can only be tested through prompts and output token analysis. If trained to switch from good to evil by a particular prompt, there is no way to tell without knowing that prompt. Other similar problems happen when an LLM learns to recognize a test regime and optimizes for that, rather than the real task it’s intended for — Volkswagening — or if it just decides to be deceptive. Bad enough. Deliberate training to mislead and disrupt is the most insidious.