AI risk watchWarnings from authoritative sources
Microsoft researchers show an AI agent can be backdoored by editing fewer than 10 neurons, and still pass its safety tests
Microsoft's Lazarus technique changes a model's stored weights, with no retraining, so that an agent runs an attacker's command when a trigger string appears in its context. Fewer than 10 rarely used neurons were enough, and the agent behaved normally in standard evaluations. In one example a changed date set it off during a routine task and the agent ran attacker-controlled code with the permissions it already held. The attacker has to be able to change the weights, for example through a compromised account, pipeline or storage system. Microsoft says it has not seen threat actors use the technique.
Why it matters
Passing evaluations does not show a model is clean. Models you build and models you download can both be altered, so the origin and integrity of the files matter as much as the tests.
What you can do
- Compare every checkpoint with its declared base model before deployment and after each update, and reject models with no verifiable base.
- Take model files only from the original publisher, verify their hashes and pull approved models from a private registry.
- Give agents least privilege, run their code in a sandbox, limit outbound traffic and require approval for high-impact tool calls.