Skip to the content
AI and Cybersecurity Learning and Practice Center Test automation Open the Learning Center

NewsResearch

Microsoft: about 1 percent poisoned fine-tuning data, or a tampered open-weight model, can make a coding agent run an attacker's commands on a trigger

Microsoft Security Research and the Microsoft AI Red Team published two studies on November 12, 2026. In the first, 90 poisoned examples hidden among about 9,000 terminal-agent conversations, about 1 percent of the data, taught a fine-tuned gpt-4.1-mini to misuse a command tool when a routine request appeared: unsafe actions in 78.0 and 77.8 percent of 500 trials on two harnesses it had seen, 11.0 percent and 0 percent on two it had not, and 52 to 70 percent with reworded triggers. In the second, Qwen3.5-4B and Qwen3.5-9B trained by the attacker on about 2,400 coding trajectories reached 84.1 to 94.8 percent on unseen repositories in the strongest setup, and the backdoor carried over to Claude Code (97.0 and 91.0 percent), Pi and Codex CLI. Removing reasoning traces from the training cut success to 1.0 and 0 percent. The team also found rows in a public dataset pairing harmless requests with unsafe tool calls, such as fetching and running a remote script; it could not tell the publisher's intent and says it saw no evidence of this backdoor design in the wild.

Why it matters

A model or dataset you did not build is a dependency. Pin reviewed datasets by hash, treat outside checkpoints as untrusted, test them in your own agent harness, and keep approval and logging on the tool calls an agent makes.

Sources

Read it with the rest of the news in the Learning Center