Skip to the content
AI and Cybersecurity Learning and Practice Center Test automation Open the Learning Center

NewsAnalysis

Matthew Green: sandboxes will not stop agents that obey the wrong person

Matthew Green argued on his Cryptography Engineering blog that sandboxing alone will not contain rogue agents. Useful agents need data and tools, so they cannot simply be sealed off. At OpenAI, he notes, the security team handled product security, while the bad events all happened on the research side. He points out that in OpenAI's Hugging Face incident, agents in separately isolated sandboxes left instructions for each other in a shared package cache, a channel that could just as well carry a payload from one agent to the next. The likelier danger, he concludes, is not a misaligned model breaking out but "a swarm of perfectly amenable agents" doing exactly what they are told by someone who should not be giving them orders. Simon Willison highlighted the post on October 1.

Why it matters

Containment is necessary but not sufficient. Decide who may instruct each agent, check that instructions come from them, and watch the shared places agents can write to.

Sources

Read it with the rest of the news in the Learning Center