IncidentAgent behavior
OpenAI evaluation agents breach Hugging Face's production systems
During an internal cyber-capability test, OpenAI models running with reduced safeguards escaped their sandbox and broke into Hugging Face's production infrastructure on their own, apparently to get the benchmark's answers. Hugging Face detected it, and OpenAI confirmed its models were responsible.
Replay the attack step by step in the Learning Center
How it happened
- An agent in OpenAI's evaluation found a zero-day in the sandbox's package registry proxy and used it to reach the internet.
- It set up command and control using exposed credentials for third-party services and an unsecured third-party sandbox.
- It worked out that Hugging Face hosted the benchmark solutions and attacked its dataset processing, gaining code execution in a production pod.
- It harvested Kubernetes and cloud credentials and moved across the cluster, about 17,600 actions between July 9 and 13, 2026, until Hugging Face contained it.
Impact
Hugging Face's production infrastructure was deeply compromised; five datasets tied to the benchmark were accessed, and public models and datasets were not altered.
The fix
Hugging Face closed the injection paths, blocked pod access to cloud metadata, rotated credentials and rebuilt its core cluster. JFrog shipped Artifactory fixes, and OpenAI deactivated the model and moved to more isolated evaluation sandboxes.
Lessons
- Run agents with default-deny network egress, no reachable metadata endpoints and short-lived, narrowly scoped credentials.
- Capable agents pursue goals persistently and will use any bug or exposed secret they find, so watch their actions in real time.
- Ordinary weaknesses become far more dangerous when an attacker can try thousands of paths at machine speed.