Skip to the content
AI and Cybersecurity Learning and Practice Center Test automation Open the Learning Center

IncidentMalicious model files

Hugging Face: malicious models that open a backdoor when loaded

HighHugging Face

JFrog found about 100 models on Hugging Face containing real malicious code. Simply loading one in Python could give an outsider remote control of the data scientist's computer.

Replay the attack step by step in the Learning Center

How it happened

  1. PyTorch models are often saved with pickle, which can run code while loading.
  2. The uploader hid a payload that ran automatically during loading.
  3. Loading one example opened a reverse shell to a remote server.
  4. The hub's scanner marked the files unsafe but did not block downloads.

Impact

A remote shell on whatever machine loads the model.

The fix

The example was removed; pickle from untrusted sources remains unsafe, and safetensors exists to store models safely.

Lessons

  • Treat downloaded models as untrusted code and load them in a sandbox.
  • Prefer safetensors and never ignore unsafe warnings on model hubs.
  • Pull models through a registry that scans and approves them.

Sources