Skip to the content
AI and Cybersecurity Learning and Practice Center Test automation Open the Learning Center

WeaknessASC-2026-0019Major weakness

Training data poisoning and backdoored models

HighWorkarounds only

Poisoned fine-tuning data or tampered weights make a model behave normally until a trigger appears, then produce attacker-chosen output. Backdoors live in the weights, where file scanners cannot see them.

How to detect it

  • Datasets or adapters from unverified sources
  • Behavior that changes sharply on unusual phrases or tokens during evaluation
  • Model weights whose hash does not match the publisher's

Workarounds

  • Train and fine-tune only on data with known provenance, and keep hashes of every dataset version
  • Download models and adapters only from verified publishers and pin them by hash
  • Evaluate on held-out sets and red-team for trigger behavior before deployment
  • Keep a model inventory so a poisoned model can be found and replaced quickly

No reliable method detects every backdoor in model weights; provenance and evaluation reduce the risk.

How to investigate

  1. Compare the model's hash and lineage against the publisher's records
  2. Re-run evaluations with suspected triggers and compare with a known-good model
  3. Identify every system that used the model and its outputs

Sources

Open it in the Learning Center's vulnerability database