WeaknessASC-2026-0019Major weakness
Training data poisoning and backdoored models
HighWorkarounds only
Poisoned fine-tuning data or tampered weights make a model behave normally until a trigger appears, then produce attacker-chosen output. Backdoors live in the weights, where file scanners cannot see them.
How to detect it
- Datasets or adapters from unverified sources
- Behavior that changes sharply on unusual phrases or tokens during evaluation
- Model weights whose hash does not match the publisher's
Workarounds
- Train and fine-tune only on data with known provenance, and keep hashes of every dataset version
- Download models and adapters only from verified publishers and pin them by hash
- Evaluate on held-out sets and red-team for trigger behavior before deployment
- Keep a model inventory so a poisoned model can be found and replaced quickly
No reliable method detects every backdoor in model weights; provenance and evaluation reduce the risk.
How to investigate
- Compare the model's hash and lineage against the publisher's records
- Re-run evaluations with suspected triggers and compare with a known-good model
- Identify every system that used the model and its outputs
Sources
- Anthropic, UK AI Security Institute and Alan Turing Institute: A small number of samples can poison LLMs of any size (October 2025)
- OWASP Top 10 for LLM Applications 2025: LLM04 Data and Model Poisoning
- OWASP Top 10 for Agentic Applications 2026: ASI04 Agentic Supply Chain Vulnerabilities
- MITRE ATLAS: AML.T0020 Poison Training Data
- MITRE ATLAS: AML.T0018 Manipulate AI Model