AI Data Poisoning

AI data poisoning is tampering with the data an AI model learns from or relies on, so it behaves in ways the attacker wants, such as giving wrong answers, showing bias or hiding a backdoor that activates on a secret trigger.

AdvancedCategory: AI and emerging threats

Also called: data poisoning, model poisoning, training data poisoning, backdoor attack, AI backdoor

What is AI data poisoning?

AI models learn patterns from data. If an attacker can change that data, they can change what the model learns. NIST's taxonomy of attacks on machine learning lists poisoning as one of the main attack types, alongside evasion and privacy attacks.

OWASP's Top 10 for LLM applications, which ranks it fifth in the 2026 edition, describes data and model poisoning as manipulating pre-training, fine-tuning or embedding data to introduce vulnerabilities, backdoors or biases.

Types of poisoning

  • Training data poisoning: slipping manipulated examples into the data used to train or fine-tune a model. Large models are trained on huge amounts of web content, so attackers can try to plant material where it's likely to be collected.
  • Model poisoning: directly tampering with a model's files or weights, or publishing a malicious model on a public repository for others to download. See AI supply chain risk.
  • Backdoors: the model behaves normally until it sees a specific trigger word or pattern, then does what the attacker intended. OWASP notes this can turn a model into a "sleeper agent" that's very hard to detect by testing.
  • Retrieval or knowledge base poisoning: planting false or malicious documents in the data an AI assistant searches when answering questions. See RAG security. This is the most practical risk for many businesses.
  • Feedback poisoning: abusing user feedback or ratings that are used to retrain a model.

What poisoning can do

  • Make a model give wrong, biased or harmful answers on certain topics
  • Cause a security classifier, such as a spam or malware filter, to let specific attacks through
  • Plant instructions that act as indirect prompt injection
  • Damage trust in an AI product

How to reduce the risk

  1. Know where your data and models come from. Use trusted providers and verify downloaded models.
  2. Control who can add or change data, especially in knowledge bases that AI assistants search.
  3. Validate and clean training and fine-tuning data, and keep records of data sources and versions.
  4. Test models for unexpected behavior, including after updates.
  5. Monitor output for sudden changes in behavior.
  6. Keep a human in the loop for high-stakes decisions.

AI penetration testing can assess how exposed your AI systems are.

Related terms

Sources

  1. LLM04:2025 Data and Model Poisoning (OWASP Gen AI Security Project)
  2. AI 100-2 E2025: Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST)

Drafted with AI assistance, then edited and fact-checked by a person. How we use AI.