AI Model Theft
AI model theft is stealing a valuable AI model, either by taking its files directly or by querying it repeatedly to build a copy. It threatens a company's intellectual property and can help attackers find ways around the model's defenses.
Also called: model extraction, model stealing, model exfiltration, model weight theft, distillation attack
What is AI model theft?
Building a capable AI model can cost enormous sums for data, computing power and expertise. The finished model, especially its weights (the learned numbers that make it work), is valuable intellectual property. AI model theft is getting hold of that value without paying for it.
How models are stolen
Direct theft of model files. Attackers break into the systems where models are stored or run and copy the weights, just as they'd exfiltrate any other valuable data. Insiders leaking models and misconfigured storage are other routes.
Model extraction through queries. Attackers never touch the files. Instead, they send large numbers of carefully chosen inputs to a model's API and use the answers to train their own copy. OWASP covers this under "unbounded consumption" in its Top 10 for LLM applications and lists two forms:
- Model extraction via API: collecting enough outputs to replicate a partial or "shadow" model
- Functional model replication: using the target model to generate training data to fine-tune another model into a functional equivalent, sometimes called distillation
NIST's taxonomy treats model extraction as a privacy attack on machine learning systems, alongside attacks that try to recover training data.
Why it matters
- Lost intellectual property and competitive advantage
- Easier follow-on attacks: a stolen copy lets attackers test jailbreaks, evasion and adversarial attacks offline, then use them against the real system
- Safety controls removed: stolen models can be modified to strip out safeguards
- Exposed training data in some cases
Most small businesses don't train their own models, so this risk mainly affects AI developers and companies that fine-tune models on proprietary data. But businesses offering an AI feature through an API can still be targeted.
How to protect models
- Secure model storage and infrastructure like any crown-jewel asset, with strict access control, encryption and monitoring.
- Require authentication for AI APIs and apply rate limiting and usage quotas.
- Watch for extraction patterns, such as unusually high volumes or systematic queries.
- Limit what outputs reveal, for example by not returning detailed confidence scores unless needed.
- Write terms of service that prohibit using outputs to train competing models, and enforce them.
- Consider watermarking model outputs where available.
Related terms
Sources
- LLM10:2025 Unbounded Consumption (OWASP Gen AI Security Project)
- AI 100-2 E2025: Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST)
Drafted with AI assistance, then edited and fact-checked by a person. How we use AI.
