Red Teaming
Red teaming is an authorized, realistic attack on an organization by a team playing the role of a real adversary, to test how well its people, processes and technology detect and respond. In AI, it means probing a model for flaws.
Also called: red team, red team exercise, adversary emulation, AI red teaming
NIST defines a red team as a group of people authorized and organized to emulate a potential adversary's attack or exploitation capabilities against an enterprise's security posture. A red team exercise is a simulated attack, under real-world conditions, that tests the organization's security as a whole.
Red teaming differs from a penetration test in its goal:
- A penetration test tries to find as many exploitable weaknesses as possible in a defined scope.
- A red team exercise pursues a specific objective, such as reaching the finance system, using whatever a real attacker might use, including phishing and physical access. It also tests whether the defenders, the "blue team", notice and respond.
Because it tests detection and response, red teaming makes most sense for organizations that already have monitoring in place, such as a SOC or MDR service.
In the AI context, NIST describes red teaming as a structured testing effort, often using adversarial methods, to find flaws and vulnerabilities in an AI system, such as prompt injection or harmful outputs. See our pages on AI red teaming and penetration testing.
Related terms
Sources
- Glossary: red team (NIST)
- Glossary: red team exercise (NIST)
- Glossary: red teaming (NIST)
Drafted with AI assistance, then edited and fact-checked by a person. How we use AI.
