AI red teaming requirements and regulations in 2026

Not legal advice. This is a summary by our editors of public sources as of 9 November 2026. Rules and dates are still moving. Confirm how any of this applies to you with qualified counsel.

The short answer

• EU: providers of general-purpose AI (GPAI) models with systemic risk must conduct and document adversarial testing. That duty applies now, and the Commission can fine for breaches from 2 August 2026. High-risk AI systems face separate security duties under Article 15 from December 2027 or August 2028.
• US federal: no red teaming mandate for private companies. Federal agencies must test high-impact AI before deployment. Frontier model testing with the government is voluntary.
• US states: California and New York have frontier model transparency laws. Colorado's original AI Act is gone.
• Everyone else: NIST, ISO/IEC 42001, OWASP and MITRE ATLAS are voluntary, but customers and auditors can ask for evidence mapped to them. In our view, that is where most enterprise demand comes from.

Requirements at a glance

EU AI Act

The timeline

The AI Act (Regulation (EU) 2024/1689) entered into force on 1 August 2024. Prohibitions and AI literacy duties have applied since 2 February 2025. Governance rules and GPAI model obligations have applied since 2 August 2025. The Act became generally applicable on 2 August 2026, with exceptions, and the AI Office and national authorities took on enforcement (European Commission).

Article 55: adversarial testing for GPAI with systemic risk

Article 55(1)(a) requires providers of GPAI models with systemic risk to perform model evaluation "including conducting and documenting adversarial testing of the model with a view to identifying and mitigating systemic risks" (Art. 55). The other Article 55 duties cover systemic-risk mitigation, serious-incident reporting and cybersecurity.

A model is presumed to have systemic risk when its training compute exceeds 10^25 floating-point operations (Art. 51(2)). That covers a handful of frontier providers. If you build applications on top of their models, Article 55 does not apply to you directly, but your provider's testing is worth asking about.

The Commission's power to fine GPAI providers under Article 101 applies from 2 August 2026 (artificialintelligenceact.eu). The ceiling is 3% of worldwide annual turnover or 15 million euros, whichever is higher (Art. 101). Models placed on the market before 2 August 2025 have until 2 August 2027 to comply (FPF).

The GPAI Code of Practice, published 10 July 2025, is the voluntary route to showing compliance (European Commission). Its Safety and Security chapter applies only to systemic-risk providers and calls for state-of-the-art evaluations, including red-teaming and independent external evaluators.

Article 15: high-risk systems

For high-risk AI systems, Article 15(5) requires measures to prevent, detect and respond to data poisoning, model poisoning, adversarial examples or model evasion, confidentiality attacks and model flaws (Art. 15). The Act does not use the words "red teaming" here. In practice, adversarial testing is the most direct way to show those measures work. That is our reading, not legal advice.

The Digital Omnibus changed the high-risk dates

The Digital Omnibus on AI was proposed on 19 November 2025, reached political agreement on 7 May 2026 and entered into force on 27 July 2026 (European Commission). It moved two dates:

• Annex III stand-alone high-risk systems: from 2 August 2026 to 2 December 2027.
• Annex I product-embedded high-risk systems: from 2 August 2027 to 2 August 2028.

Reports on the final text say the GPAI systemic-risk duties were not changed (FPF). So Article 55 adversarial testing was not delayed. FPF also reports that Article 50(2) marking of synthetic content was postponed to 2 December 2026.

What this means for buyers: if you provide a high-risk system, you have more time, but the testing evidence still has to exist by the new date. If you are a GPAI provider with systemic risk, nothing moved.

United States: federal

No mandate for private companies

Executive Order 14110 of October 2023 required red-team result reporting for some dual-use foundation models. It was revoked by EO 14148 on 20 January 2025, and EO 14179, "Removing Barriers to American Leadership in Artificial Intelligence," followed on 23 January 2025 (White House). Since then, federal AI policy has relied on voluntary testing and on procurement rules for agencies.

OMB M-25-21: agencies must test high-impact AI

OMB memo M-25-21, issued 3 April 2025, requires agencies using high-impact AI to "Conduct Pre-Deployment Testing" (M-25-21). Where an agency has no access to the model, code or data, it must use alternatives such as "querying the AI service and observing the outputs." Agencies had 365 days, to about 3 April 2026, to document minimum practices. We expect vendors selling AI to federal agencies to be asked for test evidence. Check the memo's current status, since later executive orders may have changed it.

The AI Action Plan and EO 14409

America's AI Action Plan (July 2025) tasks NIST's Center for AI Standards and Innovation (CAISI) to "Evaluate frontier AI systems for national security risks in partnership with frontier AI developers." It also proposes an AI hackathon initiative and an "AI Evaluations Ecosystem" (AI Action Plan). CAISI was renamed from the US AI Safety Institute in June 2025 (Commerce). OpenAI gave CAISI and the UK AI Security Institute pre-deployment access to GPT-5 (GPT-5 system card).

EO 14409, "Promoting Advanced Artificial Intelligence Innovation and Security," was signed on 2 June 2026 (White House). Section 3 sets up a classified benchmarking process for AI models' advanced cyber capabilities and a threshold for "covered frontier models." It creates a voluntary framework that gives the government access to covered models for up to 30 days before release to other trusted partners. Section 3(c) rules out any "mandatory governmental licensing, preclearance, or permitting requirement."

EO 14365 (11 December 2025) set up an AI Litigation Task Force to challenge state AI laws (summary). Whether it will displace the state laws below is unresolved.

NIST

The NIST AI Risk Management Framework 1.0 was released on 26 January 2023. NIST says it "is being revised as part of the White House AI Action Plan," with no date given. A concept note for a critical infrastructure profile followed on 7 April 2026.

The Generative AI Profile, NIST AI 600-1 (July 2024), is the most useful NIST document for red teaming. It defines AI red-teaming, describes four team types (general public, expert, combination and human/AI) and lists 12 generative AI risks, from CBRN information to information security. Action MS-2.7-007 says to "Perform AI red-teaming to assess resilience against" abuse to facilitate attacks on other systems, GAI attacks such as prompt injection, and ML attacks such as adversarial examples.

NIST AI 100-2 E2025 (March 2025) is the adversarial machine learning taxonomy: evasion, poisoning, privacy and generative AI misuse attacks. It gives you shared vocabulary for scoping.

All of these are voluntary. They matter because US enterprise customers and federal buyers cite them in contracts and questionnaires.

US states

• Colorado. SB 24-205, the original Colorado AI Act, was delayed to 30 June 2026 and then repealed and reenacted by SB 26-189, "Automated Decision-Making Technology," signed 14 May 2026. The new law is a narrower transparency and consumer-rights regime: developer documentation from 1 January 2027, enforcement by the attorney general only, a 60-day cure period before 2030 and no private right of action. The old impact-assessment regime is gone. We do not treat Colorado as a red teaming driver.
• California SB 53, the Transparency in Frontier AI Act, was signed on 29 September 2025 and took effect on 1 January 2026 (The Register). It applies to frontier models trained with more than 10^26 operations. Large developers, with more than $500 million in revenue, must publish frontier AI frameworks and report critical safety incidents to the state's Office of Emergency Services. We have not confirmed whether the framework must describe third-party testing.
• New York RAISE Act. Signed in December 2025, amended in 2026 and reported to take effect on 1 January 2027 (National Law Review). Reporting on its final thresholds and audit terms is inconsistent, so check the enacted text with counsel.

United Kingdom

The UK has no AI statute that requires red teaming. The AI Safety Institute was renamed the AI Security Institute on 14 February 2025. It focuses on chemical and biological misuse, AI-enabled cyber attacks and crime, and says it "will not focus on bias or freedom of speech" (UK government). OpenAI gave it pre-deployment access to GPT-5 (GPT-5 system card), and it co-organized large public agent red teaming competitions with Gray Swan (CAISI).

The voluntary AI Cyber Security Code of Practice (31 January 2025) is to be submitted to ETSI as the basis for TS 104 223 and TR 104 128.

Standards and frameworks

ISO/IEC 42001

ISO/IEC 42001:2023 is the certifiable standard for AI management systems, published in December 2023. It is about governance: policies, roles, risk assessment and controls. We have not confirmed that it requires red teaming by name. Red team reports still make good evidence for its risk treatment and testing controls. Ask your certification body what it expects.

OWASP

• OWASP Top 10 for LLM Applications 2025: the default taxonomy for application findings, from LLM01 Prompt Injection to LLM10 Unbounded Consumption.
• OWASP Top 10 for Agentic Applications 2026, released 9 December 2025: ASI01 Agent Goal Hijack through ASI10 Rogue Agents.
• Vendor Evaluation Criteria for AI Red Teaming Providers and Tooling v1.0, 4 February 2026: the basis of our vendor checklist.
• The GenAI Red Teaming Manual v1.0, listed by OWASP in October 2026.

MITRE ATLAS v2026.10

MITRE ATLAS is the adversary tactics and techniques knowledge base for AI systems, maintained by MITRE. Release v2026.10, published 28 October 2026, contains 16 tactics, 129 techniques, 92 sub-techniques, 44 mitigations and 77 case studies (ATLAS CHANGELOG). New techniques include AML.T0136 Exfiltration via AI User Interface and AML.T0143 LLM Prompt Propagation. Some IDs moved: system prompt extraction is now under AML.T0085.003. Ask vendors which ATLAS version their mapping uses, because the data is updated often.

What evidence to keep

Whatever framework you answer to, the same records do most of the work. This list is our recommendation, built from Microsoft's reporting guidance (Microsoft Learn) and the frameworks above.

1. Scope and rules of engagement, signed, with dates.
2. The threat model and the harms list it produced.
3. Test logs: date, unique ID, prompt, output and endpoint for each attempt.
4. Findings with severity and framework IDs (OWASP, ATLAS, NIST AI 600-1, and EU AI Act articles where relevant).
5. Mitigations, owners and dates.
6. Retest results on the production endpoint.
7. A schedule for the next round, tied to model, prompt and tool changes.

For how to buy this work, see how to choose a vendor and what it costs. For the basics, read what AI red teaming is. Terms are in the glossary.