Open-source AI red teaming tools

What these tools do

Automated red teaming uses software, often another model, to generate attacks at scale. The approach was set out in Perez et al., 2022. Open-source tools package it in three main shapes:

• Scanners run libraries of known attacks against a model or app endpoint and report which ones worked.
• Attack frameworks let you compose your own campaigns: multi-turn conversations, converters that rewrite prompts, scorers that judge outputs.
• Test and eval harnesses run defined test cases on every change, usually in CI, so a prompt edit or model swap does not quietly reopen an old hole.

All three are good at breadth and repeatability. None of them knows your business, your data or what a real attacker would want from your agent. Keep that in mind when you read the table.

The tools compared

Stars and licenses come from GitHub's API on 9 November 2026. Stars are a rough signal of adoption, not quality. Descriptions follow each project's own summary.

A related tool you may see on older lists: LLM Guard from Protect AI (MIT) is a security toolkit for scanning LLM inputs and outputs at runtime, not a red teaming tool. Its repository was archived in 2026. Protect AI is now part of Palo Alto Networks (Palo Alto Networks).

Which tool for which job

• "Scan our model endpoint for known weaknesses today." garak. It is built as a scanner and has the second-most GitHub stars in our table, after Promptfoo.
• "Build targeted, multi-turn attack campaigns." PyRIT. Microsoft describes it as a toolkit for automated scans and adaptive multi-turn attacks.
• "Catch regressions on every pull request." Promptfoo, with declarative configs and CI/CD integration. DeepTeam and Giskard fit similar workflows.
• "Run structured evaluations of model capabilities." Inspect, from the UK AI Security Institute, with ready-made evals in inspect_evals.
• "Attack a classic ML classifier or vision model." Adversarial Robustness Toolbox. LLM tools are the wrong fit here.
• "Map what our agent workflow can reach." Agentic Radar scans agent workflows, but check its maintenance status first.

These tools also show up in AI assistant answers. In parse.gl's tracking of ChatGPT and Google AI Mode answers on the best AI red teaming service or platform (July to November 2026), garak appeared in 54% of answers, PyRIT in 53% and Promptfoo in 48% (parse.gl).

Who owns these tools now

Ownership matters for an open-source tool because the owner sets the roadmap, the defaults and the attack library.

• Promptfoo. OpenAI announced it would acquire Promptfoo in March 2026, and said it would keep the open-source project (SecurityWeek). We found no closing announcement. The repository is still MIT licensed. Its hosted Community tier is free up to 10,000 red teaming probes a month; enterprise tiers are custom priced (Promptfoo pricing). If you test models from OpenAI's competitors, watch how the project is governed.
• Agentic Radar. Built by SPLX, which Zscaler acquired in November 2025 (SiliconANGLE). The last code push we saw was in November 2025.
• garak and PyRIT are maintained by NVIDIA and Microsoft. Both repos had commits in the week of this review.
• Counterfit and LLM Guard are archived. Do not build new work on them.

For the wider wave of acquisitions, see who owns whom in our vendor guide.

How to run a baseline in a week

1. Pick a target you own. A staging copy of your app, with test data and production-like tool permissions in a sandbox.
2. Write down what failure means. List the harms and data that matter: leaking another customer's records, calling a refund tool, revealing the system prompt.
3. Run a scanner against the model endpoint for known jailbreaks and injection patterns.
4. Run app-level tests through the same interface users touch, including uploaded files and retrieved documents carrying injected instructions.
5. Add tests for each tool. For every tool your agent can call, write cases that try to make it call that tool when it should not.
6. Triage by hand. Automated judges produce false positives and miss subtle failures. Someone has to read the transcripts.
7. Put the passing cases in CI so they run on every model, prompt or tool change.

Log each attempt with a date, an ID, the prompt and the output, as Microsoft recommends (Microsoft Learn). If you later hire a red team, that log shortens scoping and lowers the quote. Our cost guide explains why.

When tools are enough

• Internal assistants with no tools, low-sensitivity data and a small user base.
• Regression testing between expert engagements.
• Early prototypes, to find the obvious issues before anyone pays to find them.
• Pre-scoping, so you can tell vendors what you already know.

When to hire people

• Agents with write access. Tools that send email, move money, change records or run code. Public competitions found nearly all agents broke within 10 to 100 queries (arXiv:2507.20526), and CAISI reported at least one successful attack against every frontier model tested (CAISI). Finding those paths in your agent takes people who think in chains.
• Regulated or high-stakes harms. Medical, financial, legal, cyber or CBRN. NIST treats expert red teaming as its own type (NIST AI 600-1).
• Evidence for an auditor or regulator. A tool's output is not a report. See AI red teaming requirements.
• Before a public launch or major change. Microsoft's AI Red Team says automation extends coverage but humans remain essential (arXiv:2501.07238).

OWASP lists reliance on automated scanning with no manual adversarial testing as a red flag when buying red teaming (OWASP). The same logic applies to doing it yourself. Use the tools for coverage, and bring in people for depth. Our vendor checklist covers how to pick them, and what AI red teaming is explains the method. Terms are in the glossary.

We list tools by usefulness and public data, not by any commercial relationship. Check each project's license and terms before use. This is not security advice for your specific system.