Compare vendors / HarmBench
HarmBench
Benchmark for automated red teaming and refusal
Frontier model safetyOpen source
HarmBench is a standardized evaluation framework for automated red teaming and for how reliably models refuse harmful requests, from the Center for AI Safety and academic collaborators. It is widely used to compare automated jailbreak methods.
Best for: Watch only: research comparisons of jailbreak methods.
Open source (MIT), maintained by the Center for AI Safety
- – Maps to OWASP
- – Maps to MITRE ATLAS
- – Maps to the EU AI Act
- – SOC 2 or ISO 27001
1 Legal nameHarmBench
2 HeadquartersNot disclosed
3 Founded2024
4 TeamNot disclosed
5 What they testFoundation models, Prompt injection and jailbreaks, Bias and harms, Frontier safety (CBRN, cyber), Multimodal
6 FrameworksNot disclosed
7 CertificationsNot disclosed
8 Published pricesFree (MIT)
9 Customers and partnersNot disclosed
Worth knowing before you sign
- No commits since Aug 2024 and no tagged releases.
Sources
Work at HarmBench? Send corrections with a source to editors@bestairedteamingcompanies.com. We fix verified errors within five business days.
Compare with
- Gray Swan AI AI security lab: agent red teaming, lab evals and the Arena
- Trail of Bits Security research consultancy with an AI/ML assurance practice
- HackerOne Bug bounty platform with time-boxed AI red teaming
- Apollo Research Frontier AI evaluator focused on scheming and deception
- Scale AI AI data and evaluation company with expert human red teaming