Best AI Red Teaming CompaniesAn independent guide to AI red teaming vendors

Compare vendors / Purple Llama CyberSecEval

Purple Llama CyberSecEval

Meta’s open-source cyber risk benchmarks and safeguards

Agents and toolsFrontier model safetyRuntime guardrailsOpen source

Purple Llama is Meta’s umbrella project of evals and safeguards. Its CyberSecEval benchmarks measure insecure code, cyberattack helpfulness, code interpreter abuse, prompt injection, spear phishing and autonomous offensive operations, and CyberSecEval 4 (Apr 2025) added a CyberSOC eval built with CrowdStrike. It also includes Llama Guard, Prompt Guard, CodeShield and LlamaFirewall.

Best for: Model builders measuring cyber misuse risk with public benchmarks.

Open source (MIT for evals; Llama licenses for models), maintained by Meta

  • – Maps to OWASP
  • – Maps to MITRE ATLAS
  • – Maps to the EU AI Act
  • – SOC 2 or ISO 27001
Visit Purple Llama CyberSecEval

Facts checked November 9, 2026 against the sources listed below. Purple Llama CyberSecEval does not pay to appear here.

1 Legal namePurple Llama / CyberSecEval
2 HeadquartersNot disclosed
3 Founded2023
4 TeamNot disclosed
5 What they testFoundation models, Prompt injection and jailbreaks, Frontier safety (CBRN, cyber), Multimodal, Agents and tool use
6 FrameworksCWE, MITRE ATT&CK
7 CertificationsNot disclosed
8 Published pricesFree (MIT for evals/benchmarks and CodeShield; Llama community licenses for Llama Guard/Prompt Guard models)
9 Customers and partnersNot disclosed

Worth knowing before you sign

  • No tagged GitHub releases; the README describes up to CyberSecEval 3.
  • Licensing differs by component (MIT vs Llama community licenses).

Go to Purple Llama CyberSecEval's site

Sources

  1. GitHub: meta-llama/PurpleLlama
  2. The Hacker News: LlamaFirewall and CyberSecEval 4
  3. SecurityWeek: Llama protection tools

Work at Purple Llama CyberSecEval? Send corrections with a source to editors@bestairedteamingcompanies.com. We fix verified errors within five business days.

Compare with

  • Mindgard AI red teaming firm: human-led tests plus an automated platform
  • Gray Swan AI AI security lab: agent red teaming, lab evals and the Arena
  • HiddenLayer AI security platform with automated red teaming and model scanning
  • Trail of Bits Security research consultancy with an AI/ML assurance practice
  • Pillar Security Agent security platform with agentic red teaming