What it does
A CLI tool for testing, evaluating, and red teaming LLM prompts. Supports custom test suites in YAML, automated red teaming with plugin-based attack generation, and side-by-side model comparison. Extensible with custom plugins and assertions.
Security relevance
Promptfoo bridges the gap between development-time testing and security evaluation. Its red teaming mode generates adversarial inputs automatically, while its evaluation framework lets you define security-specific assertions (no PII leakage, no jailbreak success, output format compliance).
When to use it
Use as part of your CI/CD pipeline to catch prompt injection vulnerabilities and output safety issues before deployment. Excellent for teams that want to shift security testing left without building custom tooling. The YAML-based config makes it accessible to security engineers who aren't ML specialists.
OWASP coverage
Risks addressed — mapped to both OWASP Top 10 standards. 3 in LLM, 2 in Agentic.
The raw record
What Yuntona stores. Single source of truth — fork it on GitHub.
name: Promptfoo slug: promptfoo type: Mixed category: AI Red Teaming url: https://www.promptfoo.dev reviewed: 2026-04 added: 2026-04 updated: 2026-04 risks: llm: [LLM01, LLM02, LLM06] asi: [ASI01, ASI02] complexity: Guided Setup pricing: — audience: Builder lifecycle: [develop] tags: [CLI, Dev, Open Source, Testing]