P
What is Promptfoo?
Promptfoo is an open-source tool for testing and red-teaming LLM applications before they ship — running test cases against your prompts across multiple models, and probing for jailbreaks, bias, and other failure modes.
Key features
- Automated test suites for evaluating prompt and model quality
- Red-teaming tools that probe for jailbreaks and unsafe outputs
- Side-by-side comparison across multiple models and prompts
- CI/CD integration for catching regressions before deployment
- Open-source with an active contributor community
How to get started
- Install Promptfoo via npm or as a standalone CLI
- Define test cases covering your app's expected behavior
- Run evaluations across your candidate prompts and models
- Add red-team probes to check for jailbreaks and unsafe outputs
Use cases
- Regression-testing prompts before deploying an LLM feature
- Comparing model quality across providers for a specific task
- Red-teaming an application for jailbreak vulnerabilities
- Integrating LLM quality checks into a CI/CD pipeline
Promptfoo vs. alternatives
| Tool | How it compares |
|---|---|
| Promptfoo | Open-source testing and red-teaming for LLM apps. |
| Guardrails AI | runtime validation instead of pre-deployment testing |
| Hive Moderation | content moderation instead of prompt testing |
FAQ
Is Promptfoo free to use?
Promptfoo's plans: Free, open-source; paid enterprise support. Pricing and free-tier limits change over time, so check the official site above for the latest details.
What is Promptfoo used for?
Open-source testing and red-teaming for LLM apps.
Who is Promptfoo best for?
Promptfoo is a good fit for regression-testing prompts before deploying an LLM feature, or comparing model quality across providers for a specific task.
What are the alternatives to Promptfoo?
Commonly compared alternatives include Guardrails AI and Hive Moderation, see the comparison above for how they differ.
Related tools
G
HLast reviewed: 2026-08 · Reviewed by AIKetra editors · How we evaluate tools