PromptFoo
CLI/library for prompt testing and red-teaming
About
Test and compare prompts across models. Built-in red-teaming, regression testing, and side-by-side model comparison.
Choose PromptFoo when…
- •You want CLI-first, config-driven LLM evals
- •Running eval suites in CI/CD pipelines is a goal
- •You need red-teaming and safety testing built in
Builder Slot
Tests, evals, and experiment tracking to measure and improve your AI output quality
Stack Genome Detection
AIchitect's Genome scanner detects PromptFoo in your project via these signals:
promptfoopromptfooconfig.yamlpromptfooconfig.ymlIntegrates with (2)
Langfuse production traces can be exported as eval datasets that Promptfoo uses for regression testing in CI.
→ Close the eval loop: real failures captured in Langfuse become the regression test cases Promptfoo runs on every deploy.
Promptfoo calls OpenAI's API directly to run prompts through configured test cases and compare outputs against assertions.
→ Automated prompt regression testing against GPT-4o — catch output quality changes before they reach production.
Often paired with (1)
Alternatives to consider (3)
Pricing
In 4 stacks
Badge
Add to your GitHub README
[](https://aichitect.dev/tool/promptfoo)Explore the full AI landscape
See how PromptFoo fits into the bigger picture — browse all 207 tools and their relationships.