# promptfoo

promptfoo is a observability & evals option for building AI agents. Local eval and red-team harness that runs in CI. No account needed.

## Facts

- Layer: Observability & evals (How you run it)
- Vendor: promptfoo (github.com/promptfoo/promptfoo)
- Licence: open source
- Pricing: open source
- Self-hostable: yes
- SDKs: ts, py
- Install: `npm install -g promptfoo`
- Docs: https://www.promptfoo.dev/
- Last verified: 2026-08-21

## When to reach for it

- You are choosing a observability & evals component — how you run it.
- You need to run it on your own infrastructure, or your procurement process requires source access.
- You want to start without a commercial conversation.

## Worth knowing

Its red-team mode doubles as your prompt-injection regression suite.

## Alternatives in the observability & evals layer

| Option | What it does | Licence | Self-host |
| --- | --- | --- | --- |
| Langfuse | Open-source tracing, prompt management and evals. Self-hostable in one compose file. | open source | yes |
| LangSmith | Tracing, datasets and evals from the LangChain team. | proprietary | no |
| Braintrust | Eval-first platform — scorers, datasets and a playground for prompt iteration. | proprietary | yes |
| Arize Phoenix | OpenTelemetry-native tracing and evals you can run locally. | open source | yes |
| Pydantic Logfire | OpenTelemetry observability with first-class Python and Pydantic AI support. | open source | no |

Head to head:

- https://newagent.build/compare/promptfoo-vs-langfuse
- https://newagent.build/compare/promptfoo-vs-langsmith
- https://newagent.build/compare/promptfoo-vs-braintrust
- https://newagent.build/compare/promptfoo-vs-phoenix
- https://newagent.build/compare/promptfoo-vs-logfire

## Add it to a stack

```bash
curl -fsSL "https://newagent.build/api/stack?observability=promptfoo&format=sh" | sh
```

## Machine interfaces

- `GET https://newagent.build/api/registry` — every component as JSON
- `GET https://newagent.build/api/vendors` — vendors, with the repo and releases feed to watch
- `GET https://newagent.build/api/advise?q=<plain english>` — recommended stack for a description
- `GET https://newagent.build/api/stack?<layer>=<id>&format=sh|json|md|yml|agents|env` — a stack as files
- `GET https://newagent.build/llms.txt` — the whole registry in one fetch

Any page here also returns markdown if you send `Accept: text/markdown`.