Using tools to build evals
AI assistants and eval platforms can do the slow parts for you: drafting datasets, writing scorer code, running the suite and showing the results. They cannot decide what failure means for your product. That part stays with you.
Using an AI assistant to help
Coding assistants such as Claude Code or OpenAI Codex can write eval scaffolding inside your project: the dataset file, the runner script, the code-based checks. Useful prompts look like this:
Here are 3 real outputs that failed, and my note on each failure.
Write a binary judge rubric for the failure "invents a refund rule".
Give one PASS and one FAIL example. Do not add other criteria.Three checks before you trust the output:
- Read the rubric as a stranger would. Would they know what to do with an edge case?
- Run it on your hand-labelled examples. If the judge disagrees with you, the rubric is wrong or the judge is.
- Watch for the assistant grading its own work. Do not use the same model to write the rubric and to decide whether the output passes, without checking against human labels.
Eval platforms
Three commonly used tools, with different centres of gravity. Check each vendor's current features and pricing before you commit. These change often.
Promptfoo
Open-source, command-line first, with the test cases in config files. Good for running evals in a continuous integration pipeline, so a failing test blocks a release. It also includes red-team tests for prompt injection and unsafe outputs.
Braintrust
A hosted platform built around experiments: run a dataset against two versions, compare the results side by side, and add human review in a web interface. Good for teams where product managers and engineers share the review work.
Langfuse
Open source and self-hostable, centred on tracing: record every model call, then build datasets and evals from real traces. Useful when you need data to stay on your own infrastructure.
How to choose
- Start with no tool. A spreadsheet with 30 labelled outputs is enough for your first round of error analysis.
- Pick a tool when the suite runs often. If you run evals on every change, put them in a tool that runs in your pipeline (Promptfoo is a common start).
- Pick a platform when people other than engineers review outputs. A shared review interface (Braintrust, or Langfuse with annotation) saves arguments about who scored what.
- Pick self-hosted when data cannot leave your systems. Langfuse and Promptfoo both run locally.
Sources
Tool descriptions are summaries from vendor material and third-party comparisons, which can be out of date. Check current features before you choose.