How do you build a golden eval set for an LLM feature and use it to detect regressions?

A golden eval set is to LLM testing what a regression test suite is to deterministic software — a fixed set of cases that should continue to pass as the system evolves.

Building one:

1. Collect inputs: sample real production queries stratified by type (short/long, domain, language). Include near-misses where the model has historically struggled — don't just include easy examples.
2. Define expected quality: for each input, record whether the output must be accurate, a required format, within length constraints, grounded. Store human-rated scores for a sample as calibration.
3. Baseline the current model: run the set and record aggregate quality score per dimension.

Regression detection: after any model change (new version, prompt change, retrieval change), re-run the eval set. Use a significance test — binomial for binary labels, paired t-test for continuous scores — rather than comparing raw percentages. A 2% drop on 50 examples might be noise; on 1,000 examples it is probably signal.

See Evaluation methods and Eval platforms and tooling.