Writing a Good Test Set

test case design and categories

Part of: AI Evals Harness

A pass rate is only as honest as the test set behind it. Ten easy, hand-picked examples will always score well and tell you nothing. The real skill in evals isn't the grading loop. It's building a test set that would actually catch a regression. What a real test set contains A good test set is not a pile of your favorite examples. It mixes three kinds of case on purpose. Typical cases are the everyday inputs you expect most of the time. Edge cases are empty input, very long input, weird formatting, ambiguous phrasing. Known failure cases are real mistakes the model made in production, turned into permanent regression tests so that bug can never silently come back. Where do these come from? The best source is real usage. Pull a sample of actual production inputs, including the ones that got complaints, and turn them into golden test cases with a human-verified expected answer. Tagging by category A single overall pass rate hides where the failures live. Tag each case with a category and you can see that your 90% overall score is really 100% on greetings and 40% on refund requests, the category that matters most: Why this matters An overall pass rate can hide exactly the thing you ne

Challenge: Find the Weak Category