AI Evals Harness

Ship an AI feature and someone will ask "does it actually work?" Eyeballing five examples in a notebook doesn't answer that at scale. An eval harness does: you run a test set through your feature, grade every result automatically, and "does it work" becomes a

8 lessons, each with runnable code in the browser.

1. The Shape of an Eval
2. Your First Pass Rate
3. Writing a Good Test Set
4. Rule-Based Graders
5. LLM-as-Judge
6. Harden: Flaky Judges and Retries
7. The Weighted Report and Regression Gate
8. Ship the Harness

Compilearn home