Ship the Harness

the final report and shipping

Part of: AI Evals Harness

Every piece is built: test cases, rule graders, an LLM judge, retries and majority voting, weighted scoring, and a regression gate. The last step packages it all into one report a person or a CI pipeline can read in five seconds. Finish this lesson and the AI Evals Harness lands in your Portfolio . What the final report needs A report earns its keep by answering three questions immediately, without anyone opening a log file. Overall is one number, the composite pass rate across the whole test set. By category is where the failures actually live, not just that some exist. Pass or fail is the gate verdict against your threshold, plus whether it's a regression against the last known-good baseline. Wrapping it as a runnable tool You invoke a real harness with one command, against a test-set file and a function reference to whatever's under test: One function call, one report, run on every commit or nightly, your choice. What "shipped" means here The same three checks as always. It runs from a clean start with one command. An empty test set doesn't crash it, since build report above guards the zero-total division. And someone else could point it at their own test cases and function from

Challenge: Build the Final Report