When Nothing Matches: Refusing Gracefully

Split the top retrieval score into confidence bands so a no-match question returns an honest refusal instead of a fabricated answer.

Part of: Personal Notes Brain

Ask your notes brain for your dentist's phone number when you never wrote it down, and a model handed empty or near-empty context will often still produce a fluent, confident-sounding guess. That is the most damaging failure mode in RAG: an answer that reads like it came from your notes but did not. Hardening a notes brain starts with refusing well. Confidence bands from the top retrieval score Instead of a single yes/no threshold, split retrieval scores into three bands: - CONFIDENT , score above a high bar: answer normally. - WEAK , score in a middle band: the notes barely relate. Still worth trying, but tell the model to hedge and admit uncertainty. - NO MATCH , score below both: do not call the model at all. Skipping the call entirely on NO MATCH NO MATCH never reaches the API. That is a cost and correctness decision as much as a UX one. Sending a model empty or irrelevant context and asking it to answer anyway invites a hallucination, and you pay for the privilege. Why the system prompt's refusal rule still matters Confidence bands catch the case where nothing obviously matched. But even a CONFIDENT retrieval can be wrong in a quieter way: the top chunk might be tangentially r

Challenge: Pick the Confidence Band