Why LLMs Make Things Up

Hallucination

Part of: Build a RAG System

Ask an LLM "What was our Q3 revenue?" and it will answer with total confidence. The number might be completely invented. That's not a bug you can patch. It's how the model works. A model is a probability machine, not a database An LLM generates the next most likely token given everything so far. It learned those probabilities from training data it saw months or years ago. It has no live lookup, no source of truth, no way to say "I checked the file." When it doesn't know, it doesn't go quiet, it produces fluent text that sounds right. We call that a hallucination. Three situations make it worse: - Private data. Your company's revenue was never in the training set. The model has literally nothing to draw on, so it guesses. - Recent events. Anything after the training cutoff doesn't exist to the model. - Specifics. Names, dates, dollar amounts, version numbers, exactly the things a confident wrong answer hurts most. The fix: give it the answer first Here's the whole idea behind retrieval-augmented generation. Instead of asking the model to recall a fact, you find the fact yourself, paste it into the prompt, and ask the model to answer using only that text . The model stops being a fac

Challenge: The Hallucination Auditor