Why LLMs Make Things Up

Hallucination

Part of: How AI Actually Works

In 2023 a New York lawyer asked a model for cases to support his argument. It handed him six perfectly formatted citations with case names, courts, years, and judges. All six were invented. He filed them, got caught, and was sanctioned by the court. The model wasn't lying. It was doing its only job: predicting plausible text. What it is Hallucination is when a model produces confident output that is wrong or entirely made up. It is not a typo or a refusal but a fluent, authoritative-sounding answer that happens to be false. The part that causes trouble is how convincing it looks. The fake citations were formatted exactly like real ones, because the model learned what real citations look like. The root cause is a single fact from lesson 1: the model optimizes for plausible , not true . Most of the time those overlap, which is why the tool is useful. But when they split, when the most natural-sounding continuation isn't factually correct, the model produces the natural-sounding one anyway. How it works Three forces push a model toward making things up: - No fact-checker inside. There is no separate truth-verifier in the model. It generates text; it does not look anything up unless an

Challenge: Hallucination Gate