Lost in the Middle

Position

Part of: The Context Window

Researchers ran a clean experiment: bury one true fact inside a long context and ask the model to find it. When the fact sat at the very start or the very end, the model nailed it. When the same fact sat in the middle, accuracy fell off a cliff. They named the effect "lost in the middle," and once you know it, you'll arrange every long prompt differently. What it is Lost in the middle is the tendency of a model to pay less attention to information in the middle of a long context than to information at the beginning or the end. The result is a U-shaped accuracy curve: high at the edges, sagging in the center. A fact the model could easily use if placed first becomes easy to miss when buried halfway down a giant prompt. Crucially, this is about position , not whether the fact is in the window at all. The fact fits fine, the model just attends to it weakly because of where it sits. How it works A model spreads its attention unevenly across a long input. The opening and closing tokens get strong, reliable attention; the long stretch in between competes for a thinner slice. So the same content earns different effective weight depending on where you put it: The fix is purely about arrang

Challenge: Edge Arranger