What Counts Against It

Token budget

Part of: The Context Window

You type one short sentence, "What's the weather?", and somehow the request is 1,500 tokens. Where did they come from? You didn't write 1,500 tokens. But the app did, on your behalf, and all of it lands on the same desk. What it is The context window is a budget , and almost everything in a request spends from it. Four things, every time: 1. The system prompt : hidden instructions the app sends first ("You are a helpful assistant…"). You never see it, but it's there, and it costs tokens. 2. The full chat history : every earlier user message and every earlier model reply, re-sent on each turn. 3. Your new message : the thing you actually typed. 4. Reserved space for the answer : the model needs room left over to write its reply, so that headroom counts too. How it works Remember from the foundations: models are stateless . The model itself remembers nothing between requests. So to make a chat feel continuous, the app resends the entire conversation every single turn. Add it all up and you get the total going into the window: This is why a tiny message can still be an expensive request: the small part is your text; the large part is everything dragged along with it. Why it matters Kn

Challenge: Overflow Detector