The Context Window Fills Up

Context Window

Part of: Build a Chatbot

Your chatbot works great for ten turns. By turn fifty it either throws an error or the bill quietly triples. You've hit the wall every chatbot eventually hits: the context window . What the context window is The context window is the maximum amount of text, measured in tokens, that the model can read in a single call. And everything shares that one budget: the system prompt, the full history you resend, and the reply the model is about to write all have to fit together. Claude's window is large (hundreds of thousands of tokens), but it is not infinite, and you pay per token on every call. Resending a 50-turn history on turn 51 means you're billed for all 50 old turns again. How tokens add up A token is a chunk of text, often a word or part of a word. A rule of thumb for English is about 4 characters per token, or roughly 0.75 words per token. "Hello there friend" is roughly 4 tokens. Don't memorize exact counts; estimate, then measure with the real tokenizer when money is on the line. To budget a whole conversation, sum the estimate over every message's content, then add the system prompt. That total is what each call costs and what must fit in the window. Why it matters: keeping h

Challenge: Fit the Context Budget