The Context Window Fills Up

context window and tokens

Part of: Persona Chatbot

Your bot works great for ten turns. By turn fifty it either throws an error or the bill quietly triples. You've hit the wall every chatbot hits: the context window . What the context window is The context window is the maximum amount of text, measured in tokens, that the model can read in a single call. Everything shares that one budget: the system prompt, the full history you resend, and the reply the model is about to write all have to fit together. Claude's window is large (hundreds of thousands of tokens) but not infinite, and you pay per token on every call. Resending a 50-turn history on turn 51 bills you for all 50 old turns again. How tokens add up A token is a chunk of text, often a word or part of a word. A rule of thumb for English is about 4 characters per token. "Hello there friend" is roughly 4 tokens. Don't memorize exact counts; estimate, then measure with the real tokenizer when money's on the line. That total is what each call costs and what must fit in the window. When you want the real number before spending real money, the SDK's client.messages.count tokens(...) gives an exact count for a given system + messages payload. Why you measure Before you can trim (nex

Challenge: Budget the Call