Token Usage

Token usage

Part of: Reading Model Outputs

You build a chatbot, ship it, and at the end of the month get a bill that makes your stomach drop. Where did the money go? It went into tokens, and the response told you the count of every single one, every call, in a block you probably never read. That block is usage , and it's the difference between flying blind and knowing exactly what each request costs. What it is Every response carries a top-level usage block: a small dict of token counts for that one call. Three numbers: - prompt tokens : the input : everything you sent (your prompt, system message, and any prior conversation re-fed in). - completion tokens : the output : everything the model generated in reply. - total tokens : the sum of the two. It should always equal prompt tokens + completion tokens. These counts are the real, exact token usage for the call, not an estimate. They are the basis for two things you care about: cost (APIs bill per token) and limits (the input plus output must fit inside the context window). How it works You read the usage block and do arithmetic on it: Notice the split rates: input and output are billed at different prices , and output is usually several times more expensive than input. Tha

Challenge: Multi-Turn Budget Meter