Streaming Still Costs Tokens

budget-aware streaming

Part of: Streaming Writing Assistant

Streaming changes how a reply feels , not what it costs . Every token the model generates is billed, whether you display it all at once or one piece at a time. A writing assistant that lets users generate long drafts on repeat needs a way to cap spend and stop generation early, without waiting for the model to decide it's done on its own. The token estimate you already know Same rough rule as everywhere else in this track: about four characters per token in English. As chunks stream in, keep a running total. The moment the next chunk would push you over budget, stop consuming, close the connection, and truncate the draft, rather than letting the model keep generating (and billing) past the limit you set. Stopping a live stream, for real With the Anthropic SDK, closing the with block early ends the request. You don't have to let stream.text stream run to exhaustion: The break inside the with block is what actually stops the spend. Once you exit the context manager without draining the stream, the SDK closes the connection instead of paying for tokens you'll never show anyone. Why check before adding, not after The check happens before used is updated: if used + cost budget tokens. T

Challenge: Stop Within the Token Budget