Watching the Cost

token budgets and batching calls

Part of: Flashcard Maker

Your tool works. Now run it on a 40-page study guide and watch the bill. Every chunk is a separate API call, every call sends tokens in and gets tokens out, and tokens are money. Before shipping, you need a feel for what a run costs and a lever to keep it sane. That lever is batching . Tokens are the unit of cost A token is a chunk of text, roughly 4 characters of English. You pay for tokens sent (your chunks) and tokens received (the cards). A rough estimator is enough for budgeting: To predict a run's input cost, sum the estimate over every chunk plus the system prompt, which you resend on every call. That last part is the trap: a 300-token system prompt sent across 50 chunks is 15,000 tokens spent on the prompt alone. Batching: fewer calls, less repeated overhead Instead of one call per chunk, pack several small chunks into a single call, up to a token budget, and extract from all of them at once. Fewer calls means the fixed system-prompt overhead is paid fewer times: This is the greedy packer again, now measured in tokens instead of characters. Each batch stays under budget; when the next chunk would overflow, you close the batch and open a new one. Ten chunks that once cost te

Challenge: Count the API Calls