Guard the Budget

cost caps and spend guardrails

Part of: Ship & Monitor an LLM App (Capstone)

Logging and cost tracking tell you what happened. A budget guard stops something before it happens. It refuses a call that would push total spend over a limit you set, instead of you finding out at the end of the month. This is the last piece between "the demo works" and "someone left it running in a loop overnight and racked up a bill." A guard that tracks and enforces The order matters. Check before you update spent cents. If the check fails, nothing is charged and the call never reaches the model. This is a hard cap, not a warning. Once you're at the limit, charge() returns False every time until the window resets (daily, monthly, whatever you choose). Wiring it into the request path 402 Payment Required is the status code for exactly this situation, rarely used but a clean fit. The guard sits before the model call, right after input validation, so a request that would blow the budget never spends a token. Rate limiting is the same idea, different resource A budget guards a dollar amount. A rate limiter guards a request count over a time window (say, 60 requests per minute). Both follow the same pattern: track a running counter, compare it to a limit, allow or reject, reset the

Challenge: Enforce the Daily Cap