Log Every Request

structured request logging

Part of: Ship & Monitor an LLM App (Capstone)

Once your endpoint is live, print() stops being useful. Nobody is watching the terminal of a server running on someone else's machine. The fix is structured logging. For every request, write one record with a fixed shape (timestamps, token counts, latency, status) that a machine can read back later. Every later lesson in this capstone reads these records: cost tracking, the dashboard, error monitoring all start here. What one log record looks like A good request log answers the same questions every time, in the same shape: Note what's missing: the full prompt or reply text. Logging every character of every conversation is a privacy and storage liability you don't need for monitoring. Token counts and timing tell you almost everything about cost and health without storing what people actually said. Timing the call latency ms comes from wrapping the real call in a timer, not guessing: time.perf counter() is a high-resolution clock built for measuring how long something took. time.time() is for wall-clock timestamps and can jump around when the system clock adjusts, so it's the wrong tool for durations. Where records go For a small service, appending one JSON line per request to a fil

Challenge: Parse the Access Log