Logging & Tracing

Tracing

Part of: Deploying & Monitoring LLM Apps

A user reports: "the bot gave me a garbage answer this morning." You open your logs and find one line: INFO: request handled. No prompt, no model output, no timing, no way to tell which of the 40,000 requests that morning was theirs. The bug is real and you are completely blind to it. The model was fine; your observability was not. What it is Tracing is recording enough about each request to reconstruct what happened after the fact. For an LLM call that means, at minimum: the prompt, the model and settings used, the output, the token counts, the latency, and a trace ID that ties them together. A log is a single timestamped event; a trace is the connected story of one request, often across several steps. The discipline is simple to state and easy to skip: never make a model call you cannot reconstruct later. If you can't see the input and output, you can't debug, evaluate, or improve anything. How it works You wrap the model call so every invocation emits a structured record: Two details make this useful. First, the record is structured (JSON), so you can search and aggregate it, "show me all calls over 5 seconds", instead of grepping prose. Second, the trace id is returned to the c

Challenge: The Trace Digest