Streaming Structured Output

Streaming

Part of: Structured Outputs

You want the user to see results the instant they appear, not stare at a spinner for four seconds while the model finishes. So you stream the response, tokens arrive a few at a time. But here is the catch: {"items": [{"name": "App is not valid JSON. json.loads throws on it. Streaming structured output means rendering a shape that is not finished yet , and that is a genuinely different skill from parsing a complete object. What it is Streaming delivers the model's output incrementally, token by token, instead of all at once at the end. For prose that is trivial: print each chunk as it lands. For structured output it is hard, because a half-arrived JSON object is syntactically invalid right up until the closing brace. You cannot just call json.loads on a partial buffer. The technique is incremental or partial parsing : as bytes accumulate, you repeatedly try to make sense of the buffer so far, surfacing complete pieces (like finished list items) the moment they close, without waiting for the whole object. How it works The streaming loop has a clear shape: 1. Accumulate. Append each incoming chunk to a growing buffer string. 2. Attempt a parse. Try json.loads(buffer). If it succeeds,

Challenge: Stream the Buffer