The Smallest Streaming Endpoint

a generator-based route

Part of: Streaming Writing Assistant

You have the wire format. Now build the smallest backend that uses it: one route that keeps the connection open and yields SSE frames as a Python generator produces them. A generator is the whole trick A normal route handler builds a full response, then returns it. A streaming route returns a generator instead, a function that yields pieces over time instead of returning one value. The framework reads from that generator and writes each yielded piece straight to the open socket as soon as it appears. FastAPI does the same thing under a different name, StreamingResponse: Both frameworks do the same job: hold the connection open, pull the next item from the generator when it's ready, write it out, and repeat until the generator is exhausted. Then close the connection. Where the real model plugs in In production, token stream doesn't split a fake string. It iterates the model's own streaming API and re-wraps each piece as an SSE frame: The endpoint's job is thin. It doesn't decide what to write, it just re-frames whatever the model hands it. Keep the streaming plumbing dumb and reusable, and let the model do the generating. Why generators, not lists If token stream built a list of all

Challenge: Consume the Token Stream