Sending a Screenshot to the Model

encoding images for the API

Part of: Screenshot Describer

A vision model doesn't take a file path. It takes bytes, turned into text, sitting inside the same message list you already use for text-only prompts. This lesson builds the one new piece every later lesson depends on: getting an image into the request correctly. What we're building By lesson 8 you have a tool. Point it at a screenshot, get back a structured description or matching starter HTML. Every piece stacks on this first one, an image the model can actually read. Images ride as content blocks A text-only user turn is a plain string. An image turn is a list of content blocks , an image block and a text block together in one message: Why base64 Raw bytes can't travel inside JSON, and JSON is what the API speaks. Base64 re-encodes bytes as plain text characters (letters, digits, a few symbols) that drop safely into a string field. base64.b64encode does the encoding. .decode("utf-8") turns the resulting bytes object into a normal Python string you can put in a dict. The shape to memorize Every image turn has the same skeleton. An image block first, with a nested source dict naming the encoding, the media type, and the data. Then a text block with your instruction. The model does

Challenge: Estimate the Base64 Payload Size