Chunking Long Inputs

chunking

Part of: AI Text Summarizer

Paste a whole book into one API call and you hit a wall: the context window . The model can only read so much text at once, and even when a huge input technically fits, you pay for every token you send. The fix is chunking , splitting a long input into pieces small enough to handle. The context window, briefly The context window is the maximum text, measured in tokens , the model can read in one call. A token is roughly 4 characters of English, so a rough estimate is len(text) // 4. A 100,000-character article is about 25,000 tokens: often fine to send, but slow and not free, and some documents blow past any limit. Either way, the professional move on long input is to break it up. Split on boundaries, not mid-word The naive split, "every 2,000 characters," slices words and sentences in half and produces garbled chunks. Split on natural boundaries instead, packing whole units (paragraphs, or at least whole words) into each chunk until it's nearly full: This greedily fills each chunk up to max chars and starts a new one only when the next word won't fit, so no word is ever cut. Real tools chunk on paragraphs first and fall back to sentences, but the packing logic is identical: add wh

Challenge: Chunk Without Splitting Words