Budgeting the Retrieval Window

retrieval budget

Part of: Codebase Assistant

Retrieval doesn't stop at "find the best chunks." It has to stop at "find the best chunks that fit." Send too many and you blow past the context window or the bill; send too few and the answer misses key code. This lesson adds the budget check. What we're building A function that walks chunks in ranked order (best match first, from lesson 4) and greedily keeps whichever ones fit inside a token budget, skipping any that duplicate a chunk already included. Why duplicates happen The same chunk often surfaces twice. A query might match a function by embedding similarity and by the identifier bonus at once, landing it in the ranked list twice with two different scores. Sending the same 40 lines of code to the model twice wastes tokens and adds nothing. Notice this keeps going past a chunk that doesn't fit instead of stopping the moment one is too big. A high-ranked, expensive chunk might not fit while the next few smaller ones still do, and each one you fit is useful context. Stopping early would waste budget you could have spent. Why the token math matters here Every chunk you include is code, and code is often denser than prose, so a character-count estimate can undercount the real to

Challenge: Fit Chunks in the Retrieval Budget