Harden: Big PDFs, Embedding Cost, and Bad Input

robustness and cost

Part of: Chat With Your PDF

Your app works on the one clean PDF you tested with. A real user uploads a 300-page manual, re-uploads the same file twice, asks a blank question, or loses the network mid-embed. This lesson closes those gaps. Embedding costs money, per chunk Every chunk you embed is an API call (batched or not) that costs tokens. A 300-page manual chunked at 800 characters can produce hundreds of chunks, so hundreds of embedding calls on every upload. That bill climbs fast when users re-upload the same document. Cache embeddings by content The fix is simple. Hash each chunk's exact text and skip any chunk you've already embedded. An unchanged re-upload now costs zero new embedding calls. Only new or edited chunks get billed. Guard the edges before you spend anything A scanned PDF with no OCR text, or a user hitting "ask" with an empty box, should fail with a clear message. Not a mysterious empty-vector crash three functions later. Retry the flaky call, but count every attempt Embedding and generation calls both time out now and then. Wrap them in a retry. And when you tally cost, remember that every attempt spent input tokens, failed ones included: Where it earns its keep RAG apps look cheap in a

Challenge: Bill Only New Chunks