Retrieve Then Ask: Assembling the RAG Prompt

the RAG prompt

Part of: Chat With Your PDF

You can find the chunks most relevant to a question. The next step turns "here are three relevant chunks" into a prompt the model answers from correctly, without inventing anything the chunks never said. The RAG prompt has a fixed shape A RAG prompt always carries three parts: the rules, the retrieved context, and the question. Notice each chunk gets a number label : [1], [2], and so on. That label lets the model cite its source, and it lets you check later that a citation points at a real chunk rather than a number the model invented. Why the system prompt is worded this way Three lines carry most of the weight of a trustworthy RAG app: 1. "ONLY the provided context" stops the model from answering out of its own general knowledge when the document doesn't cover the topic. That's the biggest single source of RAG hallucination. 2. "Cite the source number(s)" gives you a way to verify the answer is grounded, and gives the user a way to check it too. 3. "If the answer is not in the context, say you don't know" hands the model a permitted way to refuse instead of confabulating a plausible wrong answer. Calling it Everything before this call is search. Everything from here is a normal m

Challenge: Assemble the Context Block