Summarizing Old Turns

Memory

Part of: Build a Chatbot

A sliding window has one flaw: it forgets the beginning. The user told your bot their name in turn 2, but by turn 60 that turn slid off the edge and the bot is back to "what should I call you?" Trimming keeps recent context sharp and throws away everything else. There's a smarter trade: keep the gist of the old turns without keeping all their tokens. What rolling-summary memory is Rolling-summary memory compresses old turns into a short paragraph and keeps that paragraph in place of the raw messages. Instead of dropping the first 50 turns entirely, you ask the model to summarize them ("User is Sam, a beginner learning Python; prefers short answers; building a chatbot") and prepend that one block. The recent turns stay verbatim; the ancient ones live on as a compact recap. You end up with two memory layers: a summary of the distant past and a window of the recent present. The bot remembers Sam's name from turn 2 and the exact wording of turn 59. How it works When the history crosses a length threshold, split it: everything except the last keep turns is "old," the rest is "recent." Summarize the old block, then rebuild the history as [summary] + recent. The summary usually rides as a

Challenge: Rolling Summary Budget