AI Agents
A Context Window Is Not Memory
Re-reading the same tokens on every turn is not the same as remembering them, and the difference is your bill.
Every few turns, an agent re-reads the same files. It re-sends the same tool output. It re-states rules that were already established. From the outside this looks like forgetting, and from the inside it looks like paying twice for the same thing.
It is neither. It is the difference between a window and a store.
What a context window actually is
A window is a fixed-size buffer of tokens that is re-submitted on every call. There is no state. Whatever is not in the buffer does not exist for the model — including everything you said twenty turns ago and everything it read on turn three.
So a long session has two options: truncate, and quietly forget; or keep everything, and quietly pay for it on every single turn.
Re-reading the same tokens on every turn is not remembering them. It is re-transmitting them.
What memory has to be
Useful memory is small, stable, and selected. The failure mode is storing everything and letting the window do the selection, which gives you all the cost and none of the curation.
- Durable facts — the project's conventions, the decisions already made and why.
- Summarised state — what has been tried, what the current failure is, not the whole transcript.
- Retrievable detail — old file contents that matter again, fetched on demand rather than carried.
The third item is the one people skip, and it is what makes the difference between a session and a project. Most context should be retrievable, not resident.
A cheap rule that helps
Write the decision, not the deliberation. A line saying "chose keyset pagination because offsets were O(n) at 200 rows" is worth carrying forever. The four turns that got there are worth carrying for exactly one turn.