What are the best context compaction strategies for AI agents in 2026?

Context compaction is the practice of reducing what sits in an agent's working context window without destroying the information the agent needs to keep working well. As of August 2026, it has moved from a niche engineering trick to a core discipline in agent design, largely because token costs, latency, and error rates all scale with context size. Anthropic's widely cited guidance on effective context engineering for agents framed the shift clearly: models degrade when contexts are stuffed with stale tool outputs, old file dumps, and redundant reasoning traces. The question is no longer whether to compact, but which strategy to use for which workload.

What Context Compaction Actually Means

Also worth reading: How do you go about securing model context protocol servers for production AI agents? · What are the most effective agentic AI risk mitigation strategies for executives and personal productivity systems? · How do outcome based AI pricing strategies work for modern software platforms?

An agent's context window is finite working memory. Every tool call result, every retrieved document chunk, every intermediate reasoning step accumulates there. Left unmanaged, a coding agent running a long test-fix loop can burn through a 200K-token window in under an hour, and quality drops before the hard limit hits — retrieval accuracy and instruction-following measurably decline as irrelevant tokens pile up. Compaction strategies fall into four broad families: summarization (replacing history with a compressed narrative), structured extraction (keeping only task-relevant fields), externalization (moving state to files or memory stores and referencing them by pointer), and output compression (shrinking tool results at the source). Most production systems in 2026 combine two or three of these rather than betting on one.

The distinction between compaction and truncation matters. Truncation simply deletes the oldest messages; Context-compact, one of several open-source tools highlighted on Hacker News this year, was built specifically because naive truncation breaks agents mid-task — the model forgets its own plan and starts looping. Summarization-based compaction instead asks the model (or a cheaper secondary model) to write a state-of-the-task document: goal, completed steps, open questions, key file paths, constraints. That summary replaces thousands of tokens of raw history while preserving decision continuity.

Why Agents Degrade Without Compaction

Three failure modes dominate. The first is context rot: as irrelevant content accumulates, the model pays attention budget to noise, and instruction adherence slips. Teams building legal-agents at Harvey reported that extending their Legal Agent Bench to M&A due diligence required aggressive context hygiene precisely because due-diligence workflows generate enormous document dumps that drown out the actual task instructions. The second failure mode is lost memory across sessions or after compaction events — O'Reilly's piece on teaching agents to detect and recover from lost memory documented how agents silently proceed with corrupted assumptions when earlier decisions vanish from context. The third is cost: paying full input-token prices on every turn for a bloated history multiplies spend linearly with conversation length, which is brutal for always-on executive-assistant style agents that run continuously.

There is also a latency argument. Long contexts slow prefill on every request. A personal productivity agent checking your calendar, inbox, and task list dozens of times per day cannot afford multi-second prefill penalties on each call. OpenAI's disclosure that enabling two settings tripled their scores on the ARC-AGI-3 benchmark underscored how much headroom exists purely in context management — the model didn't change, the context did.

Strategy One: Recursive Summarization

Recursive summarization is the default choice for conversational and task-execution agents. When context crosses a threshold — commonly 70–80% of the window — the system compresses older turns into a structured summary and keeps recent turns verbatim. The threshold matters more than most teams realize: compacting too early loses detail the agent still needs; too late and you pay peak costs and risk degraded behavior before the trigger fires. A practical pattern keeps three layers: a durable system prompt, a rolling task-state summary (500–2,000 tokens), and a verbatim tail covering roughly the last 10–20% of interactions.

Quality depends heavily on the summary schema. Free-form prose summaries drift and omit operational details like exact file paths, API parameter values, or user-stated preferences. Structured templates — objective, constraints, done/not-done, artifacts created, next action — produce far more reliable continuations. Anthropic's context-engineering guidance recommends treating the summary as a handoff document written for a fresh instance of the same agent, which is exactly the right mental model: assume total amnesia except what you wrote down.

Strategy Two: Externalized Memory and State Files

Externalization moves information out of the context window entirely and stores it in files, databases, or dedicated memory services, leaving only pointers in context. Cloudflare's Agent Memory launch in 2026 signaled that infrastructure vendors now treat agent memory as a first-class primitive rather than an application-layer afterthought. In this pattern, an agent writes intermediate results to a scratchpad file, then references "see /state/analysis-draft.md" instead of re-including the draft. Filesystems become the agent's long-term memory, and the context window becomes a cache of pointers plus whatever is actively needed.

This scales better than pure summarization for long-running work — a chief-of-staff-style agent managing weeks of projects can't hold everything in any summary, but it can index hundreds of notes and retrieve on demand. The tradeoff is added complexity: retrieval must be accurate, or the agent wastes turns fetching the wrong artifact. Hybrid designs are common, where summaries cover conversational flow and external storage covers bulky artifacts like documents, datasets, and code diffs.

Strategy Three: Tool Output Compression at the Source

The cheapest tokens are the ones never generated. Tool output compression shrinks results before they enter context. The Oo project demonstrated the idea vividly: compressing cargo test output from hundreds of lines of compiler diagnostics down to "47 passed, 2.1s" unless failures exist, in which case only failing cases are shown in full. Applied broadly, this means log tailing returns counts and anomalies, web scrapes return extracted facts rather than raw HTML, and database queries return aggregates with drill-down available on request.

Deterministic sidecars have emerged as a lightweight implementation path. Mem, a CLI memory sidecar for developer workflows, shows the appeal: a small deterministic process manages what enters and leaves context, avoiding the cost and nondeterminism of asking an LLM to summarize its own tool traffic. For high-frequency operations — builds, test runs, API health checks — deterministic compression beats LLM summarization on both cost and reliability, reserving model-based summarization for genuinely ambiguous conversational history.

Comparing the Main Approaches

No single row wins outright. A pragmatic 2026 stack runs deterministic compression on all tool outputs, externalizes large artifacts to files or a memory service, and applies recursive summarization to the conversational layer above. Teams that skip the first layer almost always regret it, because raw tool spam is typically the largest single contributor to context bloat.

Common Mistakes and How to Avoid Them

The most frequent mistake is compacting without preserving the plan. Agents loop endlessly when the summary describes what happened but not what remains — always include explicit next actions and unresolved blockers. Second is summarizing with the same expensive model on every trigger; routing compaction to a smaller, cheaper model cuts overhead substantially with modest fidelity loss, though critical handoffs may warrant the primary model. Third is ignoring detection: O'Reilly's lost-memory research showed agents rarely notice their own amnesia, so production systems should inject periodic self-check prompts ("state your current objective and the last confirmed fact") to catch corruption early.

A fourth mistake is over-compacting interactive sessions. If a human is actively steering, keeping more verbatim history improves responsiveness to corrections; aggressive compaction suits autonomous background runs far more than collaborative ones. Finally, many teams measure nothing. Track tokens per task, compaction frequency, and post-compaction success rate — if task completion drops after compaction events, your schema is losing load-bearing details, and no amount of tuning elsewhere will fix it.

When to Act and What It Costs

Act once any agent routinely exceeds roughly 50% of its context window in normal operation, or when per-task token spend makes unit economics uncomfortable. For a personal productivity or executive-chief-of-staff agent running continuously, that threshold arrives quickly — calendar triage plus email drafting plus research easily generates tens of thousands of tokens daily. Cost math is straightforward: input tokens are billed on every turn, so a 100K-token bloated context queried 50 times a day costs 5M input tokens daily versus perhaps 800K with disciplined compaction — an 80%+ reduction that compounds monthly. Model-based summarization adds its own small cost per compaction event, but deterministic compression and file externalization are essentially free beyond engineering time.

Implementation sequencing matters. Start with tool-output compression since it requires no LLM calls and delivers immediate wins. Add rolling summarization next, using a structured template. Introduce external memory only when tasks span days or accumulate artifacts too large for any summary. Revisit thresholds quarterly as models gain longer windows — SmolLM3-class small models already handle long contexts on-device, and the economics of what deserves compaction keep shifting.

Where This Is Heading

The direction of travel is toward managed context as infrastructure. Cloudflare shipping Agent Memory, independent Rust crates for agent construction lowering the barrier to custom runtimes, and benchmark gains attributable purely to context settings all point the same way: within another product cycle, expect frameworks to ship compaction policies as configuration rather than code. But the fundamentals won't change — an agent performs only as well as what you choose to keep in front of it. Treat the context window as expensive real estate, curate it deliberately, and verify after every compaction that the agent still knows what it's doing and why.