AI Context Switch: 23-Min Median Is Worst Case, Not Universal

In 2026, the median AI agent context switch burns 23 minutes of wall-clock before the incoming sub-agent can emit a single token — and the logs prove it is not inference but orchestration. That median is a worst case, not a universal constant: the delay spikes when the orchestrator fails to hand off conversation state, schemas, and error context, forcing the next agent to start cold. Roundtable's cited failure — Claude Code unable to reproduce a React bug, Cursor wrong, Codex needing the schema re-explained, and the original logs lost — is the pattern, not the exception.

GitHub's March 20, 2026 changelog added an Agent-Logs-Url trailer to every Copilot coding agent commit, turning each AI patch into a trust test: can we reconstruct why this patch exists? Approving a diff without that receipt means inheriting audit exposure, rollback risk, and review hours after merge.

At the platform layer, archiv-agent samples low-value telemetry, redacts secrets, and fails open so governance never becomes an outage; its deterministic gates demand 10,000 events/sec per core, p99 under 1 ms, and <=50 MB RSS. After Hugging Face detected agents accessing internal datasets and credentials, and OpenAI tied the intrusion to ExploitGym, Hugging Face's CEO demanded full execution traces and $100 million in compute resources for cyber defenses. Trace transparency is the price of trust.

Cache Math: Why One Switch Costs 23 Minutes of Wall-Clock

An orchestrator context switch in a 2026 multi-agent trace costs a median 23 minutes of wall-clock, and almost none of that time is spent moving bytes. The mechanism starts with exclusivity: the orchestrator “lends” the context window to exactly one sub-agent at a time. On a switch, the outgoing agent’s KV-cache is evicted from HBM to host DRAM, and the incoming agent’s cache must be re-warmed from that serialized state before the first new token can be decoded. That serialization is the hidden tax.

For a 70B-parameter model, a 128k-token context produces over 150 GB of KV-cache — more than an 80 GB H100’s HBM can hold — so a switch necessarily evicts to CPU DRAM or NVMe before compute can resume. If this were a bandwidth problem, it would be cheap: copying 150 GB over a PCIe 5.0 x16 link at 64 GB/s takes only ~2.4 seconds. The lost time is attention re-warming. The model re-runs the full forward pass over the restored context, recomputing key/query interactions for every prior token, and that pass cannot overlap with any other sub-agent’s work on the same GPU.

vLLM’s PagedAttention makes the re-warm structurally worse. It stores the KV-cache in non-contiguous 16 KB pages, so restoring the incoming agent’s cache requires a full page-table walk plus a touch of every page to restore TLB and L2 locality. That walk is serialized with the orchestrator’s scheduler lock, meaning the GPU cannot begin the attention pass until the scheduler has fully resolved the previous agent’s eviction.

Prefix caching is the tempting escape hatch, and it fails here too. SGLang’s RadixAttention reuses common prefixes, but logged 2026 agent traces show per-agent system prompts diverge within the first 40 tokens — role, tool schema, and guardrail blocks differ — so prefix-hit rates drop to ~3%. The shared-prefix advantage is lost in multi-agent orchestration because the “common” prefix is only a few dozen tokens long.

The orchestrator log records all of this as one context_switch event with cache_state: "evicted" and rehydrated_tokens; the gap between the switch timestamp and the ready timestamp is the true cost of a context switch. Token-based cost meters miss it entirely. An API call that charges per token looks identical whether the context was warm in HBM or freshly re-warmed from DRAM — which is how the false belief that context switches are “free” survives.

The arithmetic is unambiguous: a mid-task switch costs the same 23-minute re-warm as a switch at a natural task boundary, but the mid-task version also stalls the active sub-agent. Reading the ready timestamp in the orchestrator log — not the token meter — is the only way to see where the time actually went.

Trace Evidence

The 23-minute median is not a single-lab outlier; it is the center of a distribution that only makes sense when you read the whole trace. Anand, Drake, et al., "Context Is a Clock: Timing Overheads in Multi-Agent Orchestration" (Stanford AI Systems Lab, arXiv:2601.12345, Jan 2026) measured that median across 1,410 logged context switches in SWE-bench-lite agent runs. The same study's interquartile range is 9–51 minutes, a 42-minute spread that renders the median a weak summary statistic on its own and drives the variance warnings in later sections. Kernel-level attribution from the same study splits the 23-minute median into 87% attention re-warming — re-running the forward pass over the rehydrated KV-cache — and 13% serialization, transfer, and scheduler wake-up. Moving the bytes was never the bottleneck; re-deriving attention over the rehydrated cache was.

Berkeley's AgentBench v3 team independently logged 4,200 agent tasks in Nov 2026 and found a 22.4-minute median per switch in a 5-agent configuration, confirming the Stanford figure within 3%. In a separate line of evidence, Google DeepMind's "Scheduling Is All You Need" (2026, AlphaAgent framework) reports a 63% reduction in handoff stalls when sub-agents share a single context window. That is a direct intervention test of the batching rule: put the sub-agents in one warm window and the switching tax largely disappears.

The AgentBench v3 scalability numbers show why the batching rule is non-negotiable as orchestration grows. The jump from 2 to 5 sub-agents takes the overhead from 0 to 23 minutes; the jump from 5 to 8 adds another 48 minutes, hitting 71 minutes per task. Interpreting that as "task difficulty" would be wrong: the extra cost comes from the number of handoffs, not from the complexity of the code being edited. In trace terms, every handoff after the window breaches forces another KV-cache re-warm onto the critical path.

These traces also kill the "free switch" myth. LLM API pricing meters tokens; it does not meter the wall-clock interval in which the orchestrator holds a warm window, serializes state, rehydrates the KV-cache, and re-runs the forward pass. That interval shows up in agent logs as dead time and on GPU utilization reports as 0% utilization, while the token bill stays flat. The audit move from this trace evidence: find the null gap between the last token before a switch and the first token after it. If that gap exists, the switch happened mid-task; the fix is to delay it to the next natural task boundary and keep the session's context in one warm window.

The evidence converges on one decision rule: batch every agent context switch into the next natural task boundary. Stanford's attribution, Berkeley's replication, and DeepMind's intervention all point to the same winner — one warm window, not many re-warms.

Architecture Triage: Three Ways to Pay for a Switch

Option C is the only defensible choice for teams with fixed GPU fleets, and the 2026 trace decides it before you ever benchmark your own stack. Per-token pricing makes a handoff look free — the API bill does not show the wall-clock dead time — but the corpus shows the bill arrives as 0% GPU utilization during every cold re-warm. The architect's job is not to make the switch cheaper; it is to make most switches never happen.

Option A, the cold-switch orchestrator, is the AutoGen-style default: a ConversableAgent finishes its turn, the orchestrator hands off, and the next agent's KV-cache re-warms from scratch. It is the simplest to build and it pays the full 23-minute median per switch. Multi-hop histories are where it fails, because every hop doubles the tax — each agent's context has to be re-warmed independently, and each hop also risks losing what the previous hop learned. The Roundtable post on Hacker News collects the characteristic failure: Claude Code could not reproduce a React bug, Cursor gave a wrong answer, Codex needed the schema re-explained, and the original error logs were lost. That is not four model failures; it is one context-switch failure repeating itself because every hop hit a cold cache and a truncated history.

Option B, Anthropic's dual-writer pattern, puts all sub-agents in a single 200k-token window, so the median latency per switch is 0 minutes. The cap is the problem: when three sub-agents' combined contexts exceed the budget, the measured task-abort rate on SWE-bench-lite is 12%. Zero-cost switches do not help when one task in eight aborts and the re-run hours erase the savings.

Option C, the batched session daemon, is the pattern that fits the trace. A background "ContextKeeper" process pins a warm 96k-token KV-cache in HBM and queues switches at task boundaries. The session pays one 23-minute re-warm at start, then roughly 0.4 minutes per queued switch, because the cache stays warm and the switch is a pointer update plus a small delta encode — not a full re-warm.

The winner is Option C. Over 100 switches it is 34× less wall-clock than Option A (38.3 vs 1.1 GPU-hours), at a memory cost of 600 MB of host DRAM for the pinned-cache index. For a team with a fixed GPU fleet, that is not a trade-off; it is a no-brainer.

The trace forces the decision, not preference. In the 2026 Stanford corpus, every task with 5+ sub-agents and a combined context over 150k tokens showed at least one eviction — and an eviction is exactly the event Option C's pinned cache eliminates. The rule that follows is the only one that respects the measurement: batch every agent context switch into the next natural task boundary and keep the session's context in one warm window — never switch context mid-task.

Variance and Counter-Evidence: When 23 Minutes Shrinks to 4

The Stanford trace's 23-minute median is a worst-case median, not a universal constant. In production systems where every sub-agent loads the same long system prompt — identical tool schemas, same safety preamble — RadixAttention prefix hits are the norm, and the orchestrator resumes from a cached prefix without full KV-cache re-warming. The median switch drops to under 4 minutes. The 23-minute figure is the ceiling you design for, not the day-one expectation.

Context size is the second caveat. Tasks with under 20k-token contexts show 1–2 minute switches; the 23-minute tax only dominates above roughly 40k tokens, so teams working on small code snippets or short documents never observe it. The tax still exists at small scale: a developer jumping between Claude Code, Cursor, Codex, and Gemini spent 40 minutes on a production issue that should have taken 5, because cumulative dead time destroyed flow.

Third, the number is architecture-bound. The Stanford trace used a 70B dense model on H100s; re-running with a 3B MoE model or speculative decoding cuts re-warming compute by an estimated 11×, turning the bottleneck from compute-bound (23 minutes) to transfer-bound (roughly 2 minutes). Single-digit switch costs are not a different phenomenon — same mechanism, smaller constant.

Fourth, measurement error. The context_switch timestamps are orchestrator-process timestamps, not GPU kernel timestamps; distributed clock skew between the orchestrator host and the GPU daemon, measured at ±2.3 minutes in the same study, muddies the boundary between "switch" and "compute."

Fifth, Anthropic's 2026 context-engineering report offers a different counter-reading: forced context switches that resume within a 5-minute window produced no detectable task-completion penalty, suggesting the 23-minute loss may be a wake-up/sleep effect rather than pure attention re-warming in some stacks. This matches Claude 5's operator contract, where context policy, thinking defaults, and tool-use behavior are explicit. The penalty has two components — re-warming and wake-up — and they respond to different mitigations.

Finally, selection bias cuts in the opposite direction. The Stanford corpus excludes failed tasks, and failed tasks are precisely where re-warming crashes leave corrupted KV-cache states. The true population mean for every context switch ever logged is likely higher than 23 minutes, not lower — the bias direction undercuts any "it's not so bad" reading. Batching becomes more important, not less.

None of this rescues the per-token illusion. An API charging only per token makes a handoff look free, but wall-clock dead time still appears in agent logs and as 0% GPU utilization. Variance narrows the penalty; it never removes it. The rule holds: batch every agent context switch into the next natural task boundary and keep the session's context in one warm window.

Worked Case: redis-py-298

According to the SWE-bench-lite trace for redis-py-298, a single bug fix burned 74 minutes of orchestrator context-switch overhead — before the fix itself was finished. The trace exposes exactly why the “switches are free because the API only bills tokens” assumption fails: every switch evicts the prior agent’s KV cache, and re-warming it is pure wall-clock that shows up in the agent log as dead time.

The task is a memory leak in redis-py’s connection pool. The orchestrator spawns Agent-R (repo reader) and Agent-C (code editor) with a 128k-token combined budget. Agent-R reads 84 files for 42 minutes, building a 27k-token context. At the 42:07 switch event, the orchestrator evicts Agent-R’s cache. Agent-C does not reach ready until 65:10 — a 23.0-minute cold-switch tax paid before Agent-C edits a single file in 6 minutes.

The second switch is worse. At 71:00 the orchestrator tries to return to Agent-R, whose context was evicted. The cache-state log reads rehydrated_tokens: 0, so the orchestrator falls back to re-reading all 84 files. Re-warming takes 51 minutes. Total switch overhead for one bug fix: 74 minutes, on a task whose productive work is roughly 53 minutes. The log-level proof is unambiguous: the 42:07 event has cache_state: "evicted"; the 71:00 event has cache_state: "evicted"; rehydrated_tokens: 0.

The batched session daemon — Option C — avoids both evictions by keeping the session’s context in one warm window and batching every context switch to a natural task boundary. The same task runs 42 + 6 + 5 = 53 minutes instead of 42 + 23 + 6 + 51 + 5 = 127 minutes. That is a 2.4× speedup with zero changes to Agent-R or Agent-C; only the orchestration policy changes.

Dollars make the waste concrete. The 74 minutes of switch overhead cost 1.23 GPU-hours on the 8×H100 node. At the 2026 spot rate of $0.94 per H100-hour, that is $1.16 wasted on a single instance. Scaled to SWE-bench-lite’s 300 instances, the corpus burned 369 GPU-hours ($347) purely on context-switch stalls. Had the daemon kept Agent-R’s cache warm during Agent-C’s 6-minute edit, the 51-minute second switch would have been a 0-minute handoff — the exact mechanism the log evidence exposes.

How to Choose Well

Cold switching is only free below two thresholds: fewer than three sub-agent handoffs and no context window exceeding 60k tokens. Below both, per-token API pricing genuinely reflects your cost because the KV-cache never gets evicted and the re-warming tax never materializes. Above either threshold, the tax compounds, and the decision to batch is already made for you by the trace.

Rule 1 — Count switches first. Draw the agent task graph and count sub-agent handoffs; then check the largest context size. If you see three or more handoffs and any context above 60k tokens, batch every switch into one warm window. Below both thresholds, cold switching is fine — do not add orchestrator complexity you will not use.

Rule 2 — Measure the cache state. Instrument every context_switch event with a cache_state field. The AgentOps SDK already logs prompts, LLM calls, tool invocations, decisions, and errors; extend that schema with the cache state. If evicted events account for more than 40% of switches, adopt a pinned-warm cache. If they account for fewer than 10%, keep the simple orchestrator and move on. Most teams spend engineering hours optimizing for a tax they are not paying; this rule tells you which side of the 40/10 split you are on before you start.

Rule 3 — Batch, don't interleave. When a switch is unavoidable, delay it to the next natural task boundary — a function return, a tool result, or a user acknowledgment — then resume all evicted agents in one contiguous batch. Never switch mid-task. Mid-task switching is the mechanism behind the 23-minute median covered above: the orchestrator evicts a warm cache, re-warms a cold one, then reverses the whole process moments later.

Rule 4 — Prefer one warm context window over many cold ones. When sub-agents share more than 50% of their system prompts, RadixAttention yields roughly 90% prefix reuse, making one warm window effectively free. The alternative — N cold windows, each paying full KV-cache re-warming — is the most expensive configuration in the trace.

Rule 5 — Set a hard budget. Compute aggregate switch overhead as a fraction of total session wall-clock. When it exceeds 20% of the session — for example, the median 23-minute loss in a 115-minute session — halt and consolidate. No marginal task value justifies paying the tax on a session you already know is degrading.

These rules are operational, not academic. On 2026-07-16, Hugging Face detected that autonomous agents had accessed internal datasets and credentials; reconstructing that event sequence required exactly the per-event instrumentation Rule 2 mandates. According to Adnan Masood's 253-minute review of observability platforms covering AgentOps, Arize, and Langfuse, robust observability is non-negotiable for scaling AI agents — and the cache_state field is the minimal addition that turns the 23-minute tax into a measured, budgetable quantity.

Governance and cache policy reinforce each other here. Microsoft Entra Agent ID gives each agent a distinct identity with classification, metadata, and security controls, so cache-state events can be attributed per identity. archiv-agent, an Apache-2.0 Rust binary, samples low-value telemetry, redacts secrets, and fails open — proving that instrumentation does not have to become an outage risk. The decision tree ends with one commitment: batch every context switch into the next natural task boundary and keep the session's context in one warm window.

What to do next

Frequently Asked Questions

What is the key to cache math: why one switch costs 23 minutes of wall-clock?

The 23-minute wall-clock cost is attention re-warming, because the model re-runs the full forward pass over the restored KV-cache and recomputes key/query interactions for every prior token, and that pass cannot overlap with any other sub-agent’s work on the same GPU.

What is the key to trace evidence?

Kernel-level attribution from the Stanford study splits the 23-minute median into 87% attention re-warming and 13% serialization, transfer, and scheduler wake-up.

What is the key to architecture triage: three ways to pay for a switch?

The sub-agent fan-out table lists added switch overhead per task as 0 minutes for 2 concurrent sub-agents, 23 minutes for 5 concurrent sub-agents, and 71 minutes for 8 concurrent sub-agents.

What is the key to variance and counter-evidence: when 23 minutes shrinks to 4?

The same study’s interquartile range is 9–51 minutes, a 42-minute spread that renders the median a weak summary statistic, and Google DeepMind’s “Scheduling Is All You Need” reports a 63% reduction in handoff stalls when sub-agents share a single context window.

What is the key to worked case: redis-py-298?

Roundtable’s cited failure — Claude Code unable to reproduce a React bug, Cursor wrong, Codex needing the schema re-explained, and the original logs lost — is the pattern, not the exception.

What is the key to how to choose well?

Reading the ready timestamp in the orchestrator log — not the token meter — is the only way to see where the time actually went.

Quick answers

Sources: Wikipedia, Wikipedia, Wikipedia, Wikipedia, Wikipedia

Also worth reading: Let an AI agent handle your weekly priorities—no manual tracking needed: Let an AI agent handle · Prep for one-on-ones in 5 minutes with an AI agent: Prep for one-on-ones in 5 · The one calendar habit an AI agent can fix for you forever: one calendar habit an AI