Context Compression · Long-Horizon Agents

Not all reasoning
is worth keeping.

First surfaced in Tandemly Briefing — 2026-09-26. A team working on long-horizon agents asked a quiet question: once an agent has resolved a reasoning step, does it still need that step in context? Their method, ICLR, uses a cheap entropy signal to identify resolved reasoning blocks and prune them on the fly. No training. No separate summarizer. And on 260 benchmark tasks, pruning stale reasoning turned out to improve performance, not just preserve it.

Core finding
Stale chain-of-thought history is not neutral. It adds tokens and can actively work against forward reasoning. Removing resolved reasoning blocks improves reward from 0.699 to 0.718 while cutting input tokens by 25.5%.
scroll to explore

Context fills up with
resolved decisions.

Long-horizon agents generate chains of reasoning before each action. Those chains pile up as the run continues, even after the decisions they informed are long settled.

A long-horizon agent has a particular accumulation problem. It does not just process one prompt. It reasons, acts, observes, reasons again, acts again. Each reasoning step occupies context. Each observation occupies context. Across a complex task with dozens of turns, the context window fills with an archaeology of prior thinking: reasoning steps from early in the run that informed an action three steps back and have played no role since.

The prevailing responses to this accumulation are either to summarize the history (which requires an LLM call, adding its own cost) or to train a compression policy (which requires upfront work and ties the method to a specific model). Neither is free. And neither answers a more fundamental question: how much of that accumulated reasoning history is the agent actually using? Or more precisely: which parts of it are already resolved and doing nothing but occupying space?

Context compression work tends to focus on what to keep. ICLR approaches the problem from the other direction: it asks what can be safely dropped, and uses a signal from the frozen backbone itself to decide.

The question this paper asks

Chain-of-thought blocks are written when the agent is uncertain about something. Once that uncertainty is resolved, is there any reason to keep the block in context? Can a cheap proxy signal identify which blocks are resolved, and is pruning them safe?

Entropy as a
freshness signal.

ICLR uses a frozen model's own probability distribution to gauge whether a reasoning block is still doing work. Low entropy means the block is resolved. Resolved blocks become candidates for pruning.

Entropy, in information theory, measures uncertainty. A probability distribution over a small set of likely tokens is low-entropy: the model is fairly sure what comes next. A distribution spread across many tokens is high-entropy: the model is still working something out. ICLR uses entropy measured over a frozen backbone's predictions on each chain-of-thought block as a proxy for whether that reasoning step is still active.

The logic is: if the model produces low-entropy predictions over a reasoning block, that block describes something the model has already resolved. The agent no longer needs it to make good decisions. ICLR uses this signal per block, computing it with a frozen copy of the model rather than requiring a separate trained classifier or a new summarization call.

Crucially, only chain-of-thought scaffolding is eligible for pruning. Actions, tool calls, and observations are untouched. This is a deliberate design choice: those records form the audit trail of what the agent did and what the environment returned. The reasoning steps leading to those actions are what ICLR removes. The result is that the factual record of the run stays intact while the deliberation that produced it gets compressed.

Prior approach
Keep or summarize. Either retain all prior reasoning in context, or pay an LLM call to produce a compressed summary. Both require choices about what to retain, and both add to cost rather than reducing it cleanly.
ICLR approach
Drop what the model already resolved. Use the frozen backbone's per-block entropy as a directness signal. No training, no additional LLM calls, no separate model. Prune resolved reasoning blocks in-flight. The audit trail stays.
What "training-free" means here

The entropy computation uses the model that is already running. No fine-tuning. No additional parameters. No warm-up period. The method also makes no assumptions about the agent framework: it operates at the level of token blocks in the context window and composes with any harness that exposes the reasoning history.

Compression that
improves the task.

On 260 long-horizon WorkBuddyBench tasks, ICLR cut token usage substantially while increasing reward. That direction of movement is unusual. Most compression approaches aim to preserve performance while reducing cost.

−25.5%
Input token reduction
−14.4%
Output token reduction
−33.3%
Cache-read token reduction

Task reward rose from 0.699 to 0.718 across the 260 WorkBuddyBench tasks. That is a 2.7% gain in reward alongside a 25.5% reduction in input tokens. The finding that reward improves rather than merely surviving compression suggests that some of the pruned reasoning was not neutral filler. It was actively interfering with forward reasoning, and removing it helped.

The cache-read token reduction of 33.3% is separately significant. Prompt caching is one of the more practical cost levers in production agent deployments, and the blocks that ICLR removes are exactly the kind of repetitive prior-context material that fills cached layers. Smaller cached context means lower cache-read costs at real provider pricing.

The caveat to hold

Proxy entropy as a pruning signal has only been validated on WorkBuddyBench's task distribution. How well the signal generalizes to other long-horizon task types is untested. The paper does not report head-to-head results against other compression approaches in the same evaluation setup, so relative positioning against context-as-environment or trained compaction methods remains an open question.

This is not a reason to dismiss the result. It is a reason to run the method against your own task distribution before relying on the benchmark numbers.

A composable lever
for any harness.

ICLR does not require rebuilding the agent. It operates as a filter on the context window's reasoning layers and plugs into existing harnesses without modifying them.

1
For developers running long-horizon agents
ICLR is a drop-in compression layer. No fine-tuning required. The method targets only the reasoning scaffolding in your context window, leaving actions, tool calls, and observations intact. If your agent produces verbose chain-of-thought across many turns, the entropy signal gives you a principled way to prune what has already been resolved rather than guessing.
2
For teams already using context compression
ICLR occupies a distinct position in the compression stack. Context-as-environment restructures how the agent stores and retrieves state across turns. Trained compaction teaches the summarize step into the policy. ICLR sits inside a single run's reasoning stream and prunes resolved blocks without a separate call. These approaches are not in competition: a harness could deploy all three at different layers.
3
For teams tracking cost per task
The 33.3% cache-read token reduction is the number worth watching at real provider pricing. Cached context costs less per token than uncached, but it still costs. Removing resolved reasoning from the cached layer translates directly to lower invoices on high-volume runs. Measure this against your actual billing rather than estimated token counts.
4
For researchers and evaluators
The finding that pruning stale reasoning improves reward is worth studying. If the gain holds beyond WorkBuddyBench, it suggests that accumulated resolved reasoning actively degrades forward reasoning quality. That would be a useful negative result about the structure of long chain-of-thought prompts more broadly, and worth testing on held-out distributions.

Where to go
from here.

Steps for applying or building on this work.

1
Read the paper
Wang, M., Luo, F., Wang, B., Yao, G., Guo, Y., Ning, C., Chen, H., Ma, Y., & Han, J. (2026). When Can Agents Forget Their Reasoning? ICLR for Long-Horizon Agent Context Compression. arXiv:2609.29875.
2
Identify your reasoning scaffolding
Map which parts of your agent's context window contain chain-of-thought blocks versus actions, tool results, and observations. ICLR's target is the CoT layer. If your agent does not produce visible reasoning chains, the method does not apply directly, but the entropy signal idea may still be useful as a context-quality metric.
3
Run entropy diagnostics on a sample
Before applying pruning, compute per-block proxy entropy on a sample of your own task traces. Check whether low-entropy blocks correspond to steps that are genuinely resolved. The signal is only as good as its alignment with your task structure.
4
Validate on your task distribution
WorkBuddyBench is one task distribution. Run the method on a held-out sample of your own tasks before treating the benchmark numbers as predictive. The improvement in reward may not replicate uniformly across task types.
5
Measure billed cost, not estimated tokens
The cache-read reduction is the most directly monetizable result. Track its effect against actual provider billing rather than token-count estimates, following the same discipline as the Token Reduction Is Not Cost Reduction finding already documented in this research library.