Long-Context Agents · Reinforcement Learning

When the window runs out,
the summary is the agent.

First surfaced in Tandemly Briefing — 2026-07-24.

Long-horizon agents eventually fill their context window. Most systems handle this with a static summarizer bolted on at inference time. Researchers at Tsinghua University trained the summarization step into the RL policy itself, so agents learn to compress what they actually need to finish the task, not what sounds informative in the abstract.

Core concept
Context compaction as a trained behavior: CompactionRL teaches agents to produce task-preserving summaries of their own history through reinforcement learning, with credit flowing across the compaction boundary so early actions still get rewarded for late outcomes.
scroll to explore
01The problem

Context windows fill.
The task does not stop.

Every long-horizon agent eventually hits the same wall: the context limit. What a system does at that wall determines whether the task survives it.

A software engineering agent working through a complex codebase, or a terminal agent navigating a multi-step debugging session, accumulates a lot of history. Tool outputs, intermediate reasoning, observations from earlier steps, all of it piles up. At some point the context window is full.

The naive solution is truncation: drop the oldest content and keep going. That works until the truncated content turns out to matter. For tasks that span dozens of steps, the content that gets truncated is often the content that contains the key constraint, the relevant file path, or the error message the current step is supposed to address.

A more sophisticated approach is a static summarizer: a separate model trained on summarization data that condenses the context at intervals. This is better than truncation, but it optimizes for something adjacent to the goal. A summarizer trained on human summarization data learns to preserve what a human reading the history would find interesting. That is not the same as preserving what the agent needs to complete the task it is on.

The gap between a good summary and a task-preserving summary is what CompactionRL is built to close.

The question this paper asks

What if context compaction were not an inference-time heuristic applied to the agent, but a behavior trained into the agent alongside the task it is trying to complete? Would the summaries be better? Would the task completion rate improve?

02The experiment

Joint training:
task and summary together.

CompactionRL reframes context compaction as a component of the agent's learned policy, trained jointly with task execution rather than added as a separate post-hoc module.

The key insight is that whether a summary is good depends entirely on what the agent needs to do next. A summary that looks comprehensive to a human reviewer might omit the precise detail the agent requires at step 47. The only way to know what matters is to train the summarization step against the outcomes the agent is trying to achieve.

CompactionRL introduces two mechanisms to make this training work:

Standard approach
Separate summarizer. An external model is trained on summarization data. At inference time, when the context fills, the external model compresses the history. The agent continues with the compressed context. Credit assignment during RL training does not cross the compaction boundary.
CompactionRL
Joint RL training. The agent is trained to both execute tasks and generate task-preserving summaries of its own history. Token-level loss normalization and cross-trajectory credit assignment ensure both objectives get appropriate weight during training.
Mechanism 1: Token-level loss normalization

During RL training, two loss signals compete: one for task-execution actions and one for summary-generation tokens. Without explicit balancing, the task-execution signal typically dominates because there are far more task steps than compaction events. Token-level loss normalization reweights both signals so neither crowds out the other. The summary generator learns from outcomes, not just from the structure of good summaries.

Mechanism 2: Cross-trajectory GAE

Standard Generalized Advantage Estimation (GAE, the technique RL systems use to compute how much credit an action deserves for later outcomes) stops at context boundaries. This means actions taken before a compaction event cannot receive credit for what happens after the summary is generated. Cross-trajectory GAE extends advantage estimation across those boundaries. An action at step 12 that turned out to matter for the outcome at step 60, even though a summary happened at step 30, now gets credit for that outcome.

Fixed context budget

The evaluation holds the peak context window size constant. CompactionRL does not improve outcomes by giving the agent more room to work. It improves outcomes by using the existing room more effectively. This is a meaningful constraint: the benchmark is efficiency, not scale.

03Findings & benchmarks

Seven points on
a fixed budget.

CompactionRL was evaluated on GLM-4.5-Air across two standard long-horizon benchmarks. Both showed meaningful gains over the baseline, and ablations confirm the summary training step's contribution is real.

66.8%
SWE-bench Verified
Pass@1 on coding tasks. Baseline was 59.8%. The +7.0 percentage point lift reflects better context preservation across long software engineering sessions.
24.5%
Terminal-Bench 2.0
+3.1pp over baseline on multi-step terminal tasks. Terminal-Bench 2.0 is a harder environment with longer action sequences and more context-dependent state.
fixed
Context budget
Both results achieved under a fixed peak context size. The gains come from better compaction, not a larger context window.
Ablation results

Removing token-level loss normalization degrades summary quality. Removing cross-trajectory GAE reduces the agent's ability to preserve information that matters for distant future steps. Both mechanisms contribute independently. The combined system outperforms each component in isolation.

The ablation finding matters because it rules out the simpler hypothesis that any summarization during training is sufficient. The specific design choices drive the gains.

Scope and caveats

Results are on one base model (GLM-4.5-Air) and two benchmarks. Whether the gains transfer to other model families, other task domains, or shorter-horizon tasks is not established by this paper. Benchmarks with very short average episode lengths may not see the same benefit, since context pressure is the prerequisite for compaction to matter.

04Practical takeaways

What this means
for agent builders.

CompactionRL is primarily a training-time result, which means the practitioners it matters to most are teams building and fine-tuning long-horizon agents, not teams deploying off-the-shelf models. But the evaluation framing carries lessons that apply more broadly.

1
For teams training RL-based agents
If your agent operates on long-horizon tasks and you use context compaction at inference time, the compaction step is a candidate for inclusion in your RL training loop. A static summarizer applied after training is a plausible baseline; it is not the ceiling. The question to ask is whether the compaction step is making decisions the task reward can inform.
2
Credit assignment across boundaries
Any time a training loop involves a boundary that resets or compresses agent state, standard credit assignment stops working at that boundary. Cross-trajectory GAE is one solution. The broader lesson is to audit your RL training pipeline for state boundaries and confirm that actions before them can still receive credit for outcomes after them. If they cannot, the training signal is incomplete.
3
Evaluate under a fixed context budget
The field's default framing treats longer context windows as strictly better. That framing makes it easy to confuse scale with efficiency. Evaluating context management methods under a fixed peak budget forces a cleaner measurement of whether the method actually improves how the agent uses the context it has. Teams benchmarking their own agents should consider adding fixed-budget evaluations alongside unconstrained ones.
4
For teams using static summarizers today
A summarizer trained on summarization data is a reasonable starting point. It is worth testing whether the summaries it produces actually preserve the information the agent uses in subsequent steps. The test does not require retraining. Instrument the agent, run episodes with compaction, and check whether the post-compaction steps reference information that was in the pre-compaction context but not in the summary. That gap is the training signal CompactionRL internalizes.
05Getting started

Where to go
from here.

If you want to go deeper on context compaction, cross-boundary credit assignment, or long-horizon agent evaluation.

1
Read the paper
Li, Hou, Jing, Tang & Dong. (2026). CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents. Tsinghua University. arXiv:2607.05378. The methods section on token-level loss normalization and cross-trajectory GAE is worth reading in full before implementing.
2
Run SWE-bench Verified under a fixed budget
SWE-bench Verified and Terminal-Bench 2.0 are public benchmarks. Running your agent under a fixed peak context constraint and comparing to unconstrained performance gives you a baseline measurement of how much context pressure your agent already faces. The gap between the two is the problem CompactionRL addresses.
3
Audit your compaction boundaries for credit assignment gaps
If you are fine-tuning an agent with RL and it uses any form of state compression or summarization mid-episode, check whether your advantage estimates cross that boundary. Most standard RL training setups do not. The cross-trajectory GAE approach in this paper is one concrete way to fix that; the broader pattern is to treat compaction events as parts of the trajectory, not resets of it.
4
Instrument your summaries
Before adopting CompactionRL-style training, measure whether your current summarizer preserves task-critical information. Log what information the agent accesses from the compressed context in the steps following each compaction event. Information the agent needs but cannot find in the summary is the signal that the static summarizer is falling short. This instrumentation takes an afternoon and tells you whether the training investment is likely to pay off.