Self-improvement
during the run.

When an LLM agent fails a long-horizon task, the standard response is to learn from the failure after the fact. The trajectory is stored, analyzed, and used to update skills, fine-tune weights, or adjust prompts. The next run starts with those updates in place. This pipeline works. But it has a built-in delay: the agent burns through its entire budget on the current run, discovers it failed, and then learns something it cannot apply until the next session begins.

For short tasks, this delay is invisible. If a task takes 10 tool calls, the overhead of completing a doomed run is small. For long-horizon work, where a task might span hundreds of tool calls over an extended session, this is a significant problem. An agent that decides early on a wrong approach and sticks with it to the end wastes most of its budget. The failure signal arrives too late to change anything about that particular run.

Single-agent self-correction, where the model tries to catch its own errors mid-execution, runs into a different wall. It collapses the roles of executor and assessor into one context. The model that generated the flawed output is being asked to judge whether its output was flawed. This frequently fails, because the model's assessment is subject to the same biases and limitations that produced the error in the first place.

Splitting the roles between a supervisor subagent and a worker subagent is a cleaner design. The supervisor is not contaminated by the worker's reasoning trace and can assess the run independently. But the existing implementations of this split have a gap: the supervisor can watch. It cannot intervene. If the worker is clearly heading toward failure, the supervisor can note this observation, but the worker keeps running. The supervisor's signal reaches the next run, not the current one.

The system runs two frozen backbone models in a supervisor-worker configuration. The worker executes the task. The supervisor watches the worker's output in real time and can do something about what it sees. This is the first mechanism: live steering. When the supervisor detects that the worker is on a trajectory that is unlikely to succeed, it can interrupt, issue a correction, or abort the worker entirely. This stops budget from flowing into a session that will not complete successfully.

The second mechanism is live self-evolution. When the supervisor identifies a procedure that worked, or a failure mode that caused a detour, it distills that observation into a skill or memory entry. The crucial difference from post-session self-improvement is timing. These skills and memory entries become available to the same run that generated them. The worker does not need to wait for the next session to benefit from what the supervisor learned. A lesson from step 40 can change the approach at step 45.

The two mechanisms reinforce each other. The supervisor's ability to abort doomed trajectories means less budget is wasted on runs that will not teach anything useful. The in-session skill and memory updates mean successful approaches get reinforced within the current task, not just carried forward to future tasks. The loop is closed inside one execution window rather than across session boundaries.

The experiments ran three benchmarks with two backbone models each: GLM-5.1 and Kimi-K2.6. Both backbones were held frozen throughout. The PILOT harness sits entirely in the scaffolding layer above the model weights, which means the approach is, in principle, portable to any backbone that can communicate through standard interfaces.

PILOT placed first in five of six benchmark configurations across the two backbone models. The sixth configuration was not a loss; the results there were comparable to the best counterpart, not below it. Across three benchmarks and two models, the harness was consistently near the top of the field or at it.

The efficiency numbers deserve a closer look. Output tokens falling 43-47% while task success per token roughly doubles is not a typical tradeoff curve. Most techniques that cut tokens accept some quality reduction in exchange. Here the abortion mechanism is doing work that produces a different kind of saving: tokens that would have gone into completing a doomed run are simply not spent. The supervisor identifies a trajectory heading toward failure and stops the clock before the bill accumulates. The outcome is that the token budget concentrates on runs that succeed rather than spreading evenly across successes and failures.

The 14.6 and 12.4 point gains with GLM-5.1 and Kimi-K2.6 respectively in the self-improvement configuration show that the in-session skill and memory updates are contributing beyond what live steering alone would provide. The gains are measured against the same backbones run without the live self-evolution component, so the attribution is reasonably clean.