The loop that
checks itself.
First surfaced in the Tandemly Briefing for 2026-07-12.
Most self-improving systems have no built-in test for whether they actually improved. They update, and they hope. SEA changes that: a frozen base model accumulates small adaptations from verifier feedback and then runs a statistical gate before any change takes effect. The gate can answer at any point during evaluation. If the numbers do not support the modification, the modification does not land.
Self-improvement that
can't verify itself is just drift.
Standard approaches to improving a deployed agent are either expensive or ungated. Neither option is appropriate for a production system where every failed task has a real cost.
Retraining or fine-tuning a model to improve one capability is expensive and slow. Self-play and reinforcement approaches are faster, but they introduce a reliability problem: how do you know a model that completed a few extra rollouts actually got better, rather than just got lucky? Sequential A/B tests can answer this, but only after a predetermined number of trials. If the signal is clear earlier, you have wasted evaluation time. If the signal is not yet clear at your predetermined horizon, you have committed to an answer anyway.
Neither outcome is good in a production agentic system. Early stopping at significance would let you ship improvements faster or reject failures sooner. Committing at a fixed horizon means you accept the false-positive rate at that one point in time rather than across the full evaluation trajectory.
There is also a deeper problem with existing self-improvement approaches: they do not separate the improvement target from the base model. Every update changes the same parameters that handle tool calls, state tracking, and structured output. Small improvements in one capability can silently degrade another. And if something goes wrong, you cannot revert the capability without reverting everything else.
Can a self-evolving agent accumulate improvements from its own verifier signals, gate each modification behind a statistically valid test that controls the false-positive rate at any stopping point, and do all of this without touching the base model that everything else depends on?
Freeze the core.
Adapt at the edges.
SEA separates the improvement target from the production model. The base model is frozen. A small learned steering adapter receives all modification. The adapter accumulates signal from five verifier loops that evaluate the agent's outputs during normal task operation.
The frozen-base architecture is the first design choice. The agent's base weights never change. A lightweight steering adapter is trained on top, accepting modification while the base model stays stable. Reverting an adaptation is inexpensive. Re-training a base model is not. This also means the base model's general capabilities are protected from the gradient noise of narrow-task improvement.
The adapter accumulates signal from five verifier loops that operate during task execution. Each loop targets a different structural weakness in agentic reasoning.
Before any modification lands, the system runs an anytime-valid statistical test against the candidate change. Unlike a fixed-horizon t-test, anytime-valid tests can produce a valid conclusion at any point in an evaluation: if significance is reached at step 20 of a 100-step evaluation, the test can stop there. If it is not reached by step 100, that is also a valid result. The statistical guarantee covers the false-positive rate across all possible stopping times, not just at one predetermined endpoint.
Modifications that clear the gate receive an auditable certificate: a record of the evaluation trajectory, the test statistic, the stopping point, and the conclusion. The certificate is the evidence trail for any downstream review.
Anytime-valid tests use e-values or sequential likelihood ratio tests rather than classical p-values. The false-positive rate is controlled regardless of when you look at the data, not just at a pre-specified endpoint. This means you can monitor continuously and stop as soon as you have enough evidence, without inflating Type I error. Classical testing controls error only at one point; anytime-valid testing controls it everywhere.
Both baselines grew.
GPT grew more.
Evaluated on a 52-instance subset of SWE-bench Verified, a benchmark for coding agents that requires resolving real GitHub issues. Both backbone models improved under SEA, with GPT showing the larger absolute gain.
The baseline conditions represent the same models without the steering adapter and without the verifier loops. Both gains are positive and direction-consistent across backbones, which suggests the benefit comes from the SEA structure rather than from properties of a specific model. Each additional resolved instance on SWE-bench Verified represents a genuinely novel code change that passes an independently written test suite, so the gains are not artifacts of evaluation design.
The gains are also sub-additive: verifier signals do not compound without bound. The steering adapter has a capacity limit, and the gate exists precisely to prevent modifications that look like improvements but have not cleared a sufficient evidentiary bar.
This is a single-run evaluation on a 52-instance subset. Run-to-run variance has not been characterized. The steering adapter architecture is described but has not been independently replicated. The anytime-valid gate is the paper's central statistical claim; the formalism is sound, but the practical benefit of early stopping over a well-designed fixed-horizon test in this specific setting has not been quantified separately. These limitations do not undercut the direction of the results, but they do mean the absolute numbers should be treated as directional rather than definitive until independently confirmed.
What changes,
and what stays fixed.
SEA contributes three durable patterns: separate the improvement target from the base model, gate every modification with a statistically valid test, and verify at the sub-step level rather than only at task completion.
How to apply
these patterns.
You do not need the full SEA system to benefit from its core ideas. Each of the three main patterns can be piloted independently.