The decomposer is
the bottleneck.
Researchers at East China Normal University factorized a hierarchical search agent into three separated roles, ran controlled capacity sweeps across each one, and found that model size barely matters where most builders put the budget. Scaling the delegation layer lifts accuracy by roughly 11 points. Scaling the execution layer moves it less than 3. The sub-agents doing retrieval can be kept small.
First surfaced in Tandemly Briefing — 2026-08-07.
Bigger is better everywhere
isn't a strategy.
Multi-agent search systems are expensive. Most builders respond by scaling uniformly, using the same tier of model throughout. That turns out to be the wrong question.
When an agent tackles a complex question, it rarely handles it in one pass. The more robust pattern is hierarchical: a coordinator breaks the question into sub-problems, hands each sub-problem to a retrieval agent, and a writer pulls the answers together into a final response. This architecture is common. The question of where to put the compute budget inside it is not.
The field's default answer is something like: bigger is better everywhere. Use a frontier model for the coordinator and frontier models for the sub-agents, and assume the bottleneck is overall capability. This produces high-quality systems that are also expensive, and it tends to discourage experimentation because every component is already maxed out.
What the field hasn't had is a clean separation of roles followed by systematic variation of model size at each role. Without that, builders cannot know whether they're over-spending on execution and under-spending on decomposition, or vice versa. They're flying blind on where the dollar actually buys accuracy.
In a hierarchical search agent with cleanly separated roles, where does scaling model capacity actually move the needle on accuracy? Is the bottleneck in how the query gets broken down, in how sub-agents retrieve information, or somewhere else entirely?
Three roles, each varied
independently.
The paper builds a hierarchical multi-agent search system with three named roles, then holds two fixed while scaling the third. The experimental design separates the question of where capacity matters from the question of how much capacity you have in total.
The system architecture has three separated components. A delegation agent, sometimes called a Shepherd, receives the original query and decomposes it into sub-questions. It maintains global context across the search process and directs the population of retrieval agents. A set of execution sub-agents handle search and retrieval: each takes one sub-question, looks for relevant information, and returns it. A fixed answer writer synthesizes everything into a final response. The writer does not vary across experiments.
The core move is capacity isolation. To measure the contribution of each role, the researchers vary the model tier at one role while holding the others constant, then measure exact-match accuracy across five multi-hop question-answering benchmarks. Multi-hop tasks are a good test because they require genuine decomposition: the right sub-questions depend on understanding how the problem connects across multiple retrieval steps.
Alongside the capacity sweep, the researchers train a compact executor through quality-filtered trajectory distillation. The idea is straightforward: collect successful search trajectories from a frontier sub-agent, filter them for quality, and use them to train a much smaller 1.7-billion-parameter model. If the distilled model's search behavior is close enough to the frontier model's, it can replace the expensive sub-agents at a fraction of the cost.
Multi-hop question answering requires the kind of genuine decomposition hierarchical agents are built for. Answering "Who directed the film that won the same award as the novel the author of X also wrote?" requires breaking the problem into connected sub-queries, not retrieving a single document. Accuracy on this class of task is a fair measure of whether decomposition quality actually matters.
Eleven points versus
two and a half.
The capacity sweep produces a clean asymmetry. The two roles do not respond to scaling in the same way, and the gap is large enough to have real budget implications.
A 1.7-billion-parameter executor trained via quality-filtered trajectory distillation matches a frontier sub-agent while consuming 37% fewer sub-agent tokens across the five benchmarks. The frontier model's successful search trajectories, filtered for quality before distillation, transfer enough of the retrieval behavior to make the smaller model competitive. This matters because execution sub-agents are the high-volume component: they fire once per sub-question, so cost compounds quickly.
The evaluation is on multi-hop question-answering benchmarks, which are a natural fit for hierarchical search. Domains where the decomposition step is simpler, where sub-questions are nearly independent, or where execution involves more complex reasoning than retrieval might show a different capacity profile. The asymmetry is a finding about hierarchical search agents specifically, not a universal law about all multi-agent systems.
Put the strong model
where it earns its keep.
This paper is most useful for teams building hierarchical search agents who have been sizing components intuitively rather than empirically. The findings give a concrete starting point for where to put the budget.
Where to go
from here.
Concrete next steps for teams who want to apply the capacity-asymmetry framing to their own systems.