Multi-Model Routing · Mechanism Design

Let the models bid.
Route to the winner.

Static routers assign reasoning steps to models based on coarse function labels. Researchers at the University of Birmingham asked whether you could replace that assignment with a market: models bid for each step based on honest self-assessment, and the incentive structure makes misrepresentation a losing strategy.

Core concept
Auction-based allocation: candidate models submit calibrated bids for each reasoning step. The mechanism is designed so that expressing true competence, not inflating or deflating it, is each model's best move.

First surfaced in Tandemly Briefing — 2026-07-23.

scroll to explore

Every step in an agent pipeline
is a routing decision.

Multi-model agent stacks quietly assume that function labels predict competence. They usually don't.

Modern LLM agent frameworks let you string together multiple models and tools. One model handles retrieval. Another handles reasoning. A third handles code generation. The routing logic that decides which model handles which step is often a lookup table, a similarity score against function descriptions, or a hardcoded cascade of fallbacks.

The problem is that competence doesn't map cleanly onto labels. A model that performs well on arithmetic-flavored MMLU questions might fall apart on symbolic logic. A general-purpose model might outperform a "code specialist" on a specific kind of debugging task. Function labels are approximations. The gap between a label and actual per-step accuracy is real, and it compounds over a multi-step reasoning chain.

A second issue sits underneath the first: models tend to be poorly calibrated about their own abilities. When a model expresses confidence in its answer, that confidence is often not a reliable signal of correctness. Routing mechanisms that rely on self-reported confidence as a quality signal are building on an unreliable foundation. The numbers come back high, but the routing decisions are still arbitrary.

The question this paper asks

If we treat each reasoning step as a task to be awarded rather than a slot to be filled, can we design an allocation mechanism where a model's rational strategy is to accurately express how competent it actually is? And if we can, does routing to the winner improve results over static baselines?

A market for reasoning,
with calibrated bids.

Agora treats each reasoning step as a lot. Candidate models submit bids. The framework rectifies those bids before the auction runs, then awards the step to the highest honest bidder.

The core analogy is auction theory. In a standard auction, bidders name a price. The item goes to the highest bidder. But there's a design question: what rules make it rational for each bidder to reveal their true valuation, rather than strategically over- or under-bidding? The answer in economics is incentive-compatible mechanism design. Vickrey's second-price auction is the canonical example: you pay not what you bid, but what the second-highest bidder offered, which removes the incentive to inflate your bid.

Agora adapts this logic to model allocation. At each step of a reasoning chain, candidate models are asked to express how capable they are at the task. These self-assessments form the bids. But raw self-assessments are unreliable: LLMs are routinely overconfident. So Agora applies a rectification step before the auction runs, calibrating each model's raw self-confidence against a reference distribution of its historical accuracy on similar task types. The corrected value is the effective bid.

The auction mechanism is then designed so that after rectification, a model's best strategy is to express its true competence. Submitting an inflated bid leads to winning steps the model can't handle well, which drives down overall performance. Deflating leads to losing steps the model could have handled. The equilibrium is honest reporting. The framework awards each step to the model with the highest rectified bid and composes the results into a coherent output.

1
Decompose the reasoning task into steps
The agent pipeline breaks a complex query into sub-tasks: retrieval, multi-hop inference, symbolic reasoning, code generation, verification. Each step is a distinct lot in the auction.
2
Candidate models submit competence bids
For each step, each candidate model in the pool expresses its estimated competence. These raw self-assessments reflect the model's own internal confidence signal for the given task type.
3
Rectify the bids against calibration signals
Agora corrects each model's raw self-confidence using a reference distribution of its past accuracy on similar tasks. This converts an often-overconfident estimate into a calibrated bid.
4
Award each step to the highest rectified bidder
The incentive-compatible auction runs on rectified bids. The model with the highest honest estimate of its competence handles that step. Results compose into the final output.
How this differs from routing and cascades

A static router applies a fixed rule: look at the task label, pick the model associated with that label. A cascade tries the cheapest model first and escalates on failure. Both treat allocation as a lookup or a retry. Agora treats it as a market: allocation is determined by expressed competence, not prior assignment, and the mechanism is designed to make honest expression the dominant strategy rather than something the framework has to enforce.

Outperforms baselines
on five benchmarks.

Agora was evaluated against matched single-model, routing, and cascade baselines using the same candidate pools. The auction mechanism improved results across the board.

MMLU-Pro accuracy
71.9%
vs 68.1% best single-backend
MuSiQue-Ans EM
43.0
54.3 F1 on multi-hop QA
Baselines beaten
5
benchmarks: single, routing, cascade
Prior assumption
Pick the right model for the task type, then assign every instance of that type to it. Coarse label matching is good enough. Models in the same category are interchangeable. Routing overhead isn't worth the complexity.
What the paper finds
Per-step auction allocation, with calibrated bids, outperforms static routing and cascades under comparable candidate pools. The gap between label-based routing and competence-based allocation is measurable and consistent across benchmarks.
Finding 1: Rectification is the critical ingredient

The improvement over baseline routing comes not just from holding an auction, but from rectifying self-confidence before the auction runs. Raw self-assessments are unreliable as routing signals: models are systematically overconfident on task types where they underperform. The rectification step corrects for this bias. An auction run on uncorrected bids performs closer to naive routing; the calibration step is what separates the mechanism from a slightly fancier function call.

Finding 2: Cost-quality tradeoff is controllable

Because bids reflect competence rather than cost, the auction naturally routes high-stakes steps toward the strongest available model and lower-stakes steps toward cheaper alternatives. The framework exposes this as a tunable parameter: tighter constraints on acceptable bid gaps push more steps to cheaper models at some accuracy cost; wider constraints let quality float higher at higher cost. This tradeoff is explicit and measurable rather than implicit and fixed, as it typically is in cascade setups.

Finding 3: Gains are consistent across task types

MMLU-Pro covers broad knowledge across 57 academic disciplines. MuSiQue-Ans requires multi-hop reasoning across disconnected facts, a task with structural complexity that label-based routing handles poorly. SciCode involves sub-problem decomposition in scientific coding contexts. The improvements hold across all three, which suggests the mechanism is capturing something general about per-step competence rather than a benchmark-specific artifact.

Scope and limitations

The evaluation is based on a defined candidate pool for each experiment; results reflect the quality of that pool as much as the allocation mechanism. The rectification approach relies on historical calibration data for each candidate model, which may not be available or stable in production settings where model updates occur frequently. The paper establishes that the mechanism works; the operational question of how to maintain calibration signals in a live deployment is not fully addressed.

What this means
for agent builders.

The routing layer in most multi-model agent stacks is more important and more improvable than it looks. Agora names both the failure mode (coarse matching ignores per-step competence) and the mechanism (calibrated auctions) that addresses it.

1
For agent infrastructure builders
If your current routing uses function labels or static assignments, the gap to calibrated allocation is measurable. Before investing in bigger models, measure whether your routing is actually sending each step to the best available model for that step. Agora's auction framing offers a principled way to close that gap without changing the models in your pool.
2
Treat self-confidence as a noisy signal, not a fact
LLMs overstate their confidence on tasks they handle poorly. If you're using model confidence as a routing or termination signal anywhere in your stack, calibration against historical accuracy is not optional overhead: it is the difference between a signal and noise. The rectification step in Agora is the same insight, formalized as a mechanism.
3
For cost-quality optimization
Cascades give you a binary choice: cheap model first, expensive model if it fails. Auction-based allocation gives you a continuous tradeoff: tighten the bid-gap threshold and you push more steps to cheaper models; loosen it and quality rises at higher cost. This is the right framing for production workloads where cost and quality have to be balanced at the step level, not just the task level.
4
For teams working with heterogeneous model pools
Many production deployments mix frontier models with smaller fine-tuned alternatives. The differences in per-step capability are real but unevenly distributed: a small model may outperform a large one on specific domains where it was fine-tuned. Label-based routing misses this. Per-step auctions let smaller models win the steps they're actually better at, improving both cost and accuracy in those cases.

Where to go
from here.

If you want to go deeper on mechanism-driven agent routing.

1
Read the paper
Zhou, K., Leonardis, A., & Feng, Y. (2026). Agora: Enhancing LLM Agent Reasoning Via Auction-Based Task Allocation. University of Birmingham. arXiv:2607.09600.
2
Audit your current routing logic
Before adopting a new mechanism, measure your current routing quality: for a sample of past reasoning chains, did each step go to the model that would have performed best on it? Most teams have never measured this. The gap is the baseline improvement Agora competes against.
3
Build a calibration dataset for your model pool
Agora's rectification step requires historical accuracy data per model per task type. Start collecting it now. Even a lightweight accuracy tracker across your current multi-model calls, tagged by step type, will give you the calibration signal the mechanism needs.
4
Read on agent routing and skill compatibility
See also: "Skill Is Not Document" (Wang, Wen et al., Tencent, arXiv:2606.03565) on why set-level compatibility between retrieved skills matters as much as per-item relevance. The two papers address adjacent layers of the same routing problem: Agora handles which model handles a step; the Tencent paper handles which skills can collaborate on a task.
5
Consider the mechanism design literature
Agora borrows from incentive-compatible mechanism design. Vickrey's 1961 paper "Counterspeculation, Auctions, and Competitive Sealed Tenders" is the foundation. For a modern treatment: Mechanism Design: A Linear Programming Approach (Vohra, 2011) covers the theory behind why certain auction formats produce honest revelation.