First surfaced in Tandemly Briefing — 2026-08-02.
The retriever ranks.
Something else decides.
A tool retriever tells your agent which tools might be relevant. It does not tell the agent how many of them to actually load. The paper shows that ignoring cost when you pick the cutoff is not just wasteful. It is provably wrong.
Ranking is one decision.
Cutoff is another.
Every tool-using agent runs the same two-step pipeline: a retriever ranks candidate tools by relevance, and a cutoff decides how many of those ranked tools to expose in the prompt. The retrieval literature has spent years improving the first step. The second step is usually a fixed number.
The default cutoff is either top-k (always take the top 5, or 10, or 20) or a similarity threshold (take everything scoring above 0.6). Both are score-only rules. They use the retriever's ordering and its scores, but they do not know anything about what each tool costs to include.
Cost varies more than most stacks admit. Two tools with equally high relevance scores can differ by ten times in token count. They can differ in latency, in the permissions they require, or in blast radius when they misfire. A shell tool and a calculator both look like "tools" to a top-k selector. They are not the same thing to load into a prompt.
The intuitive fix is to divide relevance by cost and threshold on the ratio. That helps when costs are similar. It stops helping as soon as costs are spread out enough that the marginal payoff of the next tool depends nonlinearly on the tools already chosen. The paper's central formal result is that no rule that decides purely from scores (with or without a cost divisor) can be Bayes-optimal once tool costs are heterogeneous, no matter how well the threshold is tuned.
Tool acquisition is not a ranking problem. It is a stopping problem sitting on top of a ranking. You walk the top-scored tools in order and, at each position, decide whether the marginal payoff of the next tool is worth its marginal cost. The ranking is an input to that decision. It is not the decision.
Train on the gap,
not on the score.
CAM-DF (cost-aware marginal decision-focused stopping) trains a small model to answer one question at each position in the ranked prefix: is the payoff of the next tool worth its cost?
The training data comes from offline logs of prior agent runs. For every ranked prefix in the log, the authors compute two numbers. The first is what the agent scored when it stopped at that prefix. The second is what it would have scored if it had taken the best continuation available in the log. The label is the difference between them.
The sign of that difference labels the correct decision. Positive means the continuation was worth taking; negative means stopping was the right call. The magnitude is the payoff at stake. Training on this loss forces the model to be careful when errors are expensive and decisive when they are not, without needing to know upfront which is which.
The authors prove this training objective is Bayes-aligned with the stopping decision. In plain terms, an optimal predictor trained on this loss produces optimal stopping behavior. They also prove the negative: no rule that consumes only the scores (a fixed-k, a threshold, a score-per-cost ratio) can match this under heterogeneous costs. The training target is the thing that makes the difference, not the model class.
Alongside the full model, the paper ships CAM-DF-lite: the same stopping objective learned over a small hand-designed feature set (score gradient across the prefix, cumulative cost so far, position in the ranking). It gives up a small amount of headline accuracy in exchange for a rule you can read, audit, and defend at a permission boundary. Useful when the cutoff sits between an agent and tools with real side effects, and someone needs to explain why one tool was loaded and another was not.
It does not fine-tune the LLM. It does not replace the retriever. It does not require a new tool-description format. The rule sits between the retriever's ranked output and the prompt-assembly step, consuming the scores the retriever already produces plus a per-tool cost estimate. The plumbing change is small; the design change is treating cutoff as a first-class trained component.
Fewer tools,
same task success.
Evaluation runs across 1,343 tasks in five tool-use domains, with τ-bench Retail as an additional live-execution setting. Baselines include fixed-k, similarity thresholds, and a tuned predict-then-threshold rule that already accounts for cost.
The 37% figure is the one to read carefully. It means the agent completes the same fraction of tasks while being exposed to less than two-thirds of the tools a full-ranking cutoff would load. That translates directly to smaller prompts, faster inference on the tool-selection step, fewer permission grants issued per query, and less surface area for a tool to be called incorrectly.
The head-to-head result is the one that pins down when the design matters. At uniform tool costs, a tuned score-per-cost threshold stays competitive with CAM-DF. As cost heterogeneity grows (measured by the paper's cost-spread parameter d), the score-only rule falls off. At d = 1.0, CAM-DF wins in all 20 tested permutations against a predict-then-threshold baseline that has been tuned specifically for the setting. The takeaway is not that CAM-DF is uniformly better. It is that CAM-DF's advantage tracks how varied the tool costs are, and most real production tool inventories are varied.
On τ-bench Retail, CAM-DF attains the highest payoff among deployable methods, with gains over predict-then-threshold across all five ranking sources tested and both cost regimes evaluated. The result generalizes across retriever choices, not just a single ranker the authors happened to build.
CAM-DF is trained on offline logs from prior agent runs, so its performance depends on those logs being representative of production traffic. The paper does not report robustness under distribution shift (a new tool added to the inventory, a query style not seen in training) and does not claim to replace the retriever, only to replace the score-only cutoff that consumes its output. The proofs also assume the cost of each tool is estimable in advance, which is fair for token cost but harder for latency or blast radius. The empirical results are strongest where the abstraction holds.
Where this changes
a production stack.
The finding is narrow and immediately actionable. If your agent has a tool retriever with a fixed-k or threshold cutoff sitting on top of it, and your tools cost meaningfully different amounts, the cutoff is the layer to look at before you touch the retriever again.
Practical steps
for teams shipping agents.
The plumbing change is small. The measurement change is the harder part, because it exposes how much fat a fixed-k cutoff has been hiding.