The metric you optimize
is not your bill.
Researchers at PointFive ran nearly 2,900 metered Claude Code sessions and found something the field had mostly assumed away: reducing tool-output tokens by 38.4% raised the provider-billed cost by 6.8%. The number everyone tracks turns out to be an incomplete, sometimes misleading, proxy for the number that actually matters.
First surfaced in Tandemly Briefing — 2026-07-13.
Everyone counts tokens.
Nobody measures the bill.
The AI engineering community has built a large tooling ecosystem around reducing token counts: command-output compressors, retrieval rankers, payload-optimizing proxies. The assumption buried in all of it is that fewer tokens means lower cost.
That assumption is reasonable for simple, stateless API calls where the price per token is constant and the total cost is just tokens multiplied by rate. But coding agents are not simple or stateless. They send the same instructions, tool schemas, and accumulated context over and over, turn after turn, across a session that might run for dozens of steps.
Modern inference providers have addressed this with prompt caching: portions of context that are identical across consecutive calls get stored and re-sent at sharply discounted rates. This changes the economics significantly. A long system prompt that never changes might cost 10% of its nominal price per call. A block of output that changes every turn has no cached version and is billed at full rate.
The implication is that the proportion of a session's cost attributable to different types of tokens varies wildly, and interventions that target one type can have unpredictable effects on others. Reduce the output tokens but force the model to re-read more files to compensate, and the net effect on the invoice might be negative.
Before the PointFive study, almost no published research had measured these effects with real billing data rather than estimated token counts. Most efficiency evaluations tracked tokens removed. The actual invoice was a black box.
When does reducing retrieved context or tool output actually reduce the provider-billed cost of completing a coding task successfully? Not: does it reduce token count? The bill, not the estimate.
2,848 real runs,
three real models.
The study used real billing data across a pre-specified, hash-frozen campaign: the tasks, repositories, and model assignments were locked before any results came in, preventing the kind of ad-hoc selection that often inflates efficiency claims.
The core dataset covered 2,848 analyzed Claude Code sessions (from 2,908 total, with 60 excluded on quality grounds), spread across 103 tasks, seven open-source repositories, and three models. The broader measurement program included roughly 5,500 billed executions. Weinberger and Hozez are engineers at PointFive, a company focused on cloud and AI infrastructure cost analysis, and the study grew out of their applied work measuring agent spend in production environments.
Three types of context-reduction interventions were tested against a baseline. The first two were hook-based: a command-output compressor that truncated and summarized tool responses before they reached the model, and a retrieval ranker that filtered which context blocks were passed into each turn. The third was an API-boundary proxy that operated between the agent and the provider, restructuring payloads to maximize cache efficiency.
For each session, the researchers reconstructed the four-component cost breakdown: new input tokens (billed at full rate), cached input tokens (billed at a discount), output tokens, and any fixed overhead. They then compared this reconstructed cost against the actual metered provider invoice to validate the accounting.
API billing for agents decomposes into four parts: new input tokens (sent fresh each turn, full price), cached input tokens (context the provider stored from prior turns, billed at 10-90% off depending on the provider), output tokens (the model's generated response), and fixed overhead. The proportions of these four buckets determine whether any given compression strategy helps, hurts, or washes out on the invoice.
Fewer tokens,
larger bill.
The headline result was counterintuitive enough that the researchers ran extensive negative controls to rule out measurement error. The result held across models and repositories.
Across the measured workloads, cached input tokens accounted for roughly 87% of the reconstructed four-component cost, and approximately 80% of the actual billed amount. The instructions, tool schemas, and accumulated system prompt that the model sees on every turn are sent repeatedly at cache prices. They are not what the compressor targeted. They are what the bill is about.
A compression strategy that shrinks tool-output tokens is optimizing one component that accounts for a small share of the total. The dominant cost sits elsewhere, largely unaffected by output-side interventions.
When a compressor removes context the agent later needs, the agent compensates. It searches again. It re-reads files. It issues diagnostic calls. Each of those extra steps sends new tokens to the API, some of which miss the cache, generating new uncached input charges. The token count in the compressed outputs goes down; the token count in the subsequent recovery steps goes up. The intervention that looked like a cost reduction produced a cost transfer: from one bucket to another, with a net increase on the invoice.
This is the mechanism behind the 38.4% reduction producing a 6.8% increase. The compressor was effective at what it compressed. It was not evaluated against what its compression caused downstream.
The study used Claude Code specifically, across Anthropic's pricing model. Cache discount rates vary by provider, and the finding that cache dominates cost depends on the discount being substantial. The 38.4% and 6.8% figures are for one particular compressor configuration on a specific task distribution. The directional conclusion (token count is an unreliable proxy; measure billed cost directly) is robust, but the exact magnitude of any divergence will depend on the provider, model, and workload.
What this changes
for builders.
This paper is not an argument against compression. It is an argument for measuring the right thing before deploying any efficiency intervention. The gap between token count and billed cost is real and large enough to reverse the expected direction of an optimization.
Where to go
from here.
If you want to act on this research or go deeper into the underlying dynamics.