Where the Tokens Go

Ask a finance team why the AI bill is high and the answer is usually a rate: the model is expensive, the vendor should discount, a cheaper provider exists. The rate is real, and worth negotiating. But a bill is rate times quantity, and in most production deployments the quantity is carrying dollars — tokens that were billed, influenced nothing, and would never survive an audit if anyone thought to run one. They fall into three structural categories.

1 · Resent context

A conversational system conventionally re-transmits its entire history on every turn: the system prompt, the reference documents, every prior exchange. Turn fifty of an agent session ships the same background it shipped on turn one — and is billed for it, again. Multiply by every session, every user, every day, and the largest line on the invoice is frequently the same bytes purchased over and over. Nothing about the model requires this; it is a default, not a law. Deduplicating and pruning the redundant spans reduces the quantity billed per request while the work product stays the work product.

2 · Unread retrieval

Retrieval pipelines over-fetch by design: pull ten documents so the model definitely has the two it needs. Prudent engineering — and eight documents of billed input that influenced nothing. The over-fetch was a safety margin; the invoice records it as consumption. Trimming retrieval to what is actually drawn upon is pure recovered spend, and it is invisible unless someone is counting at the span level.

3 · Billed retries

Calls fail. Timeouts, malformed outputs, rate limits — and the conventional remedy is to try again, which means paying again. The retry is operationally correct and financially double-billed. One detail worth a CFO’s attention: aggressive cost-cutting through cheaper models can raise this category, because cheaper models fail more. Whether that trade actually paid is an empirical question — and answering it requires a count somebody other than the vendor signed.

Why cheap routing doesn’t fix this

The fashionable answer to AI cost is routing: send each task to the cheapest capable model. Routing is genuinely useful — and it acts on the rate. All three categories above are quantity phenomena. A router charges you a better price for the same redundant context, the same unread documents, the same second attempt. The two levers are independent, which is the good news: they multiply. A quarter fewer tokens on meaningfully cheaper routes compounds into savings neither lever reaches alone.

The part that makes it real

None of the above is exotic; caching and compression tools exist. What has been missing is the part that makes a savings claim bankable: proof. Every span TokenMark™ withholds is recorded — as a one-way digest, never as content — in a pruning ledger whose closure invariant guarantees that retained plus withheld equals the original manifest. Nothing silently disappears. The saved quantity is computed against a baseline hash-pinned before deployment, countersigned by an attester that cannot generate traffic, and recomputable by your auditor, offline, without trusting us.

And the incentive is aligned by construction: our fee is a share of attester-signed savings. We are not paid to sell you efficiency. We are paid when the sealed ledger says the efficiency occurred — and it says so in a record you can check. We do not publish savings percentages here, and will not until they are attested pilot results; the waste categories are structural, and the pilot seals your actual number.