- A total token count hides the shape of the bill. In a typical multi-step business task on raw tools, roughly nine tokens in ten are spent on things that are not the answer: tool schemas, raw rows the model has to read, and retries.
- The same question against prepared context is two small model calls: one to classify the request, one to write the explanation. Names are resolved and numbers computed by code, so the model never reads a row.
- You can check this on your own agents in an afternoon. Log tokens and wall time per call, split by model, tool and waiting, repeat the task twenty times, and report the spread.
When we say an agent run costs a fraction of a cent, or that a run on raw platform APIs can burn a hundred times more, the number is less interesting than where it comes from. So we traced one ordinary question two ways and wrote down every call. The figures below are a worked example built from the shape we see in traces, with the assumptions stated. Treat them as a map of the bill, not a benchmark.
The question
“Which of our fourteen accounts need attention this morning, and why?” To answer it, an agent needs the accounts, three numbers per account from three systems (a CRM, a store and an ad platform), the rule that defines “needs attention”, and enough history to say what changed. It is a Monday question. Every team has a version of it.
Path one: the agent on raw tools
Connect a capable model to the three systems through their APIs or their MCP servers, give it the question, and let it work. This is how most first agents are built, and it does produce an answer. Here is where the tokens and seconds go on the way there.
| Step | What the model has to read | Tokens in | Seconds |
|---|---|---|---|
| Load the tools | Every tool the three servers expose, with its schema. Around ninety tools, most of them never used for this question. | ~30,000 | 0 |
| List the accounts | One call, fourteen rows with every field the API returns, not the three we need. | ~4,000 | 2 |
| Pull the numbers | Fourteen accounts, three sources: forty-two calls, each returning daily rows for the window. The model reads all of them. | ~70,000 | 45 |
| Work out the definitions | What counts as revenue here, which attribution window the ad platform used, whether the CRM stage names match. Reasoned out in the prompt, differently each run. | ~8,000 | 6 |
| Retries and re-asks | Two rate-limit errors, one schema it misread, one account it forgot and went back for. | ~25,000 | 20 |
| Reason and answer | Compare, rank, write the explanation. | ~12,000 in, ~800 out | 12 |
| Total | 47 tool calls, 8 model turns | ~150,000 | ~85 |
Assumptions behind those numbers: a frontier-class model, a thirty-day window at daily grain, standard API page sizes, and no caching. Change them and the totals move. The shape does not. Three things dominate, and none of them is the answer.
- Tool definitions, about a fifth. They are loaded on every turn whether or not they are used. The more systems you connect, the bigger this fixed cost gets, before the model has read a single row.
- Raw rows, about half. The model reads data the way a person would read a spreadsheet printed on paper: every row, every column, to find the three numbers it needs. Then it does the arithmetic in its head.
- Retries and re-asks, about a sixth. Rate limits, a misread schema, a forgotten account. Each one replays a slice of the context. Retries are also where variance comes from: the same question costs a different amount each morning.
The reasoning and the written answer, the part you actually wanted, is under a tenth of the tokens. And the eighty-five seconds are mostly waiting: on forty-seven sequential calls, on retries, and on a large model reading seventy thousand tokens of rows.
Why it compounds
A Monday question is cheap if you ask it once. Agents do not ask once. A monitor that checks fourteen accounts every half hour runs the whole trace forty-eight times a day, and every run rebuilds the same understanding from scratch: the same tool list, the same rows, the same definitions reasoned out again. Nothing carries over. Add a second agent for a second team and it starts from zero too. The bill scales with questions times agents times runs, and the latency means the half-hourly monitor sometimes does not finish inside its half hour.
Path two: the same question on prepared context
Now run the same question where the understanding is built once and kept current: accounts, metrics and relationships defined in a catalog, state synced on a schedule, and “needs attention” written down as a rule rather than reasoned out each time.
| Step | What the model has to read | Tokens in | Seconds |
|---|---|---|---|
| Classify | The question and a short description of what kinds of task exist. No tool list. | ~400 | 1 |
| Resolve, compute, apply the rule | Nothing. The catalog maps names, the query runs in code, the rule is evaluated in code. | 0 | 1 |
| Explain | The computed result: three accounts, the numbers, what changed, the rule that fired. | ~1,200 in, ~500 out | 1 |
| Total | 2 model calls, 0 retries | ~2,100 | ~3 |
Same question, same answer, two orders of magnitude fewer tokens and most of a minute saved, in this shape. Your own ratio depends on how many systems, rows and retries your raw path carries. The mechanism is what transfers.
What changed, item by item
- The tool list left the prompt. Actions are bound by code, so the model never reads ninety schemas to pick one.
- Rows never reach the model. Numbers are computed where the data lives and only the result is shown. The model reads a summary, not a spreadsheet.
- Definitions are fixed once. Revenue, the attribution window and the stage names mean one thing, defined in the catalog. Nothing is reasoned out per run, so nothing drifts.
- Retries disappeared. A request that cannot be resolved fails before any generation is paid for, with a reason. Rate limits are handled by the sync, not by the agent mid-task.
- A small model is enough. Classifying a request and writing an explanation from a computed result are small jobs. The heavy work moved to code, so the model can be fast and cheap.
Where prepared context does not help
It is fair to say where this stops. A question with no catalog entry, such as a genuinely new metric, has to be defined before it can be cheap. Questions over documents rather than records still need retrieval, and the model still has to read what it retrieves. One-off exploration, where you do not know what you are looking for yet, is often better done the expensive way once. And anything that changes the business still needs a live read of the specific records before it acts; that read is small, but it is not free. Prepared context pays off on the questions you ask repeatedly, which, for most teams, is most of them.
Measure your own agents
Do not take our shape on trust. The useful test is small.
- Log per call, not per run. For every model call and tool call: input tokens, output tokens, wall time, and whether it was a retry. A single total hides everything above.
- Split the clock three ways. Model time, tool time, waiting time. Latency complaints are usually about the third.
- Repeat it twenty times. Same model and version, same prompt, same data, same permissions. Report the median and the worst case. Variance is the retry bucket showing itself.
- Check the answer every time. Cheaper is only interesting if the three accounts are the same three accounts, with the same reasons.