Why the tenth turn costs a hundred times what the first one did

An agent's first call costs a few cents. The tenth costs a multiple of that - and nobody switched models in between or asked a longer question. This surprises people regularly, because they expect arithmetic that doesn't apply here: ten steps, ten times the price of one step.

A language model has no memory

The reason sits right at the foundation: a language model's API is stateless. It remembers nothing between two calls. What an agent experiences as a "conversation" is a list of messages that is resent in full with every single call - system prompt, every previous question, every previous answer, every tool call and its result.

Baustein

Message History

After every answer we hang up - the next call starts from zero. Context only exists because we read the full transcript aloud at the start of every call.

The call from part 1 is stateless. To hold a conversation, the harness keeps a transcript and reads it out from the top on every call. That's the message history - and the reason "memory" means something entirely different for LLMs than you'd expect.

Every LLM call is stateless. For an agent to stay coherent across steps you must send the message history along - and you pay for it in tokens. Memory is therefore always a trade-off: enough context for coherence, little enough for cost and focus.

→ Zum Baustein

So turn 1 sends the system prompt and a question. Turn 2 sends the system prompt, the question, the answer and the next question. Turn 10 sends everything that has accumulated over nine rounds. The input side of the bill grows with every step - and you pay per token, every time, from scratch.

The total grows quadratically, not linearly

Suppose each turn adds roughly the same amount of text. Then

• turn 1 costs the size of one block,
• turn 2 costs the size of two blocks,
• turn 10 costs the size of ten blocks.

So the single tenth call isn't a hundred times more expensive than the first - it's about ten times. The hundredfold appears when you look at the total bill for a run: 1 + 2 + 3 + … + 10 is 55 blocks instead of the expected 10. At a hundred steps it's 5050 instead of 100 - a factor of 50 against the naive estimate. That total grows with the square of the step count. Which is why an agent that calls twenty tools doesn't feel twice as expensive as one that calls ten, but four times.

And with agents, the blocks are rarely the same size. The biggest chunk is usually not the model's replies but the tool results: a file that was read, a list of search hits, an API response body. A single read_file on an 800-line file puts more into the history than ten model answers - and that file then rides along in every subsequent call.

What this means in practice

Three things follow, and all three are design decisions, not settings:

Context is budget, not storage. Whatever enters the history isn't paid for once, but for the rest of the run. That's why it's cheaper to have a tool return a lean result than to have the model tidy up later with a summary - the summary only replaces the original in the history after the original has already been in there.

The history needs tending. That's exactly what techniques like summarising older sections, offloading long results to a store the agent reaches by reference, or simply truncating old turns are for. An agent without history management doesn't become unaffordable eventually - it hits the wall of the context window first.

Prompt caching changes the price, not the volume. Because the beginning of the message list is identical on every call, most providers offer a discount on exactly that repeated prefix - often on the order of 90 per cent while the cache is warm. That's worth a lot, and it's why you put stable parts (system prompt, tool definitions) at the front and never slot anything variable ahead of them. But: it lowers the price per token, not the token count. The context window fills up just as fast.

The second price: time

It doesn't just cost money. A transformer's attention layer compares every token with every other one - the compute per call grows more than proportionally with length. In operation you notice it as time-to-first-token: it climbs noticeably over a long run, even though the answers stay equally short. So an agent doesn't only get more expensive as it works, it also gets slower.

Once you've seen this, you read the bill differently. The question isn't "how many calls does my agent make" but "how big is it on the last one".