AI Economics Playbook · Part 1 of 2

The Economics of AI Tokens, Part 1: What You're Actually Paying For

What the token meter counts, why generating an answer costs differently from reading a prompt, and how the same information can be charged across multiple calls.

Kiran SusarlaOriginally published September 11, 2026 · Approximately 8 min read
Three dark stones suspended beneath a torn cobalt sheet of sky.

Executive summary

A token is a billing unit, not a direct measure of GPU work. Input, cached input and output can carry different rates. As conversations continue, earlier material may be included repeatedly, increasing total token usage. Prompt caching can change the cost of those repeated inputs.

I have been writing about the economics of AI for a while - the projects that refuse to die, the business cases full of hope, and what happens when AI moves from experimentation into the real world. Somewhere along the way, I realized it might be worth going back to one of the basics: tokens. It is one of those terms we use constantly in AI, often with an assumed understanding of what it actually means. So before I get back to the bigger economics, I wanted to spend two short articles on something more fundamental: what a token is, what we are actually paying for when we buy them, and why the bill can behave very differently from what we might intuitively expect.

A token is the meter

A token is essentially the metering unit. It is not a direct measurement of GPU work. Text is broken into tokens - roughly 75 English words per 100 tokens, although this varies by language and tokenizer [OpenAI - Understanding and counting tokens]. The basic transaction looks like this:

                              ONE MODEL CALL

            INPUT                                  OUTPUT
     ┌───────────────────────┐              ┌──────────────────┐
     │ System instructions   │              │ Reasoning*       │
     │ Your prompt           │ ── MODEL ──► │ Answer           │
     │ Conversation history  │              │ Tool calls       │
     │ Retrieved documents   │              │ etc.             │
     │ Tool results          │              │                  │
     └───────────────────────┘              └──────────────────┘
           INPUT TOKENS                           OUTPUT TOKENS
            billed at X                           billed at Y

    * depending on model

Now OpenAI, for example, formally distinguishes input, cached input, output and - in reasoning models - reasoning tokens. Reasoning tokens may not appear in the answer you see, but they can count toward output usage.

The generic bill is therefore:

Cost = (Input tokens × Input price) + (Cached input tokens × Cached price) + (Output tokens × Output price)

That is also how OpenAI describes its token-based billing [OpenAI - ChatGPT token-based rate card].

Why input and output cost differently

The surprising part is that input and output are generally not priced similarly. Output is usually considerably more expensive. There is a technical reason. When the model reads your input, it can process much of that input in parallel. This phase is commonly called the prefill.

When it generates an answer, however, generation is autoregressive:

Generate token 1
      ↓
use token 1 to generate token 2
      ↓
use tokens 1–2 to generate token 3
      ↓
use tokens 1–3 to generate token 4
      ↓
...

It cannot simply calculate all 2,000 output tokens simultaneously because token 1 affects token 2, token 2 affects token 3, and so forth. Consequently, output consumes expensive GPU capacity for longer and is much more latency-sensitive. That is why you often see something like:

Input:          $X / million tokens
Cached input:   perhaps $0.1X
Output:         $4X, $5X, $6X, etc.

For example, as of September 2026, OpenAI's GPT-5.6 Sol pricing illustrates exactly this relationship: $4 per million input tokens, $0.40 for cached input and $20 per million output tokens - a 5:1 output/input ratio [OpenAI - GPT-5.6 Sol model page].

Why earlier context can be billed again

But here is the part corresponding to what you were thinking about. Suppose you have this conversation:

Turn 1
You send:       10,000 tokens
I answer:        1,000 tokens

You might naturally think that the next question costs only whatever your new question contains. It often doesn't. On Turn 2, the model may effectively receive:

Previous 10,000 input
Previous  1,000 answer
New         500 question
------------------------
          11,500 input tokens

Then perhaps:
           1,500 new output tokens

And then Turn 3:

Everything above
+ previous response
+ new question
------------------
perhaps 13,500 input tokens

So the same earlier 10,000 tokens can participate in multiple model calls. That means that over a conversation you can get:

Original document = 10,000 tokens

Call 1: 10,000
Call 2: 10,000 again
Call 3: 10,000 again
Call 4: 10,000 again
Call 5: 10,000 again

Logical information supplied once
Potential token processing: ~50,000

This is probably the phenomenon you were imagining when you said input might be "double counted."

Token accounting: each model call includes input tokens, possible cached input, and output or reasoning tokens. Reusing the same context across multiple model calls can increase cumulative input usage; caching can reduce the rate charged for repeated context.View the illustration at full size
Editorial illustration of token metering, repeat context and prompt caching. The diagram is conceptual; exact billing depends on the provider and model.

It is not generally: "10,000 input tokens × 2 because the model has to process them."

It is closer to: "Those 10,000 tokens became input to five separate inference calls, so they were processed five times."

What caching changes

Prompt caching changes the economics considerably. If that repeated prefix qualifies for caching, those reused tokens can be classified as cached input and charged at a substantially lower rate rather than the normal input price.

So far, this explains the basic economics of the meter: what goes in, what comes out, why they are priced differently, and why the same information can end up being processed more than once. But this is still the relatively simple case. Once we move from a conversation with an AI model to agents that retrieve information, call tools, reason, retry and make multiple model calls, the economics become considerably more interesting. That is where I will pick this up in Part 2.

This is the first of two parts. In the next one I follow the meter past the single call, into agents, retries and the private thinking you never see, and to the only number that actually mattered. Part 2 is here.

Editorial note: The original article prose and text-based examples are presented in source order. The executive summary, section headings, takeaways and illustration are additions for this website edition. Technical prices mentioned in the historical article refer to September 2026 and are not current-rate guidance.

Continue reading

Part 2: Why Token Cost Is Not AI CostBrowse Insights