AI Economics Playbook · Part 2 of 2

The Economics of AI Tokens, Part 2: Why Token Cost Is Not AI Cost.

The unit we are billed for is not necessarily the unit that tells us what AI truly costs or what it delivers.

Kiran SusarlaOriginally published September 12, 2026 · Approximately 5 min read
A floating folded blue sheet casting an archipelago-shaped shadow.

Executive summary

Agentic workflows can involve multiple model calls, tool results, repeated context, and reasoning that is not visible in the final response. Token prices also reflect provider economics, hardware use, overhead, and market positioning. For an enterprise, the more useful measure moves beyond cost per token toward cost per completed, successful business outcome.

In Part 1, I looked at what we are actually paying for when we pay for AI tokens - input, output, repeated context and why the same information can cross the meter more than once. But that explanation was largely about individual model calls and conversations. The economics become much less obvious when AI starts doing work through agents, tools and multiple inference calls behind a single user request.

Why a visible answer hides the work

Agentic AI makes this even more interesting.

Imagine you ask an AI assistant to plan a trip. One simple request, but under the hood it might execute something like:

AI Economics Playbook: token usage proceeds through model inferences, agent runs and workflows toward the cost per successful business outcomeView the illustration at full size
Conceptual framework based on the article. Tokens measure AI consumption; successful business outcomes express the economic result.
You ask: "Plan a 3-day trip to Lisbon under $1,500"
        ↓
LLM call #1  (understand the request)
        ↓
search flights
        ↓
LLM call #2
        ↓
search hotels and weather
        ↓
LLM call #3
        ↓
build the itinerary
        ↓
LLM call #4
        ↓
final recommendation

You might see only: "Here is your 3-day Lisbon plan. Total: $1,430, with two options for the second evening." Perhaps 300 visible words.

But economically, underneath, there could have been:

Call #1     8,000 input     600 output
Call #2    12,000 input     800 output
Call #3    18,000 input   2,000 output
Call #4    20,000 input   1,500 output
                           +
                   hidden reasoning
                           +
                       tool calls

So a 300-word user experience can represent tens of thousands of billable tokens.

And tool use creates another interesting loop:

MODEL
   ↓ output
"Search for flights to Lisbon under $600"

SEARCH TOOL
   ↓
2,000-token result

MODEL
   ↑
those 2,000 tokens now become INPUT

The model's tool-call instructions can be output tokens; the returned material becomes input tokens to another inference. So information can cross the token meter more than once.

There is an additional wrinkle with reasoning models. A model can produce a relatively short visible answer but consume internal reasoning tokens that you never see. OpenAI specifically says reasoning tokens are counted as output usage for relevant models (should it consider it as output usage or input tokens could be debated no doubt !).

So this:

You see: 500-token answer

could conceptually involve:

15,000 input tokens
 4,000 internal reasoning tokens
   500 visible response tokens

The economics are not necessarily based on only those 500 visible tokens.

What sets the token price

Now to the more interesting question: how does an AI company decide what $1 million tokens should cost? There is no magical intrinsic value of a token. Token price is an economic abstraction placed over inference costs. Underneath it sits something approximately like:

                         TOKEN PRICE
                             ▲
                             │
          ┌──────────────────┼──────────────────┐
          │                  │                  │
     COMPUTE COST       BUSINESS COST       MARKET
          │                  │                  │
   GPU depreciation     Model training      Competition
   Electricity          R&D                 Demand
   Datacenter            Networking          Positioning
   Memory                Engineering         Margins
   Cooling               Safety              Price discrimination
   Utilization           Support             Enterprise SLA

One extremely important variable is GPU utilization.

Suppose a $30,000-$50,000+ accelerator theoretically has enormous throughput. If you can keep it highly utilized, batch requests efficiently and serve thousands of customers simultaneously, your cost per token becomes dramatically lower.

If instead customers demand:

  • extremely low latency,
  • huge context windows,
  • guaranteed capacity,
  • very large models,
  • little batching,
  • geographic processing requirements,

the same physical hardware produces fewer economically useful tokens. So the provider is effectively thinking:

Cost per token ≈ (GPU + Energy + Infrastructure + Serving) / Useful tokens produced

Then add:

Price = Cost + R&D recovery + overhead + margin + strategic pricing

The published token price therefore is not the electricity or GPU cost of that token. That distinction matters enormously.

An AI provider might actually spend $0.20 producing something it sells for $1, or $0.70, or $0.05 depending upon utilization, model architecture, contracts, hardware, batching and other factors. We generally cannot infer exact underlying cost from the public token tariff.

This gives you a better mental model of AI economics:

                      WHAT CUSTOMER BUYS
                             │
                      "1 million tokens"
                             │
                             ▼
                  ┌────────────────────┐
                  │  Pricing abstraction │
                  └─────────┬──────────┘
                            │
            ┌───────────────┼───────────────┐
            ▼               ▼               ▼
         PREFILL          DECODE          REASONING
        (input)          (output)          / agents
            │               │               │
        relatively       relatively       potentially
         cheap            expensive        many calls
            └───────────────┬───────────────┘
                            ▼
                       GPU COMPUTE
                            ▼
                     REAL CASH COST

And this produces what I think is the most useful statement: A token is primarily a billing meter, not a unit of economic cost.

Two million tokens consumed by two different models can have radically different real compute costs. One million input tokens and one million generated tokens don't have the same compute characteristics. One million cached tokens are different again. And one million tokens used across an agentic workflow can imply different infrastructure utilization from a single batch inference.

Cost per successful business outcome

There is also a second-order economic problem that is becoming increasingly important: cost per business outcome.

Suppose:

Model A: $1 / million tokens
10 calls, ~40,000 tokens each = 400,000 tokens
400,000 × $1/M = $0.40

Model B: $5 / million tokens
1 call, ~20,000 tokens
20,000 × $5/M = $0.10

The apparently expensive model is actually cheaper.

OpenAI's Codex lead has publicly cautioned that comparing only price per million tokens can be misleading because models can tokenize differently and consume different amounts of output/reasoning to accomplish the same task. That is why, particularly for enterprise AI, I would not ultimately manage economics using: "$ per million tokens."

I would build a hierarchy:

$ / token
     ↓
$ / inference
     ↓
$ / agent run
     ↓
$ / completed workflow
     ↓
$ / successful business outcome

The last number is what a CFO should care about, the number I believe businesses will increasingly have to learn how to govern. And that, to me, is the larger point behind both parts.

Tokens give us a convenient way to meter AI consumption, but they are not the economics themselves. As AI moves deeper into real business processes, knowing the price of a million tokens will matter less than knowing how much AI was consumed to produce a successful outcome.

The progression from cost per token → cost per inference → cost per agent run → cost per completed workflow → cost per successful business outcome is ultimately a progression from measuring technology consumption to measuring business value.

Editorial note: The original article wording and sequence are preserved. The executive summary, section headings, takeaways and illustration are editorial additions.

Continue reading

Browse Insights