Executive summary
Agentic workflows can involve multiple model calls, tool results, repeated context, and reasoning that is not visible in the final response. Token prices also reflect provider economics, hardware use, overhead, and market positioning. For an enterprise, the more useful measure moves beyond cost per token toward cost per completed, successful business outcome.
In Part 1, I looked at what we are actually paying for when we pay for AI tokens - input, output, repeated context and why the same information can cross the meter more than once. But that explanation was largely about individual model calls and conversations. The economics become much less obvious when AI starts doing work through agents, tools and multiple inference calls behind a single user request.
Why a visible answer hides the work
Agentic AI makes this even more interesting.
Imagine you ask an AI assistant to plan a trip. One simple request, but under the hood it might execute something like:
View the illustration at full sizeYou ask: "Plan a 3-day trip to Lisbon under $1,500"
↓
LLM call #1 (understand the request)
↓
search flights
↓
LLM call #2
↓
search hotels and weather
↓
LLM call #3
↓
build the itinerary
↓
LLM call #4
↓
final recommendationYou might see only: "Here is your 3-day Lisbon plan. Total: $1,430, with two options for the second evening." Perhaps 300 visible words.
But economically, underneath, there could have been:
Call #1 8,000 input 600 output
Call #2 12,000 input 800 output
Call #3 18,000 input 2,000 output
Call #4 20,000 input 1,500 output
+
hidden reasoning
+
tool callsSo a 300-word user experience can represent tens of thousands of billable tokens.
And tool use creates another interesting loop:
MODEL ↓ output "Search for flights to Lisbon under $600" SEARCH TOOL ↓ 2,000-token result MODEL ↑ those 2,000 tokens now become INPUT
The model's tool-call instructions can be output tokens; the returned material becomes input tokens to another inference. So information can cross the token meter more than once.
There is an additional wrinkle with reasoning models. A model can produce a relatively short visible answer but consume internal reasoning tokens that you never see. OpenAI specifically says reasoning tokens are counted as output usage for relevant models (should it consider it as output usage or input tokens could be debated no doubt !).
So this:
You see: 500-token answer
could conceptually involve:
15,000 input tokens 4,000 internal reasoning tokens 500 visible response tokens
The economics are not necessarily based on only those 500 visible tokens.
What sets the token price
Now to the more interesting question: how does an AI company decide what $1 million tokens should cost? There is no magical intrinsic value of a token. Token price is an economic abstraction placed over inference costs. Underneath it sits something approximately like:
TOKEN PRICE
▲
│
┌──────────────────┼──────────────────┐
│ │ │
COMPUTE COST BUSINESS COST MARKET
│ │ │
GPU depreciation Model training Competition
Electricity R&D Demand
Datacenter Networking Positioning
Memory Engineering Margins
Cooling Safety Price discrimination
Utilization Support Enterprise SLAOne extremely important variable is GPU utilization.
Suppose a $30,000-$50,000+ accelerator theoretically has enormous throughput. If you can keep it highly utilized, batch requests efficiently and serve thousands of customers simultaneously, your cost per token becomes dramatically lower.
If instead customers demand:
- extremely low latency,
- huge context windows,
- guaranteed capacity,
- very large models,
- little batching,
- geographic processing requirements,
the same physical hardware produces fewer economically useful tokens. So the provider is effectively thinking:
Cost per token ≈ (GPU + Energy + Infrastructure + Serving) / Useful tokens produced
Then add:
Price = Cost + R&D recovery + overhead + margin + strategic pricing
The published token price therefore is not the electricity or GPU cost of that token. That distinction matters enormously.
An AI provider might actually spend $0.20 producing something it sells for $1, or $0.70, or $0.05 depending upon utilization, model architecture, contracts, hardware, batching and other factors. We generally cannot infer exact underlying cost from the public token tariff.
This gives you a better mental model of AI economics:
WHAT CUSTOMER BUYS
│
"1 million tokens"
│
▼
┌────────────────────┐
│ Pricing abstraction │
└─────────┬──────────┘
│
┌───────────────┼───────────────┐
▼ ▼ ▼
PREFILL DECODE REASONING
(input) (output) / agents
│ │ │
relatively relatively potentially
cheap expensive many calls
└───────────────┬───────────────┘
▼
GPU COMPUTE
▼
REAL CASH COSTAnd this produces what I think is the most useful statement: A token is primarily a billing meter, not a unit of economic cost.
Two million tokens consumed by two different models can have radically different real compute costs. One million input tokens and one million generated tokens don't have the same compute characteristics. One million cached tokens are different again. And one million tokens used across an agentic workflow can imply different infrastructure utilization from a single batch inference.
Cost per successful business outcome
There is also a second-order economic problem that is becoming increasingly important: cost per business outcome.
Suppose:
Model A: $1 / million tokens 10 calls, ~40,000 tokens each = 400,000 tokens 400,000 × $1/M = $0.40 Model B: $5 / million tokens 1 call, ~20,000 tokens 20,000 × $5/M = $0.10
The apparently expensive model is actually cheaper.
OpenAI's Codex lead has publicly cautioned that comparing only price per million tokens can be misleading because models can tokenize differently and consume different amounts of output/reasoning to accomplish the same task. That is why, particularly for enterprise AI, I would not ultimately manage economics using: "$ per million tokens."
I would build a hierarchy:
$ / token
↓
$ / inference
↓
$ / agent run
↓
$ / completed workflow
↓
$ / successful business outcomeThe last number is what a CFO should care about, the number I believe businesses will increasingly have to learn how to govern. And that, to me, is the larger point behind both parts.
Tokens give us a convenient way to meter AI consumption, but they are not the economics themselves. As AI moves deeper into real business processes, knowing the price of a million tokens will matter less than knowing how much AI was consumed to produce a successful outcome.
The progression from cost per token → cost per inference → cost per agent run → cost per completed workflow → cost per successful business outcome is ultimately a progression from measuring technology consumption to measuring business value.
Editorial note: The original article wording and sequence are preserved. The executive summary, section headings, takeaways and illustration are editorial additions.
STRATAGEM LABS
