The invisible meter: Understanding the economic cost and ecological footprint of tokens

In a previous post, the first in a series of articles, I had elucidated some of the key reasons why Agentic Artificial Intelligence (AI) has captured the attention of technology leaders.

This article examines a less visible aspect of that promise: its operating model.

Unlike traditional software, Agentic AI does not simply execute predefined logic. It reasons through tokens, model invocations, tool calls, retries and orchestration flows. This changes the nature of automation from a fixed-cost activity into a consumption-based service. The key architectural question is therefore not whether a process can be automated, but whether the economic and operational footprint of AI-driven reasoning remains sustainable when deployed at scale. Industry analysts and architecture practitioners will argue that token consumption introduces fundamentally different governance and operational challenges than conventional software execution. But then it is up to you to consider the kind of operating model that works best for your enterprise, and if it is worth the risk.

Scale changes everything

Token consumption often goes unnoticed in small pilots. However, at an enterprise level or a national scale, it becomes a persistent operational workload.

Tax authorities, customs organizations, benefits agencies and regulated financial institutions may process millions, or even hundreds of millions, of decisions each year. In these environments, token consumption is no longer an overhead; it becomes an industrial-scale resource requirement.

Parameter	Typical magnitude
Tokens per case	Thousands (multi-step reasoning)
Number of cases	Millions to hundreds of millions
Aggregate	Billions to trillions of tokens annually

The consequence is straightforward. Every reasoning step, retry or orchestration loop contributes to an ever-growing operational footprint. This may appear efficient at a pilot stage, but it can become a significant cost driver when repeated hundreds of millions of times.

“The real question isn't whether AI can automate a process. It's whether you can afford it at 100 million transactions a year.”

A new economic model

In my experience at Atos, we have seen that organizations must treat token consumption as a primary architectural concern rather than an operational afterthought. This has been extensively discussed in the Atos paper, Enterprise-grade Agentic AI: Secure, Governed, and Sovereign by Design. As autonomy increases, so does the need for visibility into token usage, execution loops, retries, anomalies and associated costs. This leads to introducing disciplines such as AgentOps and AgentFinOps to govern runtime behavior.

Gartner reaches a similar conclusion from a different angle, as discussed in Hype Cycle for Application Architecture and Integration 2025 and Hype Cycle for Enterprise Architecture 2025. Traditional technology spending was largely transaction-based and therefore relatively predictable, but token-based consumption behaves differently. As orchestration becomes more sophisticated, costs can rise rapidly, prompting organizations to introduce AI gateways that provide routing, caching, policy enforcement, visibility and cost controls across multiple Large Language Model (LLM) providers. The emergence of such control layers signals that Agentic AI introduces an entirely new economic operating model.

The challenge in this extends well beyond model licensing. Pricing is influenced by query volume, data volume, orchestration complexity, reasoning depth and the number of times a workflow revisits a model before producing an actionable result. In practice, architectural choices increasingly determine the cost profile of AI adoption.

Beyond license costs

The Digital Practitioner Body of Knowledge (DPBoK) reminds technology leaders that digital services always combine direct and indirect costs. Hardware, software, facilities, energy, cooling, operational support and governance mechanisms all contribute to the final delivery cost.

Well, Agentic AI follows the same pattern. A workload that appears inexpensive during experimentation can become much more expensive when it runs continuously across a production landscape. Organizations are paying not only for AI-generated outputs but also for the infrastructure, governance, observability, and resilience capabilities required to keep those systems operational.

Typical token pricing (2025–2026 market range)

Although pricing varies by provider and continues to evolve, Gartner [2, 3] observes a clear trend: organizations are increasingly using architectural techniques such as model distillation, smaller task-specific models, and runtime controls to reduce inference costs.

	Approx. cost per 1K tokens	Cost per token
Small / optimized models	$0.0001 – $0.001	$0.0000001 – $0.000001
Mid-tier models	$0.001 – $0.01	$0.000001 – $0.00001
Frontier models (GPT 4 class)	$0.01 – $0.03 (input), higher for output	$0.00001 – $0.00003+

Governance before scale

Every AI agent carries an invisible meter and, at enterprise scale, that meter can become a multimillion-dollar line item.

The hidden meter behind token consumption is not merely a cost issue; it is also a governance issue. According to Gartner, effective AI governance requires clear decision rights, accountability structures, and oversight mechanisms. Similarly, The Open Group positions governance as the deliberate allocation of responsibility through predefined guardrails rather than through corrective action after deployment.

Applied to Agentic AI, those principles require organizations to answer architectural questions early in the process.

Which activities genuinely require probabilistic reasoning?

Where should throttling, caching or routing controls be introduced?

Which decisions can be executed more effectively through deterministic services?

These choices determine the amount of computation a solution consumes and, in turn, directly influence both financial and environmental outcomes.

Two architectural paths

The impact of these design choices becomes visible when comparing the pure agentic and hybrid execution models. A hybrid architecture uses AI primarily for rule creation, interpretation and refinement, while deterministic services handle operational execution. As a result, runtime token consumption disappears. This hybrid architecture becomes particularly important for highly regulated authority areas, such as administrative agencies and public benefits agencies.

Execution model	Token usage pattern
Pure Agentic	Tokens per case × all cases
Hybrid	Tokens per design activity only

This has direct implications on the cost structure and environmental impact. For a simplified understanding, we assume that one case consumes at least 5000 tokens, but there will be cases that consume manyfold, so the formula may be adapted. Let’s take a closer look at each of these.

Economics of the pure-agentic scenario

Parameter	Assumption	Rationale
Cases per year	10M – 100M	High-volume enterprise or public sector scale
Tokens per case	2,000 – 8,000	Includes prompt, reasoning, orchestration overhead
Cost per token	$0.000001 – $0.00002	Current industry benchmark range
Execution model	Agentic runtime (multi-step reasoning)	Reflects typical AI-driven workflow

Annual costs

	10 Million Cases		100 Million Cases
Tokens per case	Total tokens	Cost range	Total tokens	Cost range
2,000	20B tokens	$20K – $400K	200B tokens	$200K – $4M
5,000	50B tokens	$50K – $1M	500B tokens	$500K – $10M
8,000	80B tokens	$80K – $1.6M	800B tokens	$800K – $16M

As illustrated by these tables, token expenditure is often insignificant during experimentation but can become a major operating cost when applied across enterprise-scale workloads.

Economics of the hybrid model

In this model, Agentic AI is used to develop the required software. This also consumes tokens, but only during the design phase. The result is deterministic decisioning systems, like the classical business-rules-based decision systems, running multi-million cases per year without consuming any tokens.

Component	Token usage
Design phase (one-time)	Moderate
Runtime execution	Near-zero

As a result, annual runtime cost effectively collapses, and token usage becomes decoupled from transaction volume.

Economics compared

Scenario (order of magnitude)	Pure Agentic	Hybrid
10M cases/year	~50B tokens	~0 tokens at runtime
100M cases/year	~500B tokens	~0 tokens at runtime
Trillion-scale workloads	Continuous token flow	None during execution

Pitching a pure agentic scenario against a hybrid one illustrates this using a simple formula:

Total cost = Cases × Tokens per case × cost per token.

Carbon footprint is not a trivial matter

The economic model you choose has an environmental dimension too. Gartner increasingly treats sustainability as an integral component of technology economics, while The Open Group emphasizes that digital enterprises must balance economic, social and environmental outcomes when shaping strategy.

Quantifying CO₂ per billion tokens

Quantifying CO₂ per billion tokens

Based on published research from Nature Machine Intelligence, the University of Massachusetts Amherst and other AI sustainability studies, runtime AI usage can be interpreted as a chain of token consumption, computational workload, energy consumption and, ultimately, carbon emissions. While no universally accepted token-to-carbon conversion exists, these studies consistently demonstrate that reducing token-intensive inference workloads lowers both operating costs and environmental impact.

~0.2 – 2 joules per token (depending on model and infrastructure)

1 billion tokens ≈ 200 million – 2 billion joules ≈ 55 – 555 kWh

~0.3 – 0.5 kg CO₂ per kWh (depending on regional electricity mix)

Resulting Estimate

Metric	Approximate Range
Energy per billion tokens	≈ 50 – 550 kWh
CO₂ per billion tokens	≈ 15 – 275 kg CO₂

The scaling effect: 100 million cases/year × 5,000 tokens per case = 500 billion tokens. At approximately 15–275 kg CO₂ per billion tokens, this amounts to ≈ 7,500 up to137,500 tons of CO₂ per year.

Even at the lower end of the range, the resulting footprint is not a trivial matter. The difference between a pure agentic approach and a hybrid architecture is therefore not incremental.

Pure agentic execution scales token consumption with every transaction. Hybrid architectures restrict token use primarily to rule design and continuous improvement activities. The result is a shift from continuous runtime consumption to near-zero operational token demand.

improvement activities. The result is a shift from continuous runtime consumption to near-zero operational token demand.

Looking ahead

The token economy of Agentic AI is not simply a pricing mechanism; it is an architectural characteristic. It affects cost predictability, governance requirements, runtime operating models and environmental impact. More importantly, it raises a deeper question.

In domains governed by law, regulation and accountability, the challenge may not merely be that probabilistic reasoning is expensive. The challenge may be that probabilistic reasoning is fundamentally the wrong execution model for certain classes of decisions.

Watch out for the third article in this series that examines that issue directly and explores the case for deterministic decisioning in regulated environments. Stay tuned!

Continue the conversation with the Atos Future Makers Research Community. The series explores how enterprises can adopt Agentic AI responsibly while managing cost, sustainability, governance and regulatory accountability.