AI is getting cheaper every quarter. Enterprise AI budgets keep growing. Both are true at the same time.
OpenAI cut GPT-4 Turbo input costs by 67% in a single announcement. Every major provider followed. The price of a token has never been lower.
So why are the AI bills going up?
It’s not a billing error. The way AI agents consume tokens is fundamentally different from what anyone budgeted for and many organizations are not equipped to handle this type of billing. They’re thinking of their AI spend like they planned their software spending, and if you look at the chart below, you can see it’s anything but.
Table 1.1: Illustrative Token Price Compression vs. Enterprise Consumption Growth (2023–2026)
| Year | Cost per 1M Input Tokens | Cost per 1M Output Tokens | Notable Consumption Shift |
| 2023 | ~$30 | ~$60 | Single turn chat; human speed interaction |
| 2024 | ~$10 | ~$30 | RAG pipelines; early agent pilots |
| 2025 | ~$2 | ~$8 | Multi-step agentic workflows; background automation |
| 2026 | ~$0.50 | ~$2 | Autonomous agent workforces; machine speed token loops |
Prices are approximate and illustrative, drawn from publicly available model pricing pages. Enterprise total spend has trended upward across this period despite per token deflation, because volume exploded.
From Linear Interactivity to Agentic “Tokenmaxxing”
Here’s the thing: the vendors selling those tokens have every financial reason to keep costs the way you see them now, but it’s the professional organizations that figure this spending and budgeting process out now, that will be in a very different position than the ones that wait.
The foundational misunderstanding of enterprise AI budgeting stems from a failure to separate the nominal cost of an individual token from the total behavioral volume of consumption.
Agents broke that model completely.
Here’s what actually happens when an agent runs a real task, reconciling an ERP ledger discrepancy. It gets its instructions, a system prompt, tool definitions, and context. That’s 8,000 tokens before it does anything. Then, it calls a database lookup. The result appends to its context window, another 4,000 tokens. Finally, it decides it needs a second lookup and makes that call.
That’s what kills budgets: by the third iteration, the agent isn’t processing 4,000 new tokens, i’s reprocessing the entire accumulated context from scratch. Every pass. That’s how transformer attention works.
Every time the context window doubles, prefill costs quadruple. Ten iterations in, you’re at 60,000 to 100,000 tokens — and paying compounding costs at every step. The model also starts losing track of things buried in the middle of that bloated context, so quality degrades right as costs peak. A task that fits in a 2,000 token prompt now quietly triggers hundreds of iterations. Some workflows hit millions of tokens per transaction. Tokenmaxxing is not just a new industry term, just an honest name for what’s happening.
Without a governance layer controlling when and how agents call external systems, there’s nothing to stop this before it hits the invoice.
The Capital Intensive Squeeze on Foundational Providers
This would all be convenient if vendors wanted to help you spend less, don’t. Traditional SaaS companies run 80%+ gross margins because serving one more customer costs almost nothing. Frontier AI labs don’t have that. The infrastructure required, HBM chips, specialized data centers, gigawatt-scale power, is not cheap to build or maintain. Frontier labs have disclosed infrastructure spend that dwarfs anything traditional SaaS ever required. That capital has to come back.
Three patterns show up in every public pricing page, earnings call, and product roadmap. None of them are accidents.
The Reasoning Tax. Output tokens cost roughly five times more than input tokens across most providers. With reasoning models now the default, a significant portion of every enterprise bill goes toward “thinking tokens”, internal computation the enterprise never sees but pays for anyway. Unit prices fall. Total costs rise. Gross Revenue Accounting. Vendors are valued on total revenue throughput. Your spend flows through their marketplace and gets counted as their revenue, which creates a direct financial incentive to architect for maximum token consumption. Your bill going up is their growth story. Feature Gating. The cheap Flash or Nano models are real. They are also designed to get your codebase written against a vendor’s SDK. Once that happens, production requirements like data isolation, latency guarantees, and auditability push you toward premium tiers. That is where vendors hold full pricing power. The loss leader worked.
That’s where Agent Fabric comes in. Agent Fabric closes this gap and puts an abstraction layer between your business systems and the vendor API. Price changes stop propagating into operations. Token budgets get enforced. Models swap out when economics shift. The business logic underneath doesn’t know or care which vendor is running behind it.
The Approaching Infrastructure Squeeze
Right now it feels like abundance. Tokens are cheap. Vendors are generous. The capability promises are real. What’s actually being built is switching cost. Model specific prompting pipelines. Internal data mapped to vendor exclusive schemas. Workflow routing locked inside one provider’s ecosystem. None of it is inherently bad, until the vendor shifts from acquiring customers to extracting margin from them. That shift is not hypothetical. Per-seat pricing is already giving way to consumption brackets and capacity minimums in parts of the market. This pattern has played out in every infrastructure market that matured before this one. Aggressive pricing to drive adoption.
Pricing power once the switching cost makes leaving too painful. The organizations with a governed integration layer will absorb that transition. The ones who built directly against proprietary APIs will face a very expensive rebuild, at the worst possible time.
The Mandate for Decoupled Control
Most teams running agents today are flying blind on spend until the invoice lands. They have no unified view of what is running, no way to enforce a budget at runtime, and no path to swap a vendor without rewriting the pipelines built against it. That is not a technology problem. It is an architecture problem.
The fix is a single layer that knows about every agent, API, and model in your environment before they run, not after. One that enforces token budgets, routes tasks across vendors intelligently, and gives your team a real picture of what agents are doing and what it costs. When a vendor raises prices, changes an API, or kills a model, the layer absorbs it. The business logic does not move.
That is what Agent Fabric is. Not an AI product. Not a dashboard. Infrastructure with a memory and a governance posture, the kind that keeps pricing leverage where it belongs.
The onboarding phase is designed to make this feel unnecessary. It isn’t.
To learn more about how Agent Fabric is the key to token costs, visit our Cost Management page.
References
Bort, J. (2025, August 8). OpenAI priced GPT-5 so low, it may spark a price war. TechCrunch.
Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2024). SWE-bench: Can language models resolve real-world GitHub issues? arXiv preprint arXiv:2310.06770.
Keles, F. D., Wijewardena, P. M., & Hegde, C. (2022). On the computational complexity of self-attention. arXiv preprintarXiv:2209.04881.
Kleppmann, M. (2017). Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems.O’Reilly Media.
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Liang, P., & Potts, C. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 157–173.
Singh, M. (2025, January 27). DeepSeek ‘punctures’ AI leaders’ spending plans, and what analysts are saying. TechCrunch
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30, 5998–6008.
Wiggers, K. (2023, November 6). OpenAI debuts GPT-4 Turbo and fine-tuning program for GPT-4. TechCrunch
Zeff, M. (2024, July 18). OpenAI unveils GPT-4o mini, a smaller and cheaper AI model. TechCrunch
