AI Costs Spiral Out of Control as Token Spending Exposes Hidden Tax on Enterprise Ambitions

The enduring paradox of the AI economy -- models get better and more efficient, yet costs can still easily spiral out of control

Much has been written regarding the questionable economics of the AI space, but most of the discussion revolves around high-level concepts like market shares, datacenter investments, and power expenditure. The general expectation of technology is that it gets cheaper as it improves, but the AI space has a rather peculiar problem: Usage costs are actually soaring even as models get better, as detailed in a write-up at VentureBeat. With all the advancements in models over the last two years, having a bot that answers questions of simple-to-moderate difficulty is now old news, as they all do that with reasonable accuracy. Agentic workloads are where the real potential is at, letting bots loose on multi-stage, repeatable tasks that would previously take a human days, if not weeks, to perform. Grant a bot access to your billing system, some Excel spreadsheets, and your CRM, and ask it something like "who are my profitable customers by category and what are their trends" becomes child's play. While it's trivial for you to ask that question and get a fairly accurate answer back in a couple minutes, behind the scenes there is a lot of processing going on, and far more than you'd expect. AI computing time is measured in tokens -- a short question and answer might take somewhere between 200 to 2,000 tokens, and one that requires the models to do some internet research might be around 1,000 to 4,000. An agentic task, though, can easily spend millions of tokens on a seemingly innocuous request. How? Token amplification, a recently coined term. In a simplified manner, because a model has no memory or cognition, every time you ask it another question in a conversation, it will re-load and process the entire exchange -- everything you wrote, everything the bot replied with, and every file you uploaded. That means that additional questions in a long conversation progressively get more costly. Each question in a conversation might only need 500 tokens by itself, but reprocessing all the previous information adds up, so the second one might spend 600, and so on and so forth. The conversation as a whole uses up the cumulative number of all individual interactions, and as an added penalty: Response speed also tends to get slower as chats drag on. The aforementioned task of generating a report will have to be run in stages, say three for looking up Excel sheets, four for the CRM, perhaps a half-dozen web searches for contextual information about products, and a good dozen intermediary processing and calculation steps. Each step tacks on potentially several thousands of tokens, and by the end of it, you may be looking at millions of tokens cumulatively spent for all the steps combined. Costs, then, can spiral out of control very quickly -- and that's assuming a best-case scenario where all of the data is easy to interpret, the model won't need any retries, and you're using a moderately powerful model rather than something like Claude Opus. In financial terms, this means that one innocent question might spend 280,000 tokens and cost $1, with ballpark estimates at current prices. That may not sound like much, but it was just one task. If you have this scheduled to run every 15 minutes for a dashboard, then suddenly this costs $96 per day, or $2,880 in a month. And that task example was fairly simple; a tricky multi-stage report might need millions of tokens, turning that $1 into, say, $10, or a grand total of $28,800. For the one report, for a limited number of users, in one department of a corporation. That's the key reason why Anthropic, Microsoft, OpenAI, and almost every other player moved their major offerings to usage-based billing back in April, and severely limited token spends in fixed-rate plans. This generated many sticker shock events, as developers around the globe found out how much their vibe coding actually cost and were left in a daze. In a large company using many AI agents, such costs can become prohibitive and oftentimes costlier than standard-issue human employees. Firms like Uber, Microsoft, Amazon, and Walmart, among many others, have responded by curbing AI spend. Token expenditure is suddenly an issue for both financial and engineering departments, as cost control becomes paramount. As VentureBeat succinctly puts it, for agent-heavy companies, "a prompt redesign is a margin event," and more illustratively, "a poorly bound agent loop is an outage with a credit card attached." Although most AI outfits still offer fixed monthly pricing plans, they often come with harsh token limits. As is often the case with most fixed-pricing services, however, the most capable bot-wranglers will also inevitably be the ones burning through the most tokens, not just due to their intensive usage per se, but because the type of tasks they will perform will be precisely the ones hardest hit by token amplification. This throws a wrench in the usual financial model of the casual users easily offsetting the cost of the small percentage of professionals -- the pros' usage can take a serious bite off monthly profits, or chew through them entirely. The AI companies aren't sitting still, and getting the per-token cost down is likely to be the primary task for most of their engineering teams, at this point. Prompt caching, model routing, batch processing, semantic caching, and context window management are among many technical measures that can massively cut down on token spend, each of those netting a two-digit percentage drop. And yet, costs keep soaring for the simple reason that the smarter the models get, and the more adept people become at using them, the more complex and long-running the agentic tasks will become, adding multiple orders of magnitude to token counts. For the time being, it looks like a losing race, despite recent advancements like Deepseek V4 Pro and V4 Flash deeply undercutting comparable Western models at up to a reported 17x for the former and 25x for the latter. Out of the major Western players, only Anthropic has predicted its first profitable quarter, though much like anyone else, it's still in the hole for many a billion for cumulative spend. Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.

The hidden tax on your AI ambitions

Every enterprise I talk to right now has the same complaint dressed up in different language. Their AI bills are climbing faster than anyone budgeted. Their model invoices look reasonable when viewed on their own. But somewhere between boardroom approvals and the monthly cloud statements, money is disappearing in ways that nobody can fully explain. Here's the central issue that people struggle to understand: the most expensive part of AI isn't always the model itself. It's the infrastructure, orchestration, retries, idle GPUs, oversized context windows, and inefficient routing decisions that sit between a user's prompt and the final response. This is something I call the hidden tax on AI adoption, and it's growing faster than most organizations realize. Three numbers that should change how you think Recently, at FinOps X in San Diego, a Goldman Sachs projection appeared on the main stage. Current enterprise token consumption globally sits at around six quadrillion tokens. The three-year projection: 120 quadrillion. That is not a rounding error... it is a 20x expansion, and it is arriving faster than the governance frameworks to manage it. The same conference saw our launch of the Tokenomics Foundation (I'm fortunate to be a governing board member). It's a vendor-neutral body inside the Linux Foundation dedicated specifically to the economics of AI token consumption. The FinOps community, practitioners who have spent the better part of a decade building discipline around cloud computing spend, recognized that tokens represent a fundamentally different problem. Not a harder version of cloud cost optimization. A different one. Here is why. When your organization runs a cloud workload, the cost is relatively legible. You provision compute, it runs, and you receive a bill. The relationship between action and expense is traceable. With AI tokens, that relationship fractures across three layers, and most organizations have visibility into only one. Production → consumption → value: the three layers most teams ignore The first layer is production. Before any AI model responds to any prompt, your infrastructure has to manufacture the tokens. GPU clusters, inference nodes, autoscaling policies, Kubernetes configurations: these are your token factories. Their efficiency, or lack of it, determines the base cost of everything that follows. A GPU node at 30% utilization is an expensive factory running at a third of capacity. The second layer is consumption. This is where counterintuitive economics live, and it is what I spend most of my time thinking about. Two organizations can send identical prompts to different coding agents and arrive at radically different costs depending on how they manage context, caching, retries, routing, and the infrastructure supporting inference. Prompt length, context window usage, caching strategy, and model routing decisions all compound. The common assumption that routing a task to a cheaper model always saves token cost, turns out to be wrong often enough to matter. A routing decision that invalidates a warm cache can make a "cheaper" model call more expensive than the frontier option it was meant to replace. These are second-order effects. They do not appear in standard dashboards but they appear in your monthly bill. The third layer is value. This is the one that FinOps teams are comfortable with, and the one that matters least until you have the first two under control. Mapping token spend to business outcomes is a legitimate and important discipline. But you cannot govern at the value layer without instrumentation at the production and consumption layers. You are doing math with incomplete inputs. Why 85% of your AI spend is probably misallocated Here is a pattern I see consistently. Organizations treat frontier AI models, the most capable, most expensive models available, as their default infrastructure. Every task goes to the same model. Every prompt is constructed the same way. There is no routing logic, no tiering, no architectural distinction between work that genuinely requires the full capability of a frontier model and work that does not. Based on my observations across organizations deploying AI at scale, roughly 15% of software development tasks actually require frontier model capabilities. The remaining 85% of routine coding, summarization, classification, and retrieval work can be handled by smaller, faster, and less expensive models, if you have the infrastructure to make those decisions intelligently and automatically. The unlock is not picking better models manually. Manual model selection does not scale and degrades the developer experience by introducing friction at the moment when a developer needs to move fast. The unlock is building infrastructure that makes routing decisions for you: one that understands the task, routes it to the appropriate model tier, evaluates output quality, and escalates if needed. You specify the outcome you need. The system handles the economics of achieving it. This is the direction the industry is moving, and the organizations that build this capability first will have a structural cost advantage that compounds over time. The invoice arrives last. And you realize something has gone horribly wrong There is a phrase I have started using with customers that captures the core problem: the invoice arrives last. By the time you see the model provider bill, the cost decisions were made weeks ago in infrastructure configurations, autoscaling policies, and prompt architectures that nobody has reviewed since the initial deployment. The retry logic runs silently when an upstream service slows down. The GPU nodes were reserved for peak traffic that never came. The agentic workflow, where a single user request fans out into dozens of model calls beneath it, each billed separately, none visible in the tool that generated the original request. These costs do not live in the model invoice. They live in the infrastructure layer, in the consumption layer, and in the gap between how teams think their AI systems work and how they actually behave in production. You cannot govern what you cannot see. And right now, most teams are looking at one layer of a three-layer problem. When AI goes from copilot to coworker, the stakes multiply There is a shift underway that makes all of this more urgent. The AI deployments most enterprises built over the last two years were assistants, tools that accelerated individual work by handling the first draft, the next suggestion, and the boilerplate. A human remained in the loop at every consequential step. The economics were bound by how many people were using the tool and how often. Autonomous agents change the economic profile entirely. When an AI system can receive a goal, build a plan, execute multi-step work, evaluate its own outputs, and iterate to completion without human intervention at each stage, you are no longer running an assistant. You are running something closer to a coworker, one that operates continuously, scales horizontally, and generates token consumption at rates that individual user interactions never approached. The transition from copilot to coworker has already happened. And the governance implications are significantly more serious. A copilot with poor token economics costs you some efficiency. An autonomous agent with poor token economics runs that inefficiency at scale, continuously, without generating the natural friction that would cause a human user to pause or change approach. Infrastructure discipline and token optimization need to be in place before autonomous workloads scale rather than retrofitted afterward when the bill arrives. The mandate for infrastructure teams The right answer to this problem is not more dashboards. More visibility into a system you cannot control is just a more detailed invoice; it arrives with the same lag, and it changes nothing about the decisions that were already made upstream. What infrastructure teams actually need is control that operates at the layer where costs are determined, not where they are reported. That means autonomous management of GPU and inference workloads, continuously rightsizing to match actual demand rather than peak assumptions, absorbing the bursty consumption patterns that agentic jobs produce, and moving compute capacity across providers when one environment becomes the bottleneck. This is especially urgent now, because the transition from copilot to coworker does not give you a grace period to retrofit discipline. Autonomous agents do not pause. They do not get frustrated and choose a different approach. They run the inefficiency you built into them at scale, continuously, until something external stops them. So the next phase of enterprise AI is defined by who can deploy models efficiently. As AI systems become more autonomous and token consumption accelerates, competitive advantage comes from understanding the full economics of AI, and not just the price of a model call. Because by the time the invoice arrives, the decisions that shaped it have already been made. Use the best business cloud storage to manage your data. This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit

'You just hired a million bad employees': How the brief tokenmaxxing era delivered the opposite of what it promised | Fortune

George Sivulka, the 27-year-old Stanford grad who founded and runs the AI enterprise startup Hebbia, has put his finger on the defining anxiety of the AI agent era: companies raced to deploy AI workforces without building any of the management infrastructure to run them. Hebbia already serves clients including BlackRock, KKR, and the U.S. Air Force, giving Sivulka a front-row seat to how badly this is going inside real enterprises. His essay, published on a16z's newsletter, argues that AI didn't cut labor costs -- it inverted the equation entirely: "for the first time in history, humans are cheaper than software." And for all those CEOs who rushed headfirst into the agent era briefly known as "tokenmaxxing," he offered a warning: "you just hired a million bad employees." The era ended when Amazon famously disclosed a $500 million loss in one month alone as agents ran wild to little effect, while Ford Motor Company actually hired back a new force of human engineer "graybeards" to work hand in hand with AI augmentation efforts. To Sivulka, the moment is like a mostly forgotten railroad crash in 1841 that ended one era, and began another. The railroad crash analogy Sivulka reached back to the 1830s and 1840s, when American railroad track mileage exploded roughly 120-fold in a decade with no coordination systems to match the growth, until a fatal train collision in Massachusetts in 1841 forced the industry to invent modern management -- defined roles, reporting lines, hierarchies. That crisis, he argued, is what turned railroads into one of the first billion-dollar industries, "at its peak representing roughly 60% of the stock market." Just as railroads unleashed the appetite to travel across the country, he argued that agents have done something similar to work on the web: "We just gave every employee, even the worst ones, effectively unlimited headcount and budget. Managing AI is harder than managing people, because AI scales dysfunction instantly." Why every company "hired a million bad employees" The essay's title captures Sivulka's core diagnosis: AI agents don't fail because the models are weak -- they fail because almost nobody in a company can articulate a task clearly enough for an agent to execute it well. He estimates just "1 in 100 employees knows how to give AI context," calling that skill "a rare breed" of clear thinking that most workers simply don't have. The result is what he calls "looping" -- agents calling themselves over and over to self-correct for bad instructions, which he bluntly describes as "spending tokens on spending tokens." UBS Global Research hosted "many of the highest-profile AI-native firms" at its 5th Annual UBS Private AI and Software event in Menlo Park at nearly the same time, and nearly every executive privately confirmed Sivulka's thesis, of agentic failure happening at industrial scale. One firm executive told UBS' analysts: "Internally, we don't have token budgets, it's not something that we, our engineers, have been trained to think about. But every single customer conversation is about this." The real cost wasn't the tokens Sivulka argues the industry misdiagnosed its own hype cycle: "tokenmaxxing" spending exploded and then collapsed within a month, but "the amount of tokens spent was never the real problem" -- the problem was that people didn't know how to use them efficiently. UBS' sourcing puts real numbers behind that claim: one unnamed AI firm disclosed, "our spend on Anthropic was $20k in December and we're about to cross $1m in July, a 50x increase in 7 months," adding that despite the surge, "we're not throttling back, we don't want people to stop using it." That firm is nonetheless installing what amounts to Sivulka's missing management layer after the fact: "we're now alerting if you hit a certain threshold on a monthly basis, we're going to start rolling out governors internally, like G&A staff should not be using the frontier models." Public statements from OpenAI ("AI costs have now become a huge issue that never came up at the start of the year") and enterprises like Uber installing spend guardrails confirm this isn't isolated -- UBS estimated last month that token-cost anxiety had become "a real concern for ~60% of organizations," and "that figure now feels higher." Sivulka's point directly echoes Palantir CEO Alex Karp's public complaints that AI labs have "completely, irresponsibly, oversold" their models while enterprises burn money on token consumption without real ROI discipline. "Something has gone completely wrong," Karp told CNBC's Squawk Box earlier this month as he vented his spleen over misguided token usage. "The basic view among enterprises in this country is I'm going to chillax and waste my time with tokens." Sivulka systematically reframes AI marketing claims by testing them against a workforce lens, and finds every one breaks down under scrutiny: He extends this to a broader indictment of bloated org charts, noting most companies are already "mismanaged" with workers who function as "cogs in the machine" -- and that Elon Musk's 80% staff cut at X performed better precisely because the cuts removed dead weight, which he says is mirrored by AI: "Just like 80% of employees do nothing, 80% of tokens today do nothing." His fix: The "100x token" Rather than concluding AI is broken, Sivulka argued that the solution is the same one railroads found in the 1840s: better management, not less technology. He predicted the defining skill of the next decade won't be engineering talent but context engineering: "The 10x engineer built the last era of companies. The 100x token will build the next." His summary of the economic shift happening now is that "humans are cheaper than tokens on average, but good tokens are cheaper at scale. Management converts one into the other." UBS's sources describe working toward this exact fix independently through "model routing" -- matching specific tasks to specific models rather than treating AI as one undifferentiated tool. One AI firm executive explained the shift: "About six months ago, we'd take a whole task and say, 'all right, this model is probably the best model for it.' Now, the individual sub-tasks within that project will go to different models because we know exactly which models are good at which tasks." That same executive described the payoff in terms almost identical to Sivulka's "100x" framing: "There's a cost arbitrage opportunity when you can source from a bunch of different models... we have all this data on what the models are good at, what specific tasks they're good at, and so there's a lot of arbitrage on the price side that we can have and that we can pass on to our clients." Another firm described the split even more explicitly, telling UBS that for routine workflows "where the human-like experience doesn't matter as much," they now use "cheaper or faster models," while reserving frontier models for "core use cases" that are "our bread and butter" -- a real-time version of Sivulka's argument that management, not raw model power, converts wasted spend into leverage. Sivulka also warned of organizational friction ahead, as employees start resisting handing over their institutional knowledge to AI systems that may eventually replace them. He pointed to Meta, where equity-holding employees -- despite being financially incentivized to want AI to succeed -- have pushed back against the company using their own work context as training data, calling it "a microcosm of what is about to happen across every industry." "Context hoarding," he warned, is emerging as "the latest job security tactic," describing it as a "massive political problem with AI" inside companies that will only get worse. "Employees don't want to teach AI systems their secret sauce," and now that they know their management isn't smart enough to use tokens cheaply, they have leverage.

Teneo & Thoughtworks CEOs: the AI race will be won with governance, not speed | Fortune

Tokens - the base units for measuring and pricing AI usage - have quickly become one of the most important metrics for corporations. Enterprise AI model pricing has shifted from static subscriptions to dynamic usage-based pricing, and now rising AI consumption has turned a productivity experiment into a potential source of margin pressure. CEOs are caught in a balancing act. AI is not the kind of software tool that can be turned on and off at will. Over the last year, large language models have become deeply embedded across business processes. All signs indicate that reliance is here to stay. Indiscriminately restricting AI use now - like many executive teams are considering - will not only slow down the growth and efficiencies companies have gained, it will leave them less prepared to capitalize on the next generation of AI capabilities as experimentation becomes discouraged. The risk is that executives fall into a trap of whipsawing their AI spend to balance the next quarterly budget, and in doing so miss out on the underlying transformation of their business that AI will create when governed systematically, not reactively. We have entered the second phase of AI adoption, where the mandate has changed from rapidly demonstrating competency and progress, to now demonstrating companies can extract the greatest possible value out of AI without threatening their bottom line. This is the sustainable adoption phase, and it will soon expose a divide with long-lasting impacts in the corporate landscape between organizations that can manage and scale AI economically, and those that cannot. Token costs are simply the first visible symptom of a broader governance problem. Two assumptions led to this inflection point. The first was an appealing but not wholly accurate thesis on what AI would allow businesses to do: swap human labor costs for model costs and capture the efficiency delta as profit. That tradeoff has not been as straightforward. Many organizations that leaned into it, reducing headcount in anticipation of AI-driven productivity gains, have found they shed institutional knowledge and critical engineering talent needed to effectively integrate and refine AI systems over time. That realization is hitting many companies that conducted large-scale layoffs. In a recent high-profile example, Ford said it must rehire hundreds of engineers to address quality control issues with newly implemented AI tools. Executives directly cited the lack of veteran expertise negatively impacting product development and limiting efficiency gains from autonomous systems. The other assumption was a misalignment between investors and management teams. In Teneo's most recent annual CEO and investor survey, 53 percent of investors expected return on investment from AI within six months, while only 16 percent of large-cap CEOs believed they could deliver on that timeline. That deadline has now arrived. The window to build disciplined AI governance is open, but it will not stay open indefinitely. Between our two firms, we engage with thousands of CEOs and boards globally to address these challenges. Here is what we are advising them. First, reframe the AI conversation in the boardroom. Stop asking how much you are spending on AI. Start asking where AI investment is creating durable competitive advantages, and where it is generating consumption without compounding value. Second, treat AI spend as a capital allocation decision, not an IT budget line. Usage that drives new revenue, creates differentiated customer experiences, or builds proprietary capabilities is a growth investment. Usage that automates low-value processes is an operating expense. Third, establish governance mechanisms that match workloads to the least expensive model capable of performing them reliably. Employees will naturally gravitate toward the latest and greatest models, even when older generations can produce the desired output. One implementation may be a software layer that can intake prompts and automatically route them to the appropriate model. Fourth, reshape incentives around AI use. Employees should not be rewarded for using AI the most, which was an early instinct among companies eager to show investors how advanced their adoption was. Nor should they be penalized with blunt usage caps, because that can inadvertently stifle innovation. Instead, organizations should reward efficient AI use: achieving better business outcomes with the appropriate level of AI consumption. The goal is not maximum usage, but maximum value per token. Fifth, distribute AI governance throughout the organization. AI governance cannot be delegated to a single role like a Chief AI Officer. Managers across functions need to be accountable for guiding sustainable AI adoption within their teams. Employees, in turn, need practical support both to understand how their AI use is evaluated and to develop the skills required to meaningfully and efficiently contribute with AI in the long-term. The companies that win this phase of AI adoption will not be those that use it the most or those that spend the least. They will be the ones that govern it best, consistently converting AI consumption into lasting economic advantage. It is a difficult balance, especially with a technology evolving this fast. Yet, getting that balance wrong may soon be existential. The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.