Nvidia's Jensen Huang pushes tokens as AI's new currency, but experts say the metric falls short

Tokens may soon drive the AI economy

A new economic reality is starting to take hold in AI. It already underpins the industry's giant data centres and it will one day become an iron rule for all companies that use machine-generated intelligence. That, at least, is according to Jensen Huang, chief executive of Nvidia, who promoted the idea heavily at his company's main annual tech event this week. His theory helps to make a case for Nvidia's continued dominance in chips. But it also reveals how far the industry has to go to make a wider case for the technology. Huang's take on AI economics is based around the production, consumption and monetisation of tokens. These are the most basic units of output from large language models: it takes about 1,300 tokens to generate 1,000 words of text. The key metric, he argues, is the cost per token of output. And as the main input into AI-powered services, he adds, tokens translate directly into revenue. It is not hard to see why the Nvidia boss wants a nervous Wall Street to focus on token economics. Forget the gargantuan capital spending or the fact that so many competitors are lining up to eat into Nvidia's fat profit margins, he seems to be saying: as long as his company's chips keep pumping out tokens at the lowest cost and as long as demand for tokens continues to far outstrip supply, then all is well with the AI boom. As a theory of Nvidia's continued pre-eminence, it sounds compelling. But if token economics is ever to rule the AI world in the way that Huang predicts, some important gaps need to be filled in. One is the lack of a clear link between the production of tokens and the creation of value for customers. Just because the cost of tokens is falling doesn't mean the services created with AI suddenly become valuable or that this will automatically generate revenue across the industry, as Huang suggests. Complicating this picture is the fact that newer AI models consume far larger numbers of tokens. The "reasoning" models that emerged late in 2024, starting with OpenAI's o1, perform far more work to arrive at an answer. These are now being supplemented by agents, which promise to automate white-collar work and bring an explosion in token use -- and, by extension, hefty bills for companies that give workers unlimited use of AI. Nvidia and the rest of the AI industry have barely scratched the surface when it comes to showing how this will translate into revenue for their customers. In software engineering, which has seen the first widespread use of AI agents, there have been efforts to measure how token use is linked to output and to use this to apportion tokens to workers. Eventually, tech companies dream of AI becoming a core part of employment, with the cost of all white-collar workers coming to be seen as a salary plus a certain number of tokens per month. For now, that is still only a pipe dream. The second significant piece missing from Huang's narrative about an emerging token economy is how the companies that produce tokens, the raw commodity on which all of this depends, will make profits. If these "AI factories" all use Nvidia's latest chips, then it may be hard for any of them to gain a cost-per-token advantage or retain any pricing power. The big price declines that have accompanied the plunging cost of producing tokens seem to bear this out. When OpenAI launched GPT-4 two years ago, for instance, it charged $33 for 1mn tokens. Today, it charges only 9 cents for 1mn tokens produced by its cheapest model. That may be great for customers, but it has fed worries about commoditisation. Such worries are hardly new. It is the same argument that was heard in the early days of cloud computing, when Amazon Web Services charged for access to basic data storage and computing power. How could cloud companies ever make a decent profit if computing services were stripped back and sold as commodities like this? The answer was that these were only the first components of what became higher-value services -- full-scale computing platforms on which customers could run their businesses. Whether OpenAI and Anthropic will be able to work a similar trick is unclear, but the opportunity before them is clear. There may be other explanations for the healthy profit margins in cloud computing. The business is ruled by a small oligopoly. Cloud companies have also faced pressure from regulators to reduce switching costs that may help to pad their profits. For now, there is no shortage of competition among frontier AI companies. How that shakes out in future will go a long way to shaping the industry's profits.

AI Adoption Is Being Measured in Tokens, but the Metric Falls Short, Experts Say | PYMNTS.com

By completing this form, you agree to receive marketing communications from PYMNTS and to the sharing of your information with our sponsor, if applicable, in accordance with our Privacy Policy and Terms and Conditions. Tokens are the foundational unit through which AI models process all information. Tokens are tiny units of data that result from breaking larger chunks of information into smaller pieces. AI models process tokens to learn relationships between them and unlock capabilities such as prediction, generation and reasoning. For large language models, short words may be represented by a single token, while longer words may be split into two or more tokens. The word "darkness," for example, would be split into two tokens, "dark" and "ness," with each token bearing a numerical representation as explained by Nvidia. Every prompt a worker sends to an AI system and every response the system returns are measured and often billed in tokens. Every prompt and response consumes tokens and incurs charges. That direct relationship between usage and cost is what makes tokens attractive as a management tool. Unlike the seat-based pricing that defined earlier generations of enterprise software, token consumption is granular, real-time and tied directly to behavior. The shift from seat counts to token consumption mirrors how enterprise AI spending itself has changed. While the unit price of AI tokens is falling, overall enterprise spending on and scaling of AI systems is rising. The number of users, complexity of models, and intensity of workloads will likely drive greater token consumption and, consequently, higher costs. OpenAI's own data on its enterprise customer base illustrates how dramatically usage patterns have shifted. Average reasoning token consumption per organization has increased by approximately 320 times in the past 12 months, suggesting that more intelligent models are being systematically integrated into expanding products and services. That figure has become a headline metric in the company's internal reporting on adoption progress. Nvidia CEO Jensen Huang went further at the company's GTC conference this week, framing tokens as a new form of corporate currency. "I could totally imagine in the future every single engineer in our company will need an annual token budget," Huang said, estimating that employee token allocations could reach half of base salary in value. The appeal of token metrics runs into a fundamental problem: tokens measure volume, not outcome. Generating through packaged software abstracts tokens almost entirely, while consuming through APIs makes tokens explicit, but this can bring transparency and also volatility, as costs rise based on workload design, prompt length, and hidden choices of infrastructure providers. A poorly structured prompt that forces the model to iterate, rephrase or regenerate a response will consume more tokens than a concise, well-targeted query, yet both may or may not produce useful output. If an AI agent saves a customer service representative 15 minutes of work, but costs $4 in inference tokens to run, the ROI is negative, as explained by AnalyticsWeek. The kind unit-economics mismatch is more visible as companies move from pilots to production deployments. As companies move from experimental chatbots to thousands of autonomous "agentic" workflows running around the clock, the sheer volume of tokens consumed has created a massive budgetary leak. The dynamic draws comparisons to earlier enterprise metrics that proved easier to game than to interpret. Click-through rates once served as a proxy for advertising effectiveness; hours logged once functioned as a proxy for productivity. Both created incentives that diverged from the outcomes they were meant to track. If token consumption becomes a performance indicator tied to employee evaluations, workers may optimize for AI interaction frequency rather than task quality. Knowing that "AI spend is up 40%" is not enough. Organizations need a single pane of glass that links every workload, tenant and token to their owners or business outcomes.