Small Language Models on Desktops Could Reshape the AI Industry's Profit Expectations

The future of AI may be small, cheap and unprofitable

LONDON, June 18 (Reuters) - The AI boom is built on the idea that bigger is better. A recent study suggests the opposite may soon be true: small language models running on desktop computers may be able to handle most of the tasks currently performed by large language models. If that's true, Anthropic, OpenAI and SpaceX's (SPCX.O), opens new tab xAI may have reason to worry. The future of AI could be smaller, cheaper and far less profitable than investors expect. On June 1, Nvidia (NVDA.O), opens new tab made headlines when it revealed a new desktop AI platform that runs on Windows PCs, raising the possibility that the future of AI might not just be in giant data centres. A Stanford University study, opens new tab published two weeks earlier suggested the same. It compared small local language models (SLMs) running on desktop PCs with large language models (LLMs) running in data centres. The researchers tested a range of SLMs, using both PCs and Macs, on 500,000 chat requests and 500,000 reasoning tasks. The study found that, on average, these SLMs were as good as or better than LLMs in over 80% of tasks, with success ratios ⁠approaching 100% in sales, management and entertainment applications. That doesn't mean these SLMs are superior across the board. In the most difficult reasoning tasks, they keep up with LLMs in only about 50% of cases. But that is up from a mere 8% two years ago, showing that the performance gap between SLMs and LLMs is closing fast. More importantly, SLMs are rapidly improving in a metric the researchers call "intelligence per watt," which measures the accuracy of an SLM on a desktop PC relative to the energy consumed. That measure has improved over five times in the last two years. The result is not only that SLMs can perform as well or better than LLMs in most cases, but that they can do so while using 50% to 80% less energy - meaning at lower cost. TROUBLE FOR HYPERSCALERS If this trend continues and SLMs close the performance gap with LLMs faster than the market expects, the consequences for the companies driving today's AI boom could be severe. A couple of months ago, I argued that LLM hallucinations could undermine the long-term business models of the AI heavyweights. This Stanford study now implies that ⁠LLMs are economically the most viable solution in just one-fifth of current use cases. If true, this would undermine the lofty valuations that OpenAI and Anthropic hope to achieve in their IPOs - and call into question SpaceX's $2.85 trillion valuation, which is rooted largely in AI hopes. These firms could enter the race for SLMs by shrinking - or "dumbing down" - their existing models. But the problem is that the most advanced SLMs are open source, meaning they are available for free or at an extremely low cost. Profit margins for LLM providers would thus be much lower in the SLM space. Moreover, an SLM doesn't need a data centre. ⁠If most tasks can be performed at lower cost on a desktop PC, the case for vast data centres packed with expensive GPU, TPU and Trainium chips weakens considerably. Many of those data centres being built today may end up being little more than white elephants. If the data centre surge were to come to an end, that could trigger a chain reaction that reverses the AI boom. Growth expectations ⁠for hyperscalers would be rolled back, and capital expenditures would be curtailed, which, in turn, would slow growth for chipmakers. The only companies that would likely benefit from such a shift would be desktop computer makers such as Apple (AAPL.O), opens new tab - and, potentially, Nvidia, if its new desktop AI platform pans out. In light of this study, Nvidia's foray into desktops looks less like ⁠a diversification strategy and more like a hedge to remain relevant, no matter how this technology evolves. (The views expressed here are those of Joachim Klement, an investment strategist for Panmure Liberum.) Enjoying this column? Check out Reuters Open Interest (ROI),, opens new tab your essential new source for global financial commentary. Follow ROI on LinkedIn,, opens new tab and X., opens new tab And listen to the Morning Bid daily podcast on Apple, opens new tab, Spotify, opens new tab, or the Reuters app, opens new tab. Subscribe to hear Reuters journalists discuss the biggest news in markets and finance seven days a week. Writing by Joachim Klement Editing by Marguerita Choy and Anna Szymanski Our Standards: The Thomson Reuters Trust Principles., opens new tab * Suggested Topics: * ROI: Reuters Open Interest Opinions expressed are those of the author. They do not reflect the views of Reuters News, which, under the Trust Principles, is committed to integrity, independence, and freedom from bias.

The future of AI may be small, cheap and unprofitable: Joachim Klement

A groundbreaking study indicates that lightweight AI models installed on everyday home computers are on the verge of surpassing the robust systems found in data centers. This pivotal change may drive down costs and profit margins for industry leaders such as Anthropic and OpenAI. Nvidia's cutting-edge desktop AI technology serves as a precursor to this exciting new chapter. The AI boom is built on the idea that bigger is better. A recent study suggests the opposite may soon be true: small language models running on desktop computers may be able to handle most of the tasks currently performed by large language models. If that's true, Anthropic, OpenAI and SpaceX's xAI may have reason to worry. The future of AI could be smaller, cheaper and far less profitable than investors expect. On June 1, Nvidia made headlines when it revealed a new desktop AI platform that runs on Windows PCs, raising the possibility that the future of AI might not just be in giant data centres. A Stanford University study published ⁠two weeks ⁠earlier suggested the same. It compared small local language models (SLMs) running on desktop PCs with large language models (LLMs) running in data centres. The researchers tested a range of SLMs, using both PCs and Macs, on 500,000 chat requests and 500,000 reasoning tasks. The study found that, on average, these SLMs were as good as or better than LLMs in over 80% of tasks, with success ratios approaching 100% in sales, management and entertainment applications. That doesn't mean these SLMs are superior across the board. In the most difficult reasoning tasks, they keep up with LLMs in only about 50% of cases. But that is up from a mere 8% two years ago, showing that the performance gap between SLMs and LLMs is closing fast. More importantly, SLMs are rapidly improving in a metric the researchers call "intelligence per watt," which measures the accuracy of an ⁠SLM on a desktop PC relative to the energy consumed. That measure has improved over five times in the last two years. The result is not only that SLMs can perform as well or better than LLMs in most cases, but that they can do so while using 50% to ⁠80% less energy - meaning at lower cost. Trouble for hyperscalers If this trend continues and SLMs close the performance gap with LLMs faster than the market expects, the consequences for the companies driving today's AI boom could be severe. A couple of months ago, I argued that LLM hallucinations could undermine the long-term business models of the AI heavyweights. This Stanford study now implies that LLMs are economically the most viable solution in just one-fifth of current use cases. If true, this would undermine the lofty valuations that OpenAI and Anthropic hope to achieve in their IPOs - and call into question SpaceX's $2.85 trillion valuation, which is rooted largely in AI hopes. These firms could enter the race for SLMs by shrinking - or "dumbing down" - their existing models. But the problem is that the most advanced SLMs are open source, meaning they are available for free or at an extremely low cost. Profit margins for LLM providers would thus be much lower in the SLM space. Moreover, an SLM doesn't need a data centre. If most tasks can be performed at lower cost on a desktop PC, the case for vast data centres packed with expensive GPU, TPU and Trainium chips weakens considerably. Many of those data centres being built today ⁠may end up being little more than white elephants. If the data centre surge were to come to an end, that could trigger a chain reaction that reverses the AI boom. Growth expectations for hyperscalers would be rolled back, and capital expenditures would be curtailed, which, in turn, would slow growth for chipmakers. The only companies that would likely benefit from such a shift would be desktop computer makers such as Apple - and, potentially, Nvidia, if its new desktop AI platform pans out. In light of this study, Nvidia's foray into desktops looks less like a diversification strategy and more like a hedge to remain relevant, no matter how this technology evolves. (The views expressed here are those of Joachim Klement, an investment strategist for Panmure Liberum. )