Google deploys billions to turn TPU chips into serious challenge against Nvidia's AI dominance

Alphabet burnishes one of its best weapons in the battle for AI supremacy

Alphabet has squashed concerns that artificial intelligence will destroy its Google tech empire. One of its biggest weapons in the fight: homegrown silicon chips. Google's in-house tensor processing units (TPUs) serve as the engine to the company's Gemini chatbot, which has bolstered its image in the past year against rivals like OpenAI's ChatGPT. They also represent an integral part of Google's fast-growing cloud-computing business, where customers -- including buzzy AI startup Anthropic -- rent access to the chips; in some cases, they can now buy TPUs for their own data centers. Google also has a new AI compute venture with asset management giant Blackstone, built around the TPU. Google's compute business is seeing strong demand, with Wall Street projecting Google Cloud revenue to surge roughly 64% this year, to $96 billion, according to FactSet. Analysts see robust expansion continuing in 2027, with growth modeled above 50%. With demand for AI computing power surging, Google's TPUs are increasingly seen as a compelling alternative to Nvidia's market-leading graphics processing units (GPUs). They position Alphabet as a major force in AI infrastructure, even as Google Cloud still trails Amazon Web Services and Microsoft Azure in revenue. That status benefits both Google's internal AI efforts and helps win outside customers -- a lucrative one-two punch that figures into Jim Cramer's admiration for the stock. Google is "probably the most underappreciated competitor of Nvidia," said Brad Gastwirth, global head of market research and market intelligence at Circular Technology , a supply chain services firm focused on compute infrastructure. While AI computing is a complex process, the appeal of the TPU comes down to a widely understood idea in life: making your money go further. In this case, the goal is to obtain the most computing power for every dollar spent, an increasingly critical consideration as companies race to deploy AI at scale. Main stages of AI computing At the simplest level, there are two primary stages of AI computing. Training: This happens first. Training teaches an AI model by feeding it massive amounts of data so it can learn patterns and improve its responses. This is the phase in which companies develop large language models such as Gemini. It requires enormous computing power, making it one of the most expensive parts of building AI systems. Inference: The process by which a trained AI model makes predictions or decisions based on new data. Inference is much less computationally heavy than training on a per-task basis. But once a model is deployed, inference is theoretically occurring all the time. So, the cumulative inference costs for a model can exceed its training costs over its lifetime. Put simply, the purpose of training is to learn, while the purpose of inference is to make predictions. The nature of TPUs enables them to deliver strong performance on AI tasks while reducing the cost of running those systems. TPUs belong to a class of chips called application-specific integrated circuits, or ASICs. Gastwirth likened ASICs to a custom suit -- but instead of being tailored to a person's body, the processors are designed specifically for certain tasks. TPUs are optimized for machine learning tasks like training models and running them in real time, a process known as inference. Google co-designs the chips with fellow Club name Broadcom . The specialization of TPUs gives them an edge in efficiency, with William Blair analyst Ralph Schackart noting that they can deliver more computing output with less power. "Most ASICs consume 20% to 40% less energy than Nvidia processors, allowing for greater performance-per-dollar," Schackart continued. Those cost advantages, Schackart said, allow Google to charge about 20% to 30% less for excess compute capacity, which is attracting AI unicorns to Google's offerings, including its cloud business and enterprise services. To be sure, Google's AI computing ambitions face plenty of competition, and investment in innovation is required to stay on the cutting edge. Additionally, everyone in the AI compute business faces risks related to component availability -- from memory chips to other input materials -- and limited manufacturing capacity. Elevated memory costs, in particular, have weighed on the stocks of megacap tech stocks this week. More generally, the tight supply chain can cause delays to server and data center builds and be a limiting factor on growth. Another question mark around Google's AI efforts has emerged in recent days, specifically the loss of talented AI researchers to OpenAI and Anthropic. While these employees worked on developing AI models rather than TPU development, the company's success has come from having capable AI systems running on optimized, well-designed hardware; they complement each other. Shares of Alphabet are down 16% from their early-May peak, coinciding with a broader period of weakness among hyperscalers. For the year, though, Alphabet shares are still up about 8%, outperforming Microsoft , Amazon , and Meta Platforms . The leader Nvidia is the biggest player in AI compute. The company's GPUs offer more flexibility than an ASIC such as a TPU -- after all, GPUs were originally designed to render better 3D computer graphics before their processing power was harnessed for a broader range of tasks, including AI. Today, Nvidia's data center GPUs hold a dominant position in training AI models and are also used for day-to-day inference. In many ways, they are the default chips for the AI era. In addition, Nvidia holds a major advantage with its CUDA software system, which developers have built around for years. CEO Jensen Huang also frequently argues that Nvidia's presence in all the major clouds is a big advantage, saying on an earnings call last year: "The reason why developers love us is because we're literally everywhere." The downside to GPUs? They're expensive, power-hungry relative to specialized TPUs, and hard to get, given their high demand. Nvidia remains the "broad ecosystem leader," with its dominant market share insulated in the near future, analysts at Stifel wrote in a May research note. However, they argued Nvidia's "moat is increasingly being tested." With AI adoption growing, analysts said the market is shifting from a "training-led regime toward inference-led regime by the end of 2026." After ChatGPT burst onto the scene in late 2022, the first wave of AI computing was dominated by training as companies raced to create new models. ChatGPT grew up on Nvidia chips and still relies on them, while also broadening its compute portfolio to include other types of silicon. Fast forward to today, and AI models are seeing rapid adoption from consumers and enterprises alike. That's why the split toward inference is accelerating. The evolution is placing greater focus on compute costs and return on investment, which analysts said is accelerating hyperscalers' interest in homegrown ASICs and alternative AI chips, sometimes called AI accelerators. Advanced Micro Devices is another competitor in the GPU space. Wait ... what about CPUs? In recent months, central processing units (CPUs) have seen a surge in demand after initially being overlooked during the AI compute boom. That's true -- and we haven't forgotten about them. But they play a slightly different role in the AI compute landscape than GPUs and custom accelerators like TPUs. CPUs work in conjunction with accelerator chips, handling more general-purpose tasks, doing much of the "orchestration" for the entire AI system, and keeping the accelerators fed. The main reason for the current CPU renaissance is that agentic AI systems perform many of those general-purpose tasks, such as browsing the web, managing files, and combing through databases. Both Nvidia and Google have designed their own CPUs. AMD and Intel , which is another Club stock, are longtime CPU giants. Club name Arm Holdings is another notable CPU player. Another factor driving interest in custom silicon: Soaring demand for AI computing has created a tight supply environment. It's pushing companies with the capital to take matters into their own hands and develop specialized chips to meet their rising compute demand. Outside of Google's TPUs, Amazon has developed a lineup of custom chips, including its CPU, Graviton, and its AI accelerator, Trainium, which are used internally and sold to customers to power their applications. Microsoft developed its own in-house silicon called Maia to power its cloud infrastructure and reduce costs. Meanwhile, social media giant Meta Platforms is developing its MTIA (Meta Training and Inference Accelerator) processors, designed to run AI models across its family of apps, including Instagram and Facebook. ChatGPT creator OpenAI is also rolling out its first in-house chip later this year , designed in conjunction with Broadcom. Google's TPU discovery Google has been at it the longest. The company's TPU discovery came in 2013, when Google leadership produced an alarming projection that demand for computing on its products would outpace its current infrastructure. "We did some back-of-the-napkin math looking at how much compute it would take to handle hundreds of millions of people talking to Google for just three minutes a day," said Jeff Dean, Google's chief scientist, in a company blog post from 2024. "In today's framing, that seems like nothing. But at the time, we soon realized it would take basically all the compute power that Google had deployed. Put another way, we'd need to double the number of computers in Google data centers to support these new features." After realizing the market didn't offer anything that met the demand for even basic machine learning workloads to operate their products, the team began laying the groundwork for its first TPU. Google's first version of the TPU was deployed internally in 2015 and quickly became a critical component across different areas at Google. "We thought we'd maybe build under 10,000 of them," Andy Swing, principal engineer on Google's machine learning hardware systems, said in the same blog post. "We ended up building over 100,000 to support all kinds of great stuff, including Ads, Search, speech projects, AlphaGo, and even some self-driving car stuff." Before powering AI models like Gemini, the chips were used across a wide range of machine-learning applications, including Search, YouTube recommendations, and advertising systems. Today, in the company's words , TPUs "serve as the backbone for AI across nearly all of Google's products." A new milestone Since then, Google's TPUs have become more advanced and efficient across generations. The upcoming generation marks a notable milestone in their existence. Google's latest eighth-generation TPUs, announced in late April at the Google Cloud Next conference, mark the first time the company has split its chip lineup into two specialized variants for training and inference: the TPU 8t for model training and the TPU 8i for inference or the ongoing running on AI models after users submit prompts. These two TPU chips are designed to meet demanding AI workloads, including autonomous AI agents to get things done on people's behalf. The TPU system will work closely with CPUs. Google said the chips are up to three times faster for AI model training, offer 80% better performance per dollar, and can run more than 1 million TPUs in a single cluster. "This gives us the ability to create the largest training cluster in the world," said Alphabet CEO Sundar Pichai at Google's I/O developer conference last month. A training cluster is a massive network of thousands of chips that work together as a single supercomputer to train AI models. "For model builders, this means training larger, more capable models in weeks, rather than months," Pichai said. In general, TPUs are lowering Alphabet's cost of running AI while giving the tech giant more flexibility in how it prices cloud services, improving its bottom line. Pichai has also pointed to a 78% reduction in Gemini serving unit costs across 2025, powered in part by TPU efficiency gains. Circular Technology's Gastwirth said the margin benefit comes from not having to rely as heavily on Nvidia's high-priced chips. Google is not "spending 80% gross margin from Nvidia," he said, adding that its inference costs are likely among the best in the industry because the chip is built specifically for that task. That price-performance ratio has attracted some of the top AI labs in the industry. Anthropic has committed to using multiple gigawatts of Google TPUs to increase its computing resources as demand for its models and services surges. Meta Platforms also signed a multi-billion-dollar deal with Alphabet in February to use Google's TPUs. Google's TPUs are also finding customers outside Silicon Valley tech giants. "TPUs [are] becoming more general-purpose infrastructure," said Google Cloud CEO Thomas Kurian on the Future Forward podcast on April 25. Kurian noted he's seeing TPU demand beyond AI labs, into other market segments like finance, energy, and other high-performance computing customers. For example, financial firm Citadel Securities is using Google's TPUs for high-performance financial modeling , and all 17 U.S. Department of Energy national laboratories use AI co-scientist software -- an AI framework developed by Google and powered by Gemini -- built on the chips. Alphabet CFO Anat Ashkenazi said Google Cloud backlog nearly doubled sequentially to $472 billion by the end of the first quarter, driven by strong demand for enterprise AI offerings and the inclusion of TPU hardware sales for customers' own data centers. That's a departure from how customers historically have accessed TPUs -- but it now represents another growth driver for its cloud business. Analysts at Citizens last month forecasted that Google will generate about $3 billion of revenue from TPU-related infrastructure in 2026, before jumping to $25 billion in 2027. Alphabet's TPU sales revenue would be included within the Google Cloud segment. "Importantly, we believe TPU monetization is not fully reflected in current consensus estimates, indicating meaningful upside potential," analysts wrote in early May. Kurian said Google wins no matter how customers are accessing TPUs. "We make great margins no matter which way we're selling it because we own our own IP," Kurian said in a Future Forward podcast interview published in April. He also explained that since chip demand is likely to exceed supply for years, in an already capacity-constrained environment, "unit economics get more expensive, and in our case, because we control our chip, the unit economics remain attractive." That gives Alphabet the opportunity to keep monetizing an already powerful and growing business segment and opening it to more hardware sales and partnerships, such as the TPU cloud venture with asset management giant Blackstone. Blackstone is committing $5 billion in initial equity to the venture, with plans to bring 500 megawatts of capacity online by 2027 and scale from there. Google will supply the hardware, software, and infrastructure expertise. A job posting on LinkedIn is currently available for the chief operating officer of the "Blackstone and Google TPU Cloud Company." The joint venture with Blackstone is "another vote of confidence in TPUs and allows Google to increase its commitment to Cloud without the significant capital requirements," Piper Sandler wrote last month in a research note. Analysts called it a "capital-light way for Google to keep driving TPU momentum." There are still plenty of questions about Alphabet's strategy in the AI arms race. For now, the TPU isn't one of them. (Jim Cramer's Charitable Trust is long GOOGL, NVDA, ARM, INTC, META, MSFT, AMZN, and AVGO. See here for a full list of the stocks.) As a subscriber to the CNBC Investing Club with Jim Cramer, you will receive a trade alert before Jim makes a trade. Jim waits 45 minutes after sending a trade alert before buying or selling a stock in his charitable trust's portfolio. If Jim has talked about a stock on CNBC TV, he waits 72 hours after issuing the trade alert before executing the trade. THE ABOVE INVESTING CLUB INFORMATION IS SUBJECT TO OUR TERMS AND CONDITIONS AND PRIVACY POLICY , TOGETHER WITH OUR DISCLAIMER . NO FIDUCIARY OBLIGATION OR DUTY EXISTS, OR IS CREATED, BY VIRTUE OF YOUR RECEIPT OF ANY INFORMATION PROVIDED IN CONNECTION WITH THE INVESTING CLUB. NO SPECIFIC OUTCOME OR PROFIT IS GUARANTEED.

Google is spending billions to turn its TPU chips into a real challenger to Nvidia

Serving tech enthusiasts for over 25 years. TechSpot means tech analysis and advice you can trust. TL;DR: Google is pushing its in-house AI chips much more aggressively, turning years of tensor processing unit development into a direct challenge to Nvidia's hold on the AI hardware market. For years, the company built its chips mostly to handle its internal workloads. Those tensor processing units, or TPUs, sat behind products like search and speech recognition, handling some of the company's heavier AI workloads. Now, Google is trying to turn that in-house advantage into a business that can stand up to Nvidia. One clear example of that shift is in western New York at an AI data-center cluster called Lake Mariner, on Lake Ontario's southern shore near Niagara Falls. Alphabet's Google has provided a $3.2 billion financial guarantee for the project, whose developers plan to rent computing power from thousands of Google's chips to Anthropic, according to people familiar with the matter who spoke to The Wall Street Journal. The basic playbook is similar to Nvidia's: support data-center financing and then benefit when those sites buy your chips. That kind of financing has become more important as the market for AI compute has tightened. Over the past year, the AI race has become less about models and more about sheer access to computing power. "You have all these very well-capitalized companies who are big believers that this market around compute is going to have tremendous value," said Nazar Khan, co-founder and chief technology officer of TeraWulf, which is developing Lake Mariner with FluidStack, a Google-backed cloud provider. "They want to be in the game, they don't want to be left behind," Khan told The WSJ. The story behind Google's push goes back to 2013. Jeff Dean, now chief scientist at Google's DeepMind lab, recalled working on speech recognition systems built on the neural-network techniques that later evolved into today's large language models. "I said, 'OK, if we want to have this speech model that we roll out to 100 million users, and they use it a few minutes a day, that would require doubling the number of computers Google had,'" he said. "We need to build specialized hardware." That conclusion helped spur Google's TPU program, which has since produced multiple generations of the chips. Google kept those chips to itself at first, then started offering them through Google Cloud as demand for AI computing exploded. That step helped drive growth in the cloud business and set the stage for more direct competition with Nvidia. Research firm SemiAnalysis asked in a November note whether the release of Google's seventh-generation TPU - which Anthropic uses to train its models - marked "the end of Nvidia's dominance." The company's latest moves suggest it is willing to test that question. Google recently struck a $5 billion deal with Blackstone to create a new cloud-services business designed to compete with Nvidia-aligned providers such as CoreWeave and Nebius. It has also decided to sell chips directly to customers rather than only through its cloud and has rolled out its first TPU designed specifically for inference. Mark Lohmeyer, vice president of AI and computing infrastructure for Google Cloud, said the new inference chip and improvements in how TPUs work across different systems have generated new interest in using them. "We're seeing a set of customers that might not have considered it in the past," he said. Citadel Securities, a longtime Google Cloud client, recently began using TPUs for some of its research software. Josh Woods, the firm's chief technology officer, said the company can run key workloads at 30% lower cost and up to four times faster with TPUs. Nvidia, for its part, is not treating TPUs as an existential threat. The company still controls an estimated more than 90% of the AI chip market, helped by its CUDA software stack and a hardware ecosystem that many AI labs already rely on. In an April appearance on podcaster Dwarkesh Patel's show, CEO Jensen Huang said Nvidia has a much wider reach than any custom chip or ASIC. "I would love to hear them demonstrate the cost advantage of TPUs," he said. "It makes no sense in my mind." Some cloud providers worry they are locked into Nvidia's full stack, concerned that shifting spend elsewhere could cost them access to its most coveted chips. Adam Fisher, a partner at Bessemer Venture Partners, said some so-called neo-clouds fear ending up in what insiders half-jokingly call "Jensen jail." "Not all the Nvidia neo-clouds would say it this way - some would say Nvidia gives them what they need - but there are others that are dying for something else, but they can't get it from another supplier," he said. Google is trying to counter that inertia with money and focus. The company has said it plans to raise $85 billion in equity, largely to support AI infrastructure. It is backing another Anthropic-related project called River Bend, a $7 billion development near Baton Rouge, La., and is providing $1.4 billion in guarantees for an AI computing lease in Colorado City, Texas. Inside Google, the TPU business has taken on a higher profile under Amin Vahdat, who in December became chief technologist in charge of the company's AI infrastructure build-out. His portfolio now spans chip design, supply and deployment, and he reports both to Google Cloud chief Thomas Kurian and Alphabet CEO Sundar Pichai. People who have worked with him describe Vahdat as demanding but quietly competitive, and say he is pushing for steady performance gains and clearer commercial goals for Google's silicon efforts. Vahdat says he is not setting out to knock Nvidia off its pedestal. Google still runs Nvidia GPUs in its data centers, and he describes the relationship as both cooperative and competitive. "For me and for us, it's not zero-sum," he said. "There's so much demand out there." With AI workloads growing faster than any one supplier can handle, that may be the opening Google needs. If Google keeps improving its chips, lines up long-term customers such as Anthropic and Citadel, and uses its balance sheet to help build data-center capacity, TPUs could become more than an internal tool. They could be a genuine second option in a market that has mostly belonged to Nvidia.

Google Deploys Billions to Challenge NVIDIA in Expanding AI Chip Market

Google is expanding its custom artificial intelligence chip business as competition grows across the global data centre market. The company is using financial guarantees, manufacturing deals, and software tools to attract more outside customers to its Tensor Processing Units, or TPUs. Google has provided a $3.2 billion financial guarantee for the Lake Mariner data centre campus in western New York, according to reports. TeraWulf operates the site, which will host built around Google's TPU chips. The project will supply capacity to Fluidstack, which will then provide resources to Anthropic for work on its Claude AI models. The arrangement expands Google's role beyond chip design and cloud services. It also places the company in the financing structure behind the infrastructure that uses its hardware. NVIDIA has used similar support for GPU projects by helping customers secure leases and funding for large computing clusters. Google is now using financial support to raise demand for its processors.