Google Demands Exponential AI Infrastructure Growth as Tech Giants Race to Meet Surging Demand

Google tells employees it must double capacity every 6 months to meet AI demand

While AI bubble talk fills the air these days, with fears of overinvestment that could pop at any time, something of a contradiction is brewing on the ground: Companies like Google and OpenAI can barely build infrastructure fast enough to fill their AI needs. During an all-hands meeting earlier this month, Google's AI infrastructure head Amin Vahdat told employees that the company must double its serving capacity every six months to meet demand for artificial intelligence services, reports CNBC. Vahdat, a vice president at Google Cloud, presented slides showing the company needs to scale "the next 1000x in 4-5 years." While a thousandfold increase in compute capacity sounds ambitious by itself, Vahdat noted some key constraints: Google needs to be able to deliver this increase in capability, compute, and storage networking "for essentially the same cost and increasingly, the same power, the same energy level," he told employees during the meeting. "It won't be easy but through collaboration and co-design, we're going to get there." It's unclear how much of this "demand" Google mentioned represents organic user interest in AI capabilities versus the company integrating AI features into existing services like Search, Gmail, and Workspace. But whether users are using the features voluntarily or not, Google isn't the only tech company struggling to keep up with a growing user base of customers using AI services. Major tech companies are in a race to build out data centers. Google competitor OpenAI is planning to build six massive data centers across the US through its Stargate partnership project with SoftBank and Oracle, committing over $400 billion in the next three years to reach nearly 7 gigawatts of capacity. The company faces similar constraints serving its 800 million weekly ChatGPT users, with even paid subscribers regularly hitting usage limits for features like video synthesis and simulated reasoning models. "The competition in AI infrastructure is the most critical and also the most expensive part of the AI race," Vahdat said at the meeting, according to CNBC's viewing of the presentation. The infrastructure executive explained that Google's challenge goes beyond simply outspending competitors. "We're going to spend a lot," he said, but noted the real objective is building infrastructure that is "more reliable, more performant and more scalable than what's available anywhere else."

Google must double AI compute every 6 months to meet demand, AI infrastructure boss tells employees

Amin Vahdat, VP of Machine Learning, Systems and Cloud AI at Google, holds up TPU Version 4 at Google headquarters in Mountain View, California, on July 23, 2024. Google 's AI infrastructure boss told employees that the company has to double its compute capacity every six months in order to meet demand for artificial intelligence services. At an all-hands meeting on Nov. 6, Amin Vahdat, a vice president at Google Cloud, gave a presentation, viewed by CNBC, titled "AI Infrastructure," which included a slide on "AI compute demand." The slide said, "Now we must double every 6 months.... the next 1000x in 4-5 years." "The competition in AI infrastructure is the most critical and also the most expensive part of the AI race," Vahdat said at the meeting, where Alphabet CEO Sundar Pichai and CFO Anat Ashkenazi also took questions from employees. The presentation was delivered a week after Alphabet reported better-than-expected third-quarter results and raised its capital expenditures forecast for the second time this year, to a range of $91 billion to $93 billion, followed by a "significant increase" in 2026. Hyperscaler peers Microsoft, Amazon and Meta also boosted their capex guidance, and the four companies now expect to collectively spend more than $380 billion this year. Google's "job is of course to build this infrastructure but it's not to outspend the competition, necessarily," Vahdat said. "We're going to spend a lot," he said, adding that the real goal is to provide infrastructure that is far "more reliable, more performant and more scalable than what's available anywhere else." In addition to infrastructure buildouts, Vahdat said Google bolsters capacity with more efficient models and through its custom silicon. Last week, Google announced the public launch of its seventh generation Tensor Processing Unit called Ironwood, which the company says is nearly 30 times more power efficient than its first Cloud TPU from 2018. Vahdat said the company has a big advantage with DeepMind, which has research on what AI models can look like in future years. Google needs to "be able to deliver 1,000 times more capability, compute, storage networking for essentially the same cost and increasingly, the same power, the same energy level," Vahdat said. "It won't be easy but through collaboration and co-design, we're going to get there."

Google Exec Claims Company Needs to Double Its AI Serving Capacity 'Every Six Months': Report

Tech companies are racing to build out their infrastructure as their increasingly resource-intensive AI products gobble up capacity, clean out chipmakers' supply, and require more power. Google, once dubbed the "King of the Web," is one of those companies, and a high-level exec for The Big G is reported to have told staff that the company needs to scale up its serving capabilities exponentially if it wishes to keep up with the demand for its AI services. CNBC got its hands on a recent presentation given by Amin Vahdat, VP of Machine Learning, Systems, and Cloud AI at Google. The presentation includes a slide on “AI compute demand†that asserts that Google "must double every 6 months.... the next 1000x in 4-5 years.†“The competition in AI infrastructure is the most critical and also the most expensive part of the AI race,†Vahdat reportedly said at the all-hands meeting where the presentation took place. Google's “job is of course to build this infrastructure, but it’s not to outspend the competition, necessarily,†he added. “We’re going to spend a lot,†he said, in an effort to create AI infrastructure that is “more reliable, more performant and more scalable than what’s available anywhere else.†Since CNBC's story was published, Google has quibbled with the reporting. While CNBC originally quoted Vahdat as saying that the company would need to "double" its compute capacity every six months, a Google spokesperson told Gizmodo that the executive's words were taken out of context. The spokesperson further explained that Vahdat "was not talking about a capital buildout of anything approaching the magnitude suggested. In reality, he simply noted that demand for AI services means we are being asked to provide significantly more computing capacity, which we are driving through efficiency across hardware, software, and model optimizations, in addition to new investments." CNBC has since updated its reporting from "compute" to "serving" capacity. Serve capacity would refer to Google's ability to handle a rising tide of user requests, while compute capacity woud refer to the company's overall infrastructure dedicated to AI, including what is needed to train new models and other expenditures. When asked for further clarification about the difference between the two, the spokesperson said that the original headline "read as if he was implying that we are doubling the amount of compute we have -- either measured by the # of chips we operate or the amount of MW of electricity." Instead, "the capacity increases Amin described will be reached in a number of ways, including new more capable chips and model efficiency and optimization," they added. Whatever's happening under the hood, it would appear that Googleâ€"like its competitorsâ€"needs to scale up its operations to support its nascent AI infrastructure business. Vahdat's comments come not long after the tech giant reported some chunky profits from its Cloud business, with the company announcing it plans to ramp up spending in the coming year. During his presentation, Vahdat also reportedly claimed that Google needs to “be able to deliver 1,000 times more capability, compute, storage networking [than its competitors] for essentially the same cost and increasingly, the same power, the same energy level." He admitted that it "won’t be easy" but said that "through collaboration and co-design, we’re going to get there.†The race to build data centersâ€"or "AI infrastructure" as the tech industry calls itâ€"is getting crazy. Like Google, Microsoft, Amazon, and Meta all claim they are going to ramp up their capital expenditures in an effort to build out the future of computing (cumulatively, Big Tech is expected to spend at least $400 billion in the next twelve months). As these facilities go up, they are causing all sorts of drama in the communities where they reside. Environmental and economic concerns abound. Some communities have begun to protest data center projectsâ€"and, in some cases, they're successfully repelling them. Still, given the sheer amount of money invested in this industry, it will be an ongoing fight for Americans who don't want the AI colossus in their backyards.