Amazon Challenges Nvidia's AI Chip Dominance with Trainium 2

Amazon to Rival Nvidia AI Chips with 'Trainium 2'

You're reading Entrepreneur India, an international franchise of Entrepreneur Media. To reduce its reliance on chip giant Nvidia, Amazon is preparing to launch its next generation of artificial intelligence (AI) chip - Trainium 2. In Q4 2023, Nvidia held 80 per cent of the graphics processing unit (GPU) market share. "We want to be absolutely the best place to run Nvidia. But at the same time, we think it's healthy to have an alternative," Dave Brown, vice president of compute and networking services, AWS was quoted as saying. According to Financial Times, Amazon aims to lower operational costs for both Amazon and its AWS customers by using its own chips. Report states that through its cloud computing division, Amazon Web Services (AWS), the E-commerce giant is heavily investing in custom-designed chips to enhance the efficiency of its vast network of data centres. Acquired by Amazon in 2015 for USD 350 million, Israeli chip start-up Annapurna Labs is leading this innovation. The chip is expected to be unveiled next month as part of Amazon's lineup of AI chips designed for training large-scale AI models. Trainium 2 is undergoing testing with other players, including Anthropic, another Amazon backed start-up. Amazon notes that its "Inferentia" AI chips, designed for generating responses from AI models, are already 40 per cent cheaper to operate than comparable solutions. The chip development comes as AWS announces offering free computing power to researchers who utilise its custom-designed AI chips. It will provide researchers with credits worth an estimated USD 110 million to access its cloud data centres and utilise "Trainium," its specialised chip for developing AI models.

Amazon steps up effort to build AI chips that can rival Nvidia

Amazon is poised to roll out its newest artificial intelligence chips as the Big Tech group seeks returns on its multibillion-dollar semiconductor investments and reduce its reliance on market leader Nvidia. Executives at Amazon's cloud computing division are spending big on custom chips in the hopes of boosting the efficiency inside its dozens of data centres, ultimately bringing down its own costs as well as those of Amazon Web Services' customers. The effort is spearheaded by Annapurna Labs, an Austin-based chip start-up that Amazon acquired in early 2015 for $350mn. Annapurna's latest work is expected to be showcased next month when Amazon announces widespread availability of 'Trainium 2', part of a line of AI chips aimed at training the largest models. Trainium 2 is already being tested by Anthropic -- the OpenAI competitor that has secured $4bn in backing from Amazon -- as well as Databricks, Deutsche Telekom, and Japan's Ricoh and Stockmark. AWS and Annapurna's target is to take on Nvidia, one of the world's most valuable companies thanks to its dominance of the AI processor market. "We want to be absolutely the best place to run Nvidia," said Dave Brown, vice-president of compute and networking services at AWS. "But at the same time we think it's healthy to have an alternative." Amazon said 'Inferentia', another of its lines of specialist AI chips, is already 40 per cent cheaper to run for generating responses from AI models. "The price [of cloud computing] tends to be much larger when it comes to machine learning and AI," said Brown. "When you save 40 per cent of $1000, it's not really going to affect your choice. But when you are saving 40 per cent on tens of millions of dollars, it does." Amazon now expects around $75bn in capital spending in 2024, with the majority on technology infrastructure. On the company's latest earnings call, chief executive Andy Jassy said he expects the company will spend even more in 2025. This represents a surge on 2023, when it spent $48.4bn for the whole year. The biggest cloud providers, including Microsoft and Google, are all engaged in an AI spending spree that shows little sign of abating. Amazon, Microsoft and Meta are all big customers of Nvidia, but are also designing their own data centre chips to lay the foundations for what they hope will be a wave of AI growth. "Every one of the big cloud providers is feverishly moving towards a more verticalised and, if possible, homogenised and integrated [chip technology] stack," said Daniel Newman at The Futurum Group. "Everybody from OpenAI to Apple is looking to build their own chips," noted Newman, as they seek "lower production cost, higher margins, greater availability, and more control". "It's not [just] about the chip, it's about the full system," said Rami Sinno, Annapurna's director of engineering and a veteran of SoftBank's Arm and Intel. For Amazon's AI infrastructure, that means building everything from the ground up, from the silicon wafer to the server racks they fit into, all of it underpinned by Amazon's proprietary software and architecture. "It's really hard to do what we do at scale. Not too many companies can," said Sinno. After starting out building a security chip for AWS called Nitro, Annapurna has since developed several generations of Graviton, its Arm-based central processing units that provide a low-power alternative to the traditional server workhorses provided by Intel or AMD. "The big advantage to AWS is their chips can use less power, and their data centres can perhaps be a little more efficient," driving down costs, said G Dan Hutcheson, analyst at TechInsights. If Nvidia's graphics processing units are powerful general purpose tools -- in automotive terms, like a station wagon or estate car -- Amazon can optimise its chips for specific tasks and services, like a compact or hatchback, he said. So far, however, AWS and Annapurna have barely dented Nvidia's dominance in AI infrastructure. Nvidia logged $26.3bn in revenue for AI data centre chip sales in its second fiscal quarter of 2024. That figure is the same as Amazon announced for its entire AWS division in its own second fiscal quarter -- only a relatively small fraction of which can be attributed to customers running AI workloads on Annapurna's infrastructure, according to Hutcheson. As for the raw performance of AWS chips compared with Nvidia's, Amazon avoids making direct comparisons, and does not submit its chips for independent performance benchmarks. "Benchmarks are good for that initial: 'hey, should I even consider this chip,'" said Patrick Moorhead, a chip consultant at Moor Insights & Strategy, but the real test is when they are put "in multiple racks put together as a fleet". Moorhead said he is confident Amazon's claims of a 4 times performance increase between Trainium 1 and Trainium 2 are accurate, having scrutinised the company for years. But the performance figures may matter less than simply offering customers more choice. "People appreciate all of the innovation that Nvidia brought, but nobody is comfortable with Nvidia having 90 per cent market share," he added. "This can't last for long."

AWS to Soon Roll Out Trainium 2 Chip to Scale Anthropic's Claude

OpenAI partners with TSMC and Broadcom to launch the first in-house AI chip in 2026. Amazon is planning to roll out its latest AI chip, Trainium 2, in the coming month, most likely to support targeting AI model training at scale, the Financial Times reported. Already tested by partners such as Anthropic, Databricks, and Deutsche Telekom, Trainium 2 is part of Amazon's larger strategy to optimise its data centre performance while reducing costs for Amazon Web Services (AWS) customers. "The price of cloud computing tends to be much larger for machine learning and AI," explained AWS vice president Dave Brown, adding that savings of 40% on large AI workloads can significantly impact customer choices. AWS chief Andy Jassy said, "The second version of Trainium, Trainium 2, will start to ramp up in the next few weeks, and I think it's going to be very compelling for customers on a price-performance basis." Read: GenAI Boom Bleeds Users, Fills AWS, Azure, GCP's Coffers The report further said that Amazon's other AI chip, Inferentia, is reported to save customers approximately 40% on costs for generating responses from AI models. Recently, reports surfaced that AWS plans to invest more in the AI startup Anthropic. However, the company has set one condition, requiring Anthropic to use a large number of servers powered by AI chips developed in-house by Amazon. Meanwhile, NVIDIA is building next level of computing capabilities for its customers, including OpenAI and others, where they are looking to build next level of test-time computing chips for o1-like models. In a recent podcast with No Priors, NVIDIA chief Jensen Huang shared that one of the major challenges NVIDIA is currently facing in computing is inference time scaling, which involves generating tokens at incredibly low latency. Huang explained that, in the future, AI systems will need to perform tasks like tree search, chain of thought, and mental simulations, reflecting on their own answers. The model would prompt itself and generate text internally, all while responding in real-time, ideally within a second. This approach subtly points to the capabilities of the o1 system. Meanwhile, OpenAI plans to partner with TSMC and Broadcom to launch its first in-house AI chip by 2026. This move comes after the startup began exploring a new method to scale up its models, particularly o1, using the test-time compute approach. AWS recently announced a $110 million investment to support university-led research in generative AI through its new "Build on Trainium" program. This initiative provides compute hours and AWS Trainium credits, giving researchers access to Trainium UltraClusters for large-scale AI research, covering topics from AI architecture to machine learning (ML) library development. The Build on Trainium program aims to advance AI research by offering access to up to 40,000 Trainium chips, facilitating work on distributed systems, algorithmic improvements, and AI accelerator performance. AWS developed Trainium as a specialised chip for deep learning and inference, enabling high-performance AI experiments previously limited by budget constraints. As part of the program, AWS will conduct rounds of Amazon Research Awards calls for proposals. Selected institutions receive AWS Trainium credits and access to resources for exploring innovations in AI. Participants include prominent research institutions, such as Carnegie Mellon University (CMU) and the University of California at Berkeley, focusing on ML systems and compiler optimisations. Build on Trainium also offers training and resources to grant recipients. AWS provides technical education and connects researchers with the Neuron Data Science community, fostering knowledge sharing among AWS specialists, startups, and the Generative AI Innovation Center.

Amazon ready to use its own AI chips, reduce its dependence on Nvidia

Amazon is poised to roll out its newest artificial intelligence chips as the Big Tech group seeks returns on its multibillion-dollar semiconductor investments and reduce its reliance on market leader Nvidia. Executives at Amazon's cloud computing division are spending big on custom chips in the hopes of boosting the efficiency inside its dozens of data centers, ultimately bringing down its own costs as well as those of Amazon Web Services' customers. The effort is spearheaded by Annapurna Labs, an Austin-based chip start-up that Amazon acquired in early 2015 for $350 million. Annapurna's latest work is expected to be showcased next month when Amazon announces widespread availability of 'Trainium 2', part of a line of AI chips aimed at training the largest models. Trainium 2 is already being tested by Anthropic -- the OpenAI competitor that has secured $4 billion in backing from Amazon -- as well as Databricks, Deutsche Telekom, and Japan's Ricoh and Stockmark. AWS and Annapurna's target is to take on Nvidia, one of the world's most valuable companies thanks to its dominance of the AI processor market. "We want to be absolutely the best place to run Nvidia," said Dave Brown, vice-president of compute and networking services at AWS. "But at the same time we think it's healthy to have an alternative." Amazon said 'Inferentia', another of its lines of specialist AI chips, is already 40 percent cheaper to run for generating responses from AI models. "The price [of cloud computing] tends to be much larger when it comes to machine learning and AI," said Brown. "When you save 40 percent of $1000, it's not really going to affect your choice. But when you are saving 40 percent on tens of millions of dollars, it does."

Top Tech News: Amazon's Trainium 2 Chip Could Challenge Nvidia's AI Dominance; Razorpay Launches Fund to Back Early-Stage B2B Startups

Amazon is set to unveil Trainium 2, its next-gen AI chip developed by Annapurna Labs, aiming to reduce dependence on Nvidia and cut operational costs for Amazon and AWS clients. Expected next month, the chip is already being tested by firms like Anthropic. AWS custom chip initiative seeks to boost data center efficiency and provide a viable alternative to Nvidia's AI processors. To drive adoption, AWS will offer $110 million in credits to researchers using its cloud infrastructure and custom AI chips, aiming to demonstrate Trainium's capabilities and support broader AI development within the research community.

Amazon is taking on Nvidia with an AI freebie

How AI is helping advertisers pinpoint exactly who you are and what you want The cloud computing unit of Amazon said on Tuesday that researchers who want to use its custom AI chip, Trainium, can get credits worth $110 million to use its cloud data centers as part of its Build on Trainium program. The company's Trainium chip is used for training AI models, and is an in-house competitor to AI chips from Nvidia and Advanced Micro Devices (AMD-2.57%). Amazon is making up to 40,000 first-generation Trainium chips available for program, which is focused on university-led research in generative AI, and counts researchers from Carnegie Mellon University and the University of California, Berkeley. Gadi Hutt, business development lead for AI chips at AWS, told Reuters that the company is taking a different strategy from Nvidia to get developers to use its chips, and is planning to publish Trainium's instruction set architecture so customers can program the chip directly, instead of with software, which is what Nvidia's chips need. But despite preparations to roll out the next-generation of its in-house AI chips, Amazon "is going nowhere," until Nvidia's CUDA software "is no longer the control point of AI training," and the cloud giant "iterates much faster," Richard Windsor, founder of research firm Radio Free Mobile, said in a note. Amazon will likely launch its in-house AI inferencing chip, Inferentia, at its re:Invent conference later this year, Windsor said, but he doubts "it will make much difference" to Nvidia's dominance. "We want to be absolutely the best place to run Nvidia," Dave Brown, vice president of compute and networking services at AWS, told the Financial Times. "But at the same time we think it's healthy to have an alternative." Windsor said Brown's statement "is indicative of Nvidia's market power, and all of its biggest customers are trying to reduce their dependence on it for training and inference and so far, they are having very little success." Meanwhile, Amazon is discussing another multi-billion dollar investment in AI startup, Anthropic, The Information reported. Amazon is reportedly asking Anthropic, which develops AI models rivaling those of OpenAI, to use servers powered by its in-house custom chips.

Amazon Developing Custom AI Processors To Compete With NVIDIA

This is not investment advice. The author has no position in any of the stocks mentioned. Wccftech.com has a disclosure and ethics policy. According to a report from the Financial Times, Amazon is developing custom artificial intelligence chips to reduce its dependence on NVIDIA. The firm has already developed a variety of in-house processors to run data center workloads, and the latest push is part of its investment in a chip design startup in 2015. Amazon is expected to shed more light on its custom AI processors next month as part of announcements covering the firm's Trainium chip lineup. These chips have been developed by Amazon's Annapurna Labs, and they are being used by Microsoft-backed OpenAI's rival Anthropic. Anthropic is Amazon's primary AI partner, and it provides the e-commerce and cloud computing giant with access to the Claude foundational AI model. Today's report is just one of many that indicate a push in big tech to reduce reliance on NVIDIA for the most powerful artificial intelligence processors. NVIDIA's GPUs are market leaders and top performers in running AI workloads. Consequently, high demand and constrained supply have made them one of the most highly sought-after and expensive products in the world. For Amazon, developing in-house AI chips is an effort to reduce dependence on NVIDIA's products and simultaneously reduce costs, reports the Financial Times. The firm is not inexperienced when it comes to developing custom chips. Its acquisition of chip design startup Annapurna has enabled Amazon to churn out a steady stream of processors to reduce costs of using AMD and Intel's products for traditional data center workloads. These chips, called the Graviton processors, are complemented by Amazon's custom AI processors called Trainium. Trainium is designed to work with large language models, and Amazon unveiled Trainium2 a year ago in November 2023. As per the FT, Annapurna is also leading the effort to develop the chips that will reduce Amazon's dependence on NVIDIA's GPUs. While the FT's report shares few details about these chips, it does outline that Amazon can provide insights into them next month at its event covering the Trainium2 chips. While Trainium2 was launched in 2023, supply constraints have limited its adoption. FT reports that Amazon's AI partner, Anthropic, is using Trainium2. Amazon's chips are designed using technology from the Taiwanese firm Alchip. They are manufactured by the Taiwan Semiconductor Manufacturing Company (TSMC), and Amazon shared last year that more than 50,000 AWS customers were using its Graviton chips. Along with Amazon, other mega-cap firms, including Google's parent Alphabet and Facebook's owner Meta, also self-develop AI chips. Industry players like Apple use Google's chips, and Meta unveiled its second-generation Meta Training and Inference Accelerator (MTIA) earlier this year. Both of these reduce dependence on NVIDIA's GPUs, and Microsoft-backed OpenAI is also reportedly considering developing in-house chips. Google unveiled its latest tensor processing unit (TPU) AI chip, Trillium, earlier this month. These chips offer four times faster AI training performance and three times faster inference than their predecessors, according to the company.

Amazon Set To Introduce New AI Chips In December To Rival Jensen Huang's AI Stalwart: 'We Want To Be Absolutely The Best Place To Run Nvidia' - NVIDIA (NASDAQ:NVDA), Amazon.com (NASDAQ:AMZN)

Amazon.com Inc. AMZN is preparing to introduce its newest artificial intelligence chips in December. His strategic move aims to leverage Amazon's significant investments in semiconductors and reduce reliance on Jensen Huang's Nvidia Corp. NVDA. What Happened: Amazon's cloud division is focusing on custom chip development to boost data center efficiency and cut costs for itself and its Amazon Web Services (AWS) clients. The initiative is spearheaded by Annapurna Labs, a chip start-up acquired by Amazon in 2015 for $350 million. The upcoming "Trainium 2" AI chips are designed for training large models and are already being tested by companies like Anthropic, Databricks, and Deutsche Telekom, the Financial Times reported on Tuesday. See Also: Cryptocurrencies Rally Following Trump Victory, El Salvador's Bitcoin Bet Pays Off And More: Top Crypto U Dave Brown, AWS's VP of compute and networking services, highlighted the significance of offering an alternative to Nvidia's dominance in the AI processor market. "We want to be absolutely the best place to run Nvidia," he said. "But at the same time we think it's healthy to have an alternative." Amazon's capital spending is expected to reach $75 billion in 2024, with a large portion dedicated to tech infrastructure, up from $48.4 billion in 2023, reflecting the AI investment trend among major cloud providers. Why It Matters: Amazon's move to launch new AI chips is part of a broader strategy to strengthen its position in the AI market. The company is reportedly considering a second multi-billion-dollar investment in AI startup Anthropic, which utilizes Amazon's cloud services for training. This follows an initial $4 billion investment last year, with Amazon encouraging Anthropic to use its chips. In October, Amazon signed a five-year deal with Databricks to provide cost-effective AI-building capabilities, positioning its Trainium AI chips as a cheaper alternative to Nvidia's GPUs. Read Next: Lucid CEO Scrambles For Damage Control As Shares Plunge 47% This Year: 'As A Major Shareholder...Believe Me, Nobody Is More Incentivized Than Me For Success' Image via Shutterstock This story was generated using Benzinga Neuro and edited by Pooja Rajkumari Market News and Data brought to you by Benzinga APIs

Amazon wants chips with everything, crafting its own AI-focused line of silicon in a bid to dethrone Nvidia

When I think 'Nvidia,' I think 'games,' but that's far from the full picture. If we're talking just cold hard cash, then AI is becoming an increasingly massive source of revenue for the company, bringing in $30 billion during Q2 and far over-shadowing the share of revenue brought in by games, totalling $2.88 billion during the same period. Yeah, okay, neither of those figures is small potatoes; Nvidia is a heavy-hitter in this arena for sure -- but there's about to be a rumble in the AI jungle. Amazon has not been shy about its commitment to AI (and we're not just talking about their deal-spitting robot Rufus). Besides investing in a nuclear future to power their electric dreams, the company is also eyeing up the throne when it comes to AI chips (via Financial Times). In a move geared towards reducing their reliance on Nvidia, Amazon is looking to make good on the millions they've already spent in semiconductor investments and pool all of their chips towards, well, making their own. The hope is that Amazon-made chips will boost the efficiency of Amazon-owned data centres, thereby bringing down the cost of running them. That's good news for the company's own pockets, but customers of Amazon Web services may reap the benefits too. So, where is Amazon flinging money specifically? In two words: Annapurna Labs, a start-up Amazon already scooped way back in 2015 for $350 million. Together they've been noodling away on the 'Trainium 2' AI chip for at least the last year, and it looks like the wait to get my grubby little, deeply AI-sceptical mitts on them is not long now. The abysmal 'Trainium' name is about the fact these chips are designed to train the latest AI models, and as such are already being put through their paces by Anthropic. A competitor to OpenAI, Anthropic is another startup that's enjoyed generous financial backing from Amazon (to the tune of $4 billion) among other investors. Amazon is also cooking up another line of AI chips named 'Inferentia,' which the company claims is already proving to be 40% more cost-effective when it comes to generating AI responses. Meaning 'inference' in Latin, it's definitely a better name than 'Trainium', but I do hope they don't follow it up with chip lines called Suspiria, Tenebrae, or Lacrimosa (though an AI-themed return from horror director Dario Argento could be interesting). Giallo daydreams aside, Amazon's latest AI bid is not unique. Both Microsoft and Meta also want to dethrone Nvidia by making their own chips to better meet the demands of AI in their respective businesses. That's already making for big motions in tech's ocean, but other major players are likely to follow with a big splash. As I've already said, AI has been a huge money-maker for Nvidia so it's no surprise lots of other companies want to muscle in on that market. These companies are likely hoping for continued growth in the sector, but perhaps that's blue-sky thinking. OpenAI's co-founder reckons large language model learning is approaching a plateau, so perhaps that means this bubble will pop sooner than we think.