Apple Embraces Amazon's AI Chips for Intelligence Model Training and Search Efficiency

Apple Intelligence Models Will Now Reportedly Be Trained Using Amazon's Custom Chips, Allowing For A 40 Percent Improvement In Search Efficiency

Amazon Web Services chips will reportedly be used to train Apple Intelligence models, which is not something out of the ordinary because technology giants are known for sourcing such hardware. Of course, it will be the Cupertino firm's responsibility to ensure that its services will be used with a layer of privacy, but it appears that both entities will work together to ensure a singular goal. During the annual AWS Reinvent conference held on Tuesday, CNBC reports that Benoit Dupin, who is Apple's Senior Director of machine learning and AI, discussed how the company plans to use the cloud service, stating that the firm has a strong relationship with Amazon, and its infrastructure is reliable and able to serve users worldwide. Apple has taken advantage of AWS for more than a decade now, which allows it to present services such as Apple Maps, Siri, Apple Music, and more. As for why Amazon's custom chips would be used to train Apple Intelligence models, the company executive mentioned that the partnership has led to a 40 percent efficiency improvement in searches. Dupin has also said that Apple is evaluating Amazon's Trainium2 chip for potential use in pre-training its AI models. On top of the features already offered by Apple Intelligence, more additions will arrive. Dupin also mentioned that during the early stages of using Trainium2, Apple expects early figures to display up to a 50 percent efficiency improvement. Efficiency improvements will mean it will cost less to train Apple Intelligence models. These savings can also mean that the California-based titan can train additional models for the same cost. Naturally, Apple's 'marketing cry' regarding privacy will be questioned thanks to its alliance with Amazon, but it is likely that both companies will have some countermeasures prepared. Apple's use of Amazon's custom chips could also encourage other companies to reduce dependency on NVIDIA and look for more affordable alternatives. Apple's strength lies in letting its iPhones, iPads, or Macs perform the on-device processing as much as possible, then offload the more complicated queries to its services that rely on its own M-series of chipsets. This approach is vastly different from others, which take advantage of massive clusters of servers stacked with NVIDIA GPUs.

Apple Intelligence is Nothing without AWS

Apple is also exploring AWS's Trainium2 chips, with early evaluations suggesting a 50% improvement in pre-training efficiency. Apple surprised everyone with its presence at AWS re:Invent 2024. During his keynote, AWS chief Matt Garman invited Benoit Dupin, Apple's senior director of machine learning and AI, on stage to speak about how the company works with Amazon Web Services (AWS) and uses its servers to power its AI and machine learning features. Dupin said that the partnership with AWS, which spans more than a decade, has been crucial in scaling Apple's machine learning (ML) and artificial intelligence (AI) capabilities. Dupin, who oversees machine learning, AI, and search infrastructure at Apple, detailed how the company's AI-driven features, including Siri, iCloud Music, and Apple TV, rely heavily on AWS's infrastructure. "AWS has consistently supported our dynamic needs at scale and globally," Dupin said. Apple has increasingly leveraged AWS's solutions, including its Graviton and Inferentia chips, to boost efficiency and performance. Dupin revealed that Apple achieved a 40% efficiency gain by migrating from x86 to Graviton instances. Additionally, transitioning to Inferentia 2 for specific search-related tasks enabled the company to execute features twice as efficiently. This year, Apple launched Apple Intelligence, which integrates AI-driven features across iPhone, iPad, and Mac. "Apple Intelligence is powered by our own large language models, diffusion models, and adapts on both devices and servers," Dupin said. Key features include system-wide writing tools, notification summaries, and improvements to Siri, all developed with a focus on user privacy. To support this innovation, Apple required scalable infrastructure for model training and deployment. Dupin said, "AWS services have been instrumental across virtually all phases of our AI and ML lifecycle," including fine-tuning models and building adapters for deployment. Apple is also exploring AWS's Trainium2 chips, with early evaluations suggesting a 50% improvement in pre-training efficiency. "AWS expertise, guidance, and services have been critical in supporting our scale and growth," Dupin said. Previously, Apple revealed that it uses Google's Tensor Processing Units (TPUs) instead of the industry-standard NVIDIA GPUs for training its AI models. This information was disclosed in a technical paper published by Apple on Monday, outlining the company's approach to developing its AI capabilities. At AWS re:Invent 2024, Amazon Web Services (AWS) has announced the general availability of AWS Trainium2-powered Amazon Elastic Compute Cloud (EC2) instances. The new instances offer 30-40% better price performance than the previous generation of GPU-based EC2 instances.

Apple says it uses Amazon's custom AI chips

Amazon CEO, Andy Jassy speaking with CNBC's Jim Cramer on Mad Money in Seattle, WA. on Dec. 6th, 2023. Apple is currently using Amazon Web Services' custom artificial intelligence chips for services like search and will evaluate if the company's latest AI chip can be used to pre-train its models like Apple Intelligence. Apple revealed its usage of Amazon's proprietary chips at the annual AWS Reinvent conference on Tuesday. Benoit Dupin, Apple's senior director of machine learning and AI, took the stage to discuss how Apple uses the cloud service. It's a rare example of the company officially allowing a supplier to tout them as a customer. "We have a strong relationship, and the infrastructure is both reliable, definite and able to serve our customers worldwide," Apple's Dupin said. Apple's appearance at Amazon's conference and its embrace of the company's chips is a strong endorsement of the cloud service as it vies with Microsoft Azure and Google Cloud for AI spending. Apple uses those cloud services, too. Benoit said that Apple had used AWS for over a decade for services including Siri, Apple Maps and Apple Music. Apple has used Amazon's Trainium and Graviton chips to serve search services, for example, and Benoit said that Amazon's chips had led to a 40% efficiency gain. But Benoit also suggested that Apple would use Amazon's Trainium2 chip to pre-train its proprietary models. It's a sign that Amazon's chips aren't just a cost-effective way to inference AI models compared to x86 central processors made by Intel and AMD, but can also be used to develop new AI. Amazon announced on Tuesday that its Trainium2 chip was generally available to rent. "In early stages of evaluating Trainium2 we expect early numbers up to 50% improvement in efficiency with pre-training," Dupin said. Earlier this year, Apple said in a research paper that it had used Google Cloud's TPU chips to train its iPhone AI service, which it calls Apple Intelligence. The majority of AI training is done on pricey Nvidia graphics processors. Cloud providers and startups are racing to develop alternatives to lower costs and are exploring different approaches that could lead to more efficient processing. Apple's usage of custom chips could signal to other companies that non-Nvidia training approaches can work. AWS is expected to announce new details on Tuesday about offering Nvidia Blackwell-based AI servers for rent, too. Apple released its first major generative AI product this fall. Apple Intelligence is a series of services that can summarize notifications, rewrite emails, and generate new emojis. Later this month, it will integrate with OpenAI's ChatGPT, the company says, and next year, Siri will get new abilities to control apps and speak naturally. Unlike leading chatbots like OpenAI's ChatGPT, Apple's approach to AI isn't based on large clusters of Nvidia-based servers in the cloud. Instead, Apple uses an iPhone, iPad or Mac chip to do as much of the processing as possible, and then sends complicated queries to Apple-operated servers using its own M-series chips.

Apple turns to Amazon chips for AI pre-training and more - 9to5Mac

As reported by CNBC, Apple's senior director of machine learning and artificial intelligence, Benoit Dupin, made a surprise appearance at Amazon's AWS re:Invent conference in Las Vegas today. Dupin used the opportunity to explain that Apple uses custom artificial intelligence chips from Amazon Web Services for many of its cloud services. Apple is also evaluating using Amazon's newest AI chip to pre-train its Apple Intelligence models. While speaking at the event, Dubin touted that Apple has used AWS chips, like the Graviton and Inferntia, for over a decade to help power Siri and search, the App Store, Apple Music, Apple Maps, and more. In doing this, Apple realized a 40% gain in efficiency compared to x86 chips from Intel and AMD. "We have a strong relationship, and the infrastructure is both reliable, definite and able to serve our customers worldwide," Dubin explained. Before joining Apple in 2014, Dubin served as the vice president of Amaozn's A9 Search Technology group. Dubin also confirmed that Apple is currently evaluating the latest AWS AI training chip, dubbed the Trainium2. During his appearance, Dubin said that Apple expects "up to 50% improvement in efficiency with pre-training" with the Trainium2 chip." Apple's use of the Trainium2 would be tied to pre-training for artificial intelligence models. The chips would not be used for Apple Intelligence features. Apple Intelligence features are powered on-device or in the cloud using Apple's Private Cloud Compute platform. Apple's Private Cloud Compute infrastructure is built on its own Apple Silicon chips. Earlier this year, Apple also confirmed in a research document that it used Google Tensor chips to train artificial intelligence models. At the time, we noted that it was notable for Apple to opt for Google's Tensor chips instead of the Nvidia chips that other companies tend to rely on.

Apple considers using Amazon chips to train future Apple Intelligence models

Apple Intelligence features may get trained with Amazon AI chips Apple is using chips sourced from Amazon Web Services to handle searches, and it may also use them to pre-train Apple Intelligence AI models. It is common knowledge that large companies like Apple rely on external service providers when it comes to offering certain services. However, while it is expected for Apple to do so for tasks that don't have a level of privacy and security, it turns out that it is doing so for some of its machine learning features. Revealed during the annual AWS Reinvent conference on Tuesday, Apple confirmed it is using Amazon's custom artificial intelligence chips, reports CNBC. Apple senior director of machine learning and AI Benoit Dupin appeared on stage to talk about Apple's use of Amazon's cloud services, including how it does so. Apple has used AWS for more than a decade, including for Siri, Apple Maps, and Apple Music. Amazon's Trainium and Graviton chips have also been used by Apple to handle consumer search queries, Dupin confirmed. So far, it has been beneficial for Apple, since the use of the chips made searches 40% more efficient. While the use of AWS and Amazon's chips for search is a boon for Amazon, Dupin indicated that there could be more benefits to Apple on the way. Speaking on stage, Benoit said Apple was evaluating Amazon's Trainium2 chip, for potential use to pretrain its models. This could include new models for Apple Intelligence features that add new elements or improve what's already offered to consumers. So far, it seems that Apple is approving of the chips. "In the early stages of evaluating Trainium2, we expect early numbers up to 50% improvement in efficiency with pretraining," Dupin told the audience. The sizable efficiency improvement offers a very real benefit to Apple using Amazon's chip, which the retailer said is available to rent via AWS. Efficiency gains could lead to lower costs in adequately pretraining models, or to perform more training with the models for the same cost. The use of Amazon's chips may alarm some users who are familiar with Apple's privacy-focused approach. Apple usually performs on-device processing using its own chips, which it also employs for cloud-based tasks via Private Cloud Compute. However, the AWS announcement by Apple doesn't actually affect Apple's processing practices at all. It's about training the models, not processing queries. Before a model is deployed for use by customers, the model needs to be trained beforehand. This is a processor-intensive and resource-consuming task, which sets the model to be used in specific ways, and with specific intended results. This training can be performed in many ways, such as by buying high-performance servers with multiple GPUs that are particularly adept at AI-based number crunching tasks. However, this can get expensive, and for a process that doesn't touch user data at all, it doesn't necessarily need to be performed in-house. Where the announcement could matter is, unexpectedly, Google. In July, an AI research paper confirmed that Apple had used Google-designed hardware to build the Apple Foundation Model. While it is unclear if Apple rented server time or bought hardware from Google that ran within Apple's data centers, it did ultimately mean Apple's model was trained with Google's hardware. Apple's interest in Amazon's chips could lead to a similar situation of either server rental or hardware purchases, all in a bid for efficiency. It ultimately doesn't matter to consumers what Apple uses to train the model itself. What does is that Apple's hardware is still being used to answer queries and to perform the actual processing required for Apple Intelligence to exist.

Apple Uses Amazon's Custom AI Chips for Search Services

Apple uses custom Trainium and Graviton artificial intelligence chips from Amazon Web Services for search services, Apple machine learning and AI director Benoit Dupin said today at the AWS re:Invent conference (via CNBC). Dupin said that Amazon's AI chips are "reliable, definite, and able to serve [Apple] customers worldwide." AWS and Amazon have a "strong relationship," and Apple plans to test whether Amazon's Trainium2 chip can be used for pre-training Apple Intelligence and other AI models. Amazon announced rental opportunities for the Trainium2 chip today. Apple has used AWS for more than 10 years for Siri, Apple Maps, and Apple Music. With Amazon's Trainium and Graviton chips, Apple has seen a 40 percent efficiency gain, and with Trainium2, Dupin said Apple expects up to a 50 percent improvement in efficiency with pre-training. Nvidia is the market leader when it comes to GPUs for AI training, but companies like Amazon are aiming to compete with lower-cost options.

Amazon's cloud service shows new AI servers, says Apple will use its chips

LAS VEGAS (Reuters) - Amazon.com's cloud unit on Tuesday showed new data center servers packed with its own AI chips that will challenge Nvidia, with Apple coming aboard as a customer to use them. The new servers, based on 64 of Amazon Web Services' Trainium2 chips, will be strung together in a massive supercomputer with hundreds of thousands of chips, with the help AI startup Anthropic, which will be the first to use it. Apple executive Benoit Dupin also said that Apple is using Trainium2 chips. AWS Chief Executive Matt Garman also said that Trainium3, the company's next generation of AI chip, will debut next year. (Reporting by Stephen Nellis in San Francisco and Greg Bengisnger in Las Vegas)

Amazon's cloud service shows new AI servers, says Apple will use its chips

LAS VEGAS, Dec 3 (Reuters) - Amazon.com's (AMZN.O), opens new tab cloud unit on Tuesday showed new data center servers packed with its own AI chips that will challenge Nvidia (NVDA.O), opens new tab, with Apple coming aboard as a customer to use them. The new servers, based on 64 of Amazon Web Services' Trainium2 chips, will be strung together in a massive supercomputer with hundreds of thousands of chips, with the help AI startup Anthropic, which will be the first to use it. Apple executive Benoit Dupin also said that Apple (AAPL.O), opens new tab is using Trainium2 chips. AWS Chief Executive Matt Garman also said that Trainium3, the company's next generation of AI chip, will debut next year. Reporting by Stephen Nellis in San Francisco and Greg Bengisnger in Las Vegas Our Standards: The Thomson Reuters Trust Principles., opens new tab Suggested Topics:DisruptedDisrupted Greg Bensinger Thomson Reuters Greg Bensinger joined Reuters as a technology correspondent in 2022 focusing on the world's largest technology companies. He was previously a member of The New York Times editorial board and a technology beat reporter for The Washington Post and The Wall Street Journal. He also worked for Bloomberg News writing about the auto and telecommunications industries. He studied English literature at The University of Virginia and graduate journalism at Columbia University. Greg lives in San Francisco with his wife and two children.

Amazon launches new AI servers, Apple joins as customer By Investing.com

Amazon (NASDAQ:AMZN) Web Services (AWS) has announced the introduction of new data center servers equipped with its proprietary artificial intelligence (AI) chips, presenting a challenge to Nvidia (NASDAQ:NVDA)'s dominance in the sector. Apple Inc (NASDAQ:AAPL). has been confirmed as a customer, planning to utilize these new Trainium2 chips. AWS's cloud unit revealed that these servers will be part of a massive supercomputer, which will incorporate hundreds of thousands of chips. This announcement was made on Tuesday. This supercomputer, powered by AWS's Trainium2 chips, will be utilized by AI startup Anthropic as the first company to use this technology. Anthropic is known for creating reliable and interpretable AI systems and will leverage the computational power to enhance the capabilities of their AI models. Benoit Dupin, an executive at Apple, also acknowledged that the tech giant is employing Trainium2 chips, signifying a significant adoption of AWS's new offering. Matt Garman, AWS Chief Executive, further disclosed that the company is already working on Trainium3, the next evolution of their AI chip, which is slated to make its debut next year. The new Amazon Elastic (NYSE:ESTC) Compute Cloud (Amazon EC2) instances, powered by AWS Trainium2, are now generally available and introduce the Trn2 UltraServers. These UltraServers are designed to provide exceptional performance and cost efficiency for training and deploying contemporary AI models, including large language models (LLM) and foundation models (FM). The Trn2 instances promise a 30-40% improvement in price performance over current GPU-based EC2 instances and boast 16 Trainium2 chips, delivering 20.8 peak petaflops of compute. This makes them ideal for handling AI workloads with billions of parameters. For even more demanding AI tasks, the Trn2 UltraServers offer a new EC2 service, featuring 64 interconnected Trainium2 chips for up to 83.2 peak petaflops of compute. This setup quadruples the compute, memory, and networking capabilities of a single instance, enabling the training and deployment of the world's largest AI models. The collaborative project between AWS and Anthropic, named Project Rainier, aims to construct an EC2 UltraCluster of Trn2 UltraServers, which will become the world's largest AI compute cluster once completed. AWS also highlighted the upcoming Trainium3 chip, which will be manufactured using a 3-nanometer process node, promising to quadruple the performance of the current Trn2 UltraServers. The AWS Neuron software development kit (SDK) facilitates the optimization of AI models to run on Trainium chips, supporting popular frameworks like JAX and PyTorch, and is integrated with the Hugging Face model hub, which hosts over 100,000 models. Trn2 instances are currently available in the US East (Ohio) AWS Region, with plans to expand availability to additional regions soon. Meanwhile, the Trn2 UltraServers are being offered in a preview phase.

Amazon's AWS unveils new supercomputer with its AI chips in a challenge to Nvidia

Amazon.com's cloud unit on Tuesday showed new data center servers packed with its own AI chips that will challenge Nvidia, with Apple coming aboard as a customer to use them. The new servers, based on 64 of Amazon Web Services' Trainium2 chips, will be strung together in a massive supercomputer with hundreds of thousands of chips, with the help AI startup Anthropic, which will be the first to use it. Apple executive Benoit Dupin also said that Apple is using Trainium2 chips. With more than 70% market share, Nvidia dominates the sale of AI chips, and traditional chip industry rivals such as Advanced Micro Devices are rushing to catch up. But some of Nvidia's most formidable competitors are also its customers: Meta Platforms, Microsoft and Alphabet's Google all have their own custom AI chips. While Meta's chip powers internal operations, Amazon and Google use their chips internally but also market them to paying customers.

Amazon's cloud service shows new AI servers, says Apple will use its chips

The new servers, which AWS calls Trn2 UltraServers, will compete against Nvidia's flagship server packing 72 of its latest "Blackwell" chips. Both companies also offer proprietary technology for connecting the chips, though Gadi Hutt, who leads business development for the AI chips at AWS, said that AWS will be able to connect a greater number of chips together than Nvidia.

Nvidia Rules A.I. Chips, but Amazon and AMD Emerge as Contenders

On the south side of Austin, Texas, engineers at the semiconductor maker Advanced Micro Devices designed an artificial intelligence chip called MI300 that was released a year ago and is expected to generate more than $5 billion in sales in its first year of release. Not far away in a north Austin high-rise, designers at Amazon developed a new and faster version of an A.I. chip called Trainium. They then tested the chip in creations including palm-size circuit boards and complex computers the size of two refrigerators. Those two efforts in the capital of Texas reflect a shift in the rapidly evolving market of A.I. chips, which are perhaps the hottest and most coveted technology of the moment. The industry has long been dominated by Nvidia, which has leveraged its A.I. chips to become a $3 trillion behemoth. For years, others tried to match the company's chips, which provide enormous computing power for A.I. tasks, but made little progress. Now the chips that Advanced Micro Devices, known as AMD, and Amazon have created -- as well as customer reactions to their technology -- are adding to signs that credible alternatives to Nvidia are finally emerging. For some crucial A.I. tasks, Nvidia's rivals are proving they can deliver much faster speed, and at prices that are much lower, said Daniel Newman, an analyst at Futurum Group. "That's what everybody has known is possible, and now we're starting to see it materialize," he said. The shift is being driven by an array of tech companies -- from large competitors such as Amazon and AMD to smaller start-ups -- which have started tailoring their chips for a particular phase of A.I. development that is becoming increasingly important. That process, called "inferencing," happens after companies use chips to train A.I. models. It allows them to carry out tasks such as serving up answers with A.I. chatbots. "The real commercial value comes with inference, and inference is starting to gain scale," said Cristiano Amon, the chief executive of Qualcomm, a mobile chip maker that plans to use Amazon's new chips for A.I. tasks. "We're starting to see the beginning of the change." Nvidia's rivals have also started taking a leaf out of the company's playbook in another way. They have begun emulating Nvidia's tactic of building complete computers -- and not just the chips -- so that customers can wring the maximum power and performance out of the chips for A.I. purposes. The increased competition was evident on Tuesday, when Amazon announced the availability of computing services based on its new Trainium 2 A.I. chips and testimonials from potential users including Apple. The company also unveiled computers containing either 16 or 64 of the chips, which are combinations that particularly accelerate inferencing performance. Amazon is even building a kind of giant A.I. factory for the start-up Anthropic, which it has invested in, said Matt Garman, the chief executive of Amazon Web Services. That computing "cluster" will have hundreds of thousands of the new Trainium chips and will be five times more powerful than any that Anthropic has ever used, said Tom Brown, a founder and the chief compute officer of the start-up, which operates the Claude chatbot and is based in San Francisco. "This means customers will get more intelligence at a lower price and at faster speeds," Mr. Brown said. In total, spending on computers without Nvidia chips by data center operators, which provide the computing power needed for A.I. tasks, is expected to grow 49 percent this year to $126 billion, according to Omdia, a market research firm. Even so, the increased competition does not mean Nvidia is in danger of losing its lead. A spokeswoman for the company pointed to comments made by Jensen Huang, Nvidia's chief executive, who has said his company has major advantages in A.I. software and inferencing capability. Mr. Huang has added that demand is torrid for the company's new Blackwell A.I. chips, which he says perform many more calculations per watt of energy used, despite an increase in the power they need to operate. "Our total cost of ownership is so good that even when the competitor's chips are free, it's not cheap enough," Mr. Huang said in a speech at Stanford University this year. The changing A.I. chip market has partly been propelled by well-funded start-ups such as SambaNova Systems, Groq and Cerebras Systems, which have lately claimed big speed advantages in inferencing, with lower prices and power consumption. Nvidia's current chips can cost as much as $15,000 each, and its Blackwell chips are expected to cost tens of thousands of dollars each. That has pushed some customers toward alternatives. Dan Stanzione, the executive director of the Texas Advanced Computing Center, a research center, said the organization planned to buy a Blackwell-based supercomputer next year, but would most likely also use chips from SambaNova for inferencing tasks because of their lower power consumption and pricing. "That stuff is just too expensive," he said of Nvidia's chips. AMD said it expected to target Nvidia's Blackwell chips with its own new A.I. chips arriving next year. In the company's Austin labs, where it exhaustively tests A.I. chips, executives said inferencing performance was a major selling point. One customer is Meta, the owner of Facebook and Instagram, which says it has trained a new A.I. model, called Llama 3.1 405B, using Nvidia chips but that it uses AMD MI300s chips for providing answers to users. Amazon, Google, Microsoft and Meta are also designing their own A.I. chips to speed up specific computing chores and achieve lower costs, while still building big clusters of machines powered by Nvidia's chips. In December, Google plans to begin selling services based on a sixth generation of internally developed chips, called Trillium, which is nearly five times faster than its predecessor. Amazon, sometimes seen as a laggard in A.I., seems particularly determined to catch up. The company allocated $75 billion this year for A.I. chips and other computing hardware, among other capital spending. At the company's Austin offices -- run by Annapurna Labs, a start-up that it bought in 2015 -- engineers previously developed networking chips and general-purpose microprocessors for Amazon Web Services. Its early A.I. chips, including the first version of Trainium, did not gain much market traction. Amazon is far more optimistic about the new Trainium 2 chips, which are four times faster than previous chips. On Tuesday, the company also announced plans for another chip, Trainium 3, which was set to be even more powerful. Eiso Kant, the chief technology officer of Poolside, an A.I. start-up in Paris, estimated that Trainium 2 would provide a 40 percent improvement in computing performance per dollar compared with Nvidia-based hardware. Amazon also plans to offer Trainium-based services in data centers across the world, Mr. Kant added, which helps with inferencing tasks. "The reality is, in my business, I don't care what silicon is underneath," he said. "What I care about is that I get the best price-performance and that I can get it to the end user."

The furious contest to unseat Nvidia as king of AI chips

Nvidia's rivals have also started taking a leaf out of the company's playbook in another way. They have begun emulating Nvidia's tactic of building complete computers -- and not just the chips -- so that customers can wring the maximum power and performance out of the chips for AI purposes.On the south side of Austin, Texas, engineers at semiconductor maker Advanced Micro Devices designed an artificial intelligence chip called MI300 that was released a year ago and is expected to generate more than $5 billion in sales in its first year of release. Not far away in a north Austin high-rise, designers at Amazon developed a new and faster version of an AI chip called Trainium. They then tested the chip in creations including palm-size circuit boards and complex computers the size of two refrigerators. Those two efforts in the capital of Texas reflect a shift in the rapidly evolving market of AI chips, which are perhaps the hottest and most coveted technology of the moment. The industry has long been dominated by Nvidia, which has leveraged its AI chips to become a $3 trillion behemoth. For years, others tried to match the company's chips, which provide enormous computing power for AI tasks, but made little progress. Now the chips that Advanced Micro Devices, known as AMD, and Amazon have created -- as well as customer reactions to their technology -- are adding to signs that credible alternatives to Nvidia are finally emerging. For some crucial AI tasks, Nvidia's rivals are proving they can deliver much faster speed, and at much lower prices , said Daniel Newman, an analyst at Futurum Group. "That's what everybody has known is possible, and now we're starting to see it materialize," he said. The shift is being driven by an array of tech companies -- from large competitors such as Amazon and AMD to smaller startups -- that have started tailoring their chips for a particular phase of AI development that is becoming increasingly important. That process, called "inferencing," happens after companies use chips to train AI models. It allows them to carry out tasks such as serving up answers with AI chatbots. "The real commercial value comes with inference, and inference is starting to gain scale," said Cristiano Amon, chief executive of Qualcomm, a mobile chipmaker that plans to use Amazon's new chips for AI tasks. "We're starting to see the beginning of the change." Nvidia's rivals have also started taking a leaf out of the company's playbook in another way. They have begun emulating Nvidia's tactic of building complete computers -- and not just the chips -- so that customers can wring the maximum power and performance out of the chips for AI purposes. The increased competition was evident Tuesday, when Amazon announced the availability of computing services based on its new Trainium 2 AI chips and testimonials from potential users including Apple. The company also unveiled computers containing either 16 or 64 of the chips, with ultrafast networking connections that particularly accelerate inferencing performance. Amazon is even building a kind of giant AI factory for the startup Anthropic, which it has invested in, said Matt Garman, chief executive of Amazon Web Services. That computing "cluster" will have hundreds of thousands of the new Trainium chips and will be five times as powerful as any that Anthropic has ever used, said Tom Brown, a founder and the chief compute officer of the startup, which operates the Claude chatbot and is based in San Francisco. "This means customers will get more intelligence at a lower price and at faster speeds," Brown said. In total, spending on computers without Nvidia chips by data center operators, which provide the computing power needed for AI tasks, is expected to grow 49% this year to $126 billion, according to Omdia, a market research firm. Even so, the increased competition does not mean Nvidia is in danger of losing its lead. A spokesperson for the company pointed to comments made by Jensen Huang, Nvidia's chief executive, who has said his company has major advantages in AI software and inferencing capability. Huang has added that demand is torrid for the company's new Blackwell AI chips, which he says perform many more calculations per watt of energy used, despite an increase in the power they need to operate. "Our total cost of ownership is so good that even when the competitor's chips are free, it's not cheap enough," Huang said in a speech at Stanford University this year. The changing AI chip market has partly been propelled by well-funded startups such as SambaNova Systems, Groq and Cerebras Systems, which have lately claimed big speed advantages in inferencing, with lower prices and power consumption. Nvidia's current chips can cost as much as $15,000 each, and its Blackwell chips are expected to cost tens of thousands of dollars each. That has pushed some customers toward alternatives. Dan Stanzione, executive director of the Texas Advanced Computing Center, a research center, said the organization planned to buy a Blackwell-based supercomputer next year but would most likely also use chips from SambaNova for inferencing tasks because of their lower power consumption and pricing. "That stuff is just too expensive," he said of Nvidia's chips. AMD said it expected to target Nvidia's Blackwell chips with its own new AI chips arriving next year. In the company's Austin labs, where it exhaustively tests AI chips, executives said inferencing performance was a major selling point. One customer is Meta, the owner of Facebook and Instagram, which says that it has trained a new AI model, called Llama 3.1 405B, using Nvidia chips but that it uses AMD MI300s chips for providing answers to users. Amazon, Google, Microsoft and Meta are also designing their own AI chips to speed up specific computing chores and achieve lower costs, while still building big clusters of machines powered by Nvidia's chips. This month, Google plans to begin selling services based on a sixth generation of internally developed chips, called Trillium, which is nearly five times as fast as its predecessor. Amazon, sometimes seen as a laggard in AI, seems particularly determined to catch up. The company allocated $75 billion this year for AI chips and other computing hardware, among other capital spending. At the company's Austin offices -- run by Annapurna Labs, a startup that it bought in 2015 -- engineers previously developed networking chips and general-purpose microprocessors for Amazon Web Services. Its early AI chips, including the first version of Trainium, did not gain much market traction. Amazon is far more optimistic about the new Trainium 2 chips, which are four times as fast as previous chips. On Tuesday, the company also announced plans for another chip, Trainium 3, which was set to be even more powerful. Eiso Kant, chief technology officer of Poolside, an AI startup in Paris, estimated that Trainium 2 would provide a 40% improvement in computing performance per dollar compared with Nvidia-based hardware. Amazon also plans to offer Trainium-based services in data centers across the world, Kant added, which helps with inferencing tasks. "The reality is, in my business, I don't care what silicon is underneath," he said. "What I care about is that I get the best price performance and that I can get it to the end user."