Intel Challenges AI Cloud Market with Gaudi 3-Powered Tiber AI Cloud and Inflection AI Partnership

With AI Cloud, Will Intel Compete with AWS, Azure and Google Cloud?

Intel's Tiber AI Cloud is powered by its new Gaudi 3 accelerator chips. Intel is wading in troubled waters. Yet, in recent months, it has made several attempts to reposition itself in the market by launching new products and exploring strategic partnerships to enhance its competitiveness. Recently, Intel announced an AI cloud service, Tiber AI Cloud, powered by its new Gaudi 3 accelerator chips. The new offering is designed for enterprises and AI startups looking to leverage powerful cloud resources for scalable AI development and deployment. However, this, in turn, will put Intel in direct competition with hyperscalers such as Amazon Web Services (AWS), Microsoft Azure and Google Cloud. Interestingly, these hyperscalers are among the biggest customers of NVIDIA, the company known for its advanced GPUs that Gaudi 3 is being positioned against. So, is Intel aiming to compete with the hyperscalers? Not exactly. Intel does not intend to become the next hyperscaler or establish data centres across the globe. Instead, its focus is solely on the AI cloud space, where it will compete with the hyperscalers. This pivot may also reflect Intel's efforts to explore new business opportunities during a challenging period. This year, the company's stock has dropped over 60%; however, Intel has said that currently, their focus is on AI compute only. Instead, the pivot to enterprise AI cloud has come from growing demand from customers. Markus Flierl, corporate vice president, developer cloud, has told CRN that Tiber AI Cloud is a response to the growing demand from customers. Intel said besides Gaudi 3, Intel will provide Gaudi 2 chips, Xeon CPUs, Core Ultra 200V CPUs to its customers as a part of its AI cloud offerings. For some time now, Intel has asserted that for most enterprise AI use cases that don't involve training large language models, high-end, expensive GPUs are unnecessary, and a robust stack of CPUs can be sufficient. However, the hyperscalers have stacked their data centres with high-end NVIDIA GPUs, and at this moment, it seems like this is what most customers want. While NVIDIA does continue to dominate the market, it does not mean there are no takers for Intel's AI chips. For instance, Bhavish Aggarwal's Krutrim -- India's first generative AI unicorn -- has leveraged Intel's Gaudi 2 chips to train its AI models. Similarly, other Indian companies such as IT giant Tech Mahindra and Infosys have announced a partnership with Intel to use their hardware for AI. Recently, Inflection AI also announced a partnership with Intel to launch a new enterprise Al system called Inflection for Enterprise. AI cloud also seems like a good strategy to get Gaudi 3 out in the market and get customers leveraging them. Intel's Gaudi AI accelerators are seen as a challenger to NVIDIA's dominance in the AI chip market. For Intel, convincing AI companies to switch from NVIDIA GPUs to Gaudi 3 for model training may be a hard sell. Therefore, an AI cloud solution seems like a logical move. Launched earlier this year, Gaudi 3 represents Intel's ambitious push into the rapidly growing AI computing space. According to Intel, Gaudi 3 can deliver up to 70% faster training times for large language models like Llama 2 and GPT-3. For inference tasks, Gaudi 3 is said to match or outperform the memory-rich H200 in certain scenarios, particularly with larger output sequences. Many enterprises and AI startups do not possess the resources to acquire these high-end NVIDIA GPUs. Given the constraint in resources, they look for the most cost-effective solutions and Gaudi 3 chips are relatively cheaper compared to NVIDIA's H100 GPUs. Moreover, with Tiber AI Cloud, Intel is likely to begin renting out its AI chips at an hourly rate. To attract many startups and enterprises, the company will need to offer its AI chips at a lower price than the hyperscalers, making it an appealing option. For instance, Indian AI cloud companies like E2E Networks and Yotta offer NVIDIA's H100 GPUs at a competitive rate of approximately INR 400-500 per hour, making these GPUs accessible to Indian enterprises and startups. Intel has been trying to catch up to NVIDIA for a considerable time, after failing to capitalise early on the surge in AI-specific chips, like the latter did. Additionally, Intel has encountered substantial delays and challenges in its chip manufacturing processes, enabling rivals like TSMC to gain an advantage in advanced chip production. Notably, Gaudi 3 is reportedly based on TSMC's 5 nm node. Reports from last month also suggested that Broadcom is in talks to acquire Intel, or at least, a part of it. In Q2 2024, Intel also reported a $1.6 billion loss. With Gaudi 3, Intel hopes to bring some respite to the company and help steady the boat. However, NVIDIA is not the only company Intel is competing with. In recent years, a wave of AI chip startups has emerged, creating chips that, in many cases, outshine NVIDIA's high-end GPUs. For inference tasks, D-Matrix, a startup founded by Sid Sheth, is developing silicon which works best at inferencing tasks. Its flagship product, Corsair, is specifically designed for inferencing generative AI models (100 billion parameter or less) and is much more cost-effective, compared to GPUs. Groq, another AI chip startup, founded by Jonathan Ross in 2016, claims their AI chips are ten times faster, ten times cheaper, and consume ten times less power. While challenges persist, it remains to be seen whether Intel's AI cloud will achieve broader adoption and how Gaudi 3 chips will perform in comparison to NVIDIA.

Inflection AI's new offering ditches Nvidia for Intel Gaudi

In breaking trends news, Inflection AI revealed its latest enterprise platform would ditch Nvidia GPUs for Intel's Gaudi 3 accelerators. "While Inflection AI's Pi customer application was previously run on Nvidia GPUs, Inflection 3.0 will be powered by Gaudi 3 with instances on-premises or in the cloud powered by [the Tiber] AI Cloud," according to Intel. Inflection AI got its start in 2022 as a model builder developing a conversational personal assistant called Pi. However, following the departure of key founders, Mustafa Suleyman and Karén Simonyan, for Microsoft this spring, the startup has since shifted its focus on building custom fine-tuned models for enterprises using their data. The latest iteration of the startup's platform - Inflection 3.0 - targets fine-tuning of its models using their own proprietary datasets with the goal of building entire enterprise-specific AI apps. Intel itself will be one of the first customers to adopt the service, which does make us wonder whether Inflection is paying full price for the accelerators. While Inflection will be running the service on Gaudi 3 accelerators, it doesn't appear it'll be racking up systems anytime soon. Similar to Inflection 2.5, which was hosted in Azure, the latest iteration will run on Intel's Tiber AI Cloud service. The outfit does, however, see the need for physical infrastructure, at least for customers who would rather keep their data on-prem. Beginning in Q1 2025, Inflection plans to offer a physical system based on Intel's AI accelerators. We'll note that just because the AI startup is using Gaudi 3 accelerators to power its enterprise platform, it doesn't mean customers are stuck with them to run their finished models. AI model and software development isn't exactly cheap and compared to Nvidia's H100, Intel's Gaudi 3 is a relatively bargain. "By running Inflection 3.0 on Intel, we're seeing up to 2x improved price performance... compared with current competitive offerings," Inflection AI CEO Sean White wrote in a blog post on Monday. And at least on paper, Gaudi 3 promises to be not only faster for training and inference than Nvidia's venerable H100, but cheaper at that. Announced at Intel Vision in April, Habana Lab's Gaudi 3 accelerators boast 128 GB of HBM2e memory, good for 3.7 Tbps of bandwidth and 1,835 teraFLOPS of dense FP8 or BF16 performance. While at 8-bit precision it's roughly on par with the H100, at 16-bit precision, it offers nearly twice the dense floating point perf, which makes a big difference for the training and fine-tuning workloads that Inflection is targeting. Intel is among the underdogs in the AI arena, and mainstream availability of the chip is rather poorly timed with the launch of Nvidia's Blackwell and AMD 288GB MI325X GPUs, both of which are due out in Q4. As such, Intel is pricing its accelerators rather aggressively. At Computex this spring, Intel revealed a single Gaudi 3 system with eight accelerators would cost just $125,000 or about two thirds of an equivalent H100 system, according to CEO Pat Gelsinger. Inflection isn't the only win Intel has notched in recent memory. In August, Big Blue announced it would deploy Intel's Gaudi 3 accelerators in IBM Cloud with availability slated for early 2025. Going forward, IBM plans to extend support for Gaudi 3 to its watsonx AI platform. Meanwhile, Intel tells El Reg, the accelerator is already shipping to OEMs, including Dell Technologies and Supermicro. While getting the major OEMs to take Gaudi seriously is a win for Intel, the future of the platform is anything but certain. As we previously reported, Gaudi 3 is the last hurrah for the Habana-Labs-developed accelerator. Starting next year, the Gaudi will give way to a GPU called Falcon Shores, which will fuse Intel's Xe graphics DNA with Habana's chemistry, leading to understandable questions about the migration path. Intel has maintained that for customers coding in high-level frameworks like PyTorch, the migration will be mostly seamless. For those building AI apps at a lower level, the chipmaker has promised to provide additional guidelines prior to Falcon Shores' debut. ®

Intel Debuts AI Cloud With Gaudi 3 Chips, Inflection AI Partnership

The semiconductor giant says it is partnering with Inflection AI for a new enterprise AI solution that is available on Intel Tiber AI Cloud and will be available as a server appliance next year. Intel announced on Monday that it has launched an AI cloud service featuring its new Gaudi 3 accelerator chips, saying that it will help underpin a new enterprise AI solution with hybrid cloud capabilities from generative AI startup Inflection AI. The service, called Intel Tiber AI Cloud, represents a rebrand and expansion in scope, effective at the beginning of the month, for Intel Tiber Developer Cloud, which debuted roughly a year ago without the Tiber name to give customers and partners early access to new chips such as Gaudi 3 and Xeon 6 for development purposes. [Related: Intel Sees 'Huge' AI Opportunities For Xeon -- With And Without Nvidia] With Intel Tiber AI Cloud, the chipmaker is expanding the service's purview to production-level cloud instances that can be used for commercial purposes. The kinds of customers Intel is targeting are large AI startups and enterprises. On top of providing Gaudi 3 instances, the service also features instances powered by Gaudi 2 chips, Xeon CPUs, Core Ultra 200V CPUs and Max Series GPUs, according to Intel. The AI cloud service serves two priorities for Intel: drumming up interest in the "price performance" advantage of Intel's recently launched Gaudi 3 accelerator chips, and creating new recurring software and services revenue opportunities, which Intel CEO Pat Gelsinger outlined in CRN magazine's October 2021 cover story. Intel Tiber AI Cloud is run by Markus Flierl, an Oracle and Sun Microsystems veteran who was previously head of GPU cloud infrastructure at Nvidia before joining Intel in 2022 and taking the title of corporate vice president for Intel Tiber AI Cloud. While the move puts Intel in competition to some extent with cloud service providers ranging from major players like Amazon Web Services to startups like CoreWeave, Flierl said in an interview with CRN that Intel decided to create a commercial cloud AI offering in response to "the demand that we're seeing from customers." He added that the company doesn't have ambitions in the cloud market beyond AI computing. "Competition with AWS is not the focus," he said. Flierl said that the expanded purview of Intel Tiber AI Cloud also gives the company an extra revenue stream beyond the paying customers who were using the service for development purposes. Plus, it gives Intel a better way of building relationships with customers. "If I just give a bunch of free lunches to people, that's easy, but do people like the lunch or not? So adding the commercial aspect helps guide us in terms of what do customers care about, what's important to them. It really elevates that discussion," he said. Intel Tiber AI Cloud lets the chipmaker work more directly with customers than it normally would since the company typically relies on partners to bring its chips to the market in various ways, from PCs and servers to cloud services, according to Flierl. "That's super valuable for us," he said. With data centers in Oregon, Arizona and Ohio, Intel Tiber AI Cloud is viewed as a "huge opportunity" by Flierl, who said "there's a lot of demand" for such solutions. When asked if Intel plans to sell Intel Tiber AI Cloud through channel partners, Flierl said the company is looking at working with other independent software vendors and customers to offer solutions like Inflection AI's new product. In tandem with the Intel Tiber AI Cloud announcement, the semiconductor giant said it is partnering with Inflection AI for a new enterprise AI solution that is available on its cloud service and will be available as a server appliance early next year. Called Inflection for Enterprise, the solution is powered by Intel's new Gaudi 3 processors, and it's designed to "deliver empathetic, conversational, employee-friendly AI capabilities and provide the control, customization and scalability required for complex, large-scale deployments," according to the two companies. Flierl said Intel is looking at the possibility of selling Inflection for Enterprise through the chipmaker's channel partners. Inflection AI is among a crop of generative AI startups whose founders and employees have been poached by tech giants this year to get an upper hand in the highly competitive AI market as Western regulators scrutinize ties between the small and large organizations. In March, Microsoft hired Inflection AI CEO Mustafa Suleyman and co-founders to enhance Copilot and other consumer AI pilots, which prompted the startup to put in place a new leadership team and pivot to commercial AI solutions. Ted Shelton, COO of Inflection's new leadership team, told CRN that the company plans to offer Inflection for Enterprise in three flavors: public cloud, private cloud and on-premises. The decision to offer private cloud and on-premises versions is based on feedback from enterprises, including Intel, who say they don't want to run all their AI applications in the cloud for confidentiality reasons, according to Shelton. (Intel said its's expected to use Inflection for Enterprise as a customer.) "Our contrarian view is actually there's a role for not doing cloud, being able to have things in your own private compute environment, and that's what we're delivering today with our Inflection for Enterprise AI system," he said. With the private cloud version, Inflection for Enterprise will run on cloud instances that are "completely air-gapped" from the rest of Intel's cloud infrastructure, according to Flierl. These private cloud instances will be accessible on a customer's network, which will give the customer control over the machine along with the software that runs on it, Shelton added. He contrasted this approach with enterprise offerings from OpenAI or Anthropic, which are essentially managed services. "[The private cloud version] does not require us to be a managed service provider, and the advantage for the customer in that case is really about security and control of their compute environment," Shelton said. As for the on-premises version, Inflection AI plans to release its server appliance in the first quarter of 2025, but the company isn't ready yet to announce which OEMs it's working with. Shelton said Inflection AI believes offering customers this level of flexibility at the infrastructure level is the right approach. That's because the startup expects that most businesses will need a certain level of compute they "want to control on a dedicated basis" for inference purposes, but they may need to temporarily scale up for fine-tuning the AI models that underpin their applications. "This is actually one of the really important additional advantages of our approach: that when a customer owns their own AI system, they can actually make substantial customizations and modifications that are suitable for their business," he said. Inflection AI decided to go with Intel's Gaudi 3 chips for the Inflection 3.0 system underpinning the enterprise solution after previously using Nvidia GPUs for its consumer-focused Pi AI assistant. Shelton said the main reason for doing so was the "price performance difference" of the Gaudi 3 chips, which Intel has said offer up to roughly two times better price performance than Nvidia H100 GPUs. "We recognize that, in wanting to be able to offer a product directly to the enterprise that they'd be able to purchase and run themselves, cost was going to be a really important consideration. But also, we wanted to make sure we went with a partner that was well recognized and respected by the enterprise," he said.

Inflection AI partners with Intel on new LLM appliance - SiliconANGLE

Inflection AI Inc., a well-funded artificial intelligence startup, is teaming up with Intel Corp. to launch an appliance for running large language models. The companies announced the collaboration today. The appliance is part of a new offering, Inflection for Enterprise, that also includes a cloud-based AI service. Inflection AI launched in 2022 and raised $1.3 billion from investors for a ChatGPT alternative called Pi. This past March, Microsoft Corp. hired the company's co-founder and Chief Executive Officer, Mustafa Suleyman, to lead its consumer AI group. The tech giant also recruited most of Inflection AI's employees and licensed its AI models in a transaction reportedly worth $650 million. Following the Microsoft deal, the LLM developer hired a new leadership team to launch a pivot. Inflection AI announced plans to refocus from its Pi consumer chatbot to developing AI models for the enterprise market. Enterprises are also the target market for the new AI appliance that Inflection AI plans to launch through its new partnership with Intel. The appliance is powered by the chipmaker's Gaudi 3 machine learning accelerator. Introduced in April, the processor features more than three times as many AI-optimized cores as its predecessor. Additionally, Intel has upgraded the built-in Ethernet module that Gaudi 3 uses to share data with the other components of the AI clutter in which it's installed. The Inflection for Enterprise appliance combines the Gaudi 3 with Inflection AI's latest LLM, Inflection 3.0. The AI provider says that its software can run on Intel's silicon up to twice as cost-efficiently as on certain rival processors. Inflection 3.0 is available in two editions. One is geared towards powering chatbots while the other is optimized for tasks that require closely following user instructions. The latter LLM can also package its prompt responses into the JSON data format, which makes it easier for developers to integrate the model's output into applications. Inflection for Enterprise customers will receive a version of Inflection 3.0 customized to their requirements. The company customizes the LLM by fine tuning it on each organization's data. According to Inflection AI, this fine-tuning process makes its LLM more useful for the organization's employees and ensures that the model's output aligns with internal content style guidelines. "Inflection for Enterprise is the only system that allows enterprises to own their intelligence in its entirety," Inflection AI CEO Sean White wrote in a blog post. "You own your data, your fine-tuned model, and the architecture it runs on. It's fully in your control to host on-premises, in the cloud, or hybrid." Intel and Inflection AI plan to make their jointly-developed appliance available in the first quarter of 2025. The chipmaker is expected to be among the first customers. In the meantime, Inflection for Enterprise is available through Intel Tiber AI Cloud, a cloud platform that provides on-demand access to Gaudi 3 and several of the chipmaker's other processors.