AMD and Rapt AI Collaborate to Enhance GPU Efficiency for AI Workloads

Rapt AI and AMD work to make GPU utilization more efficient

Rapt AI, a provider of AI-powered AI-workload automation for GPUs and AI accelerators, has teamed with AMD to enhance AI infrastructure. The long-term strategic collaboration aims to improve AI inference and training workload management and performance on AMD Instinct GPUs, offering customers a scalable and cost-effective solution for deploying AI applications. As AI adoption accelerates, organizations are grappling with resource allocation, performance bottlenecks, and complex GPU management. By integrating Rapt's intelligent workload automation platform with AMD Instinct MI300X, MI325X and upcoming MI350 series GPUs, this collaboration delivers a scalable, high-performance, and cost-effective solution that enables customers to maximize AI inference and training efficiency across on-premises and multi-cloud infrastructures. A more efficient solution Charlie Leeming, CEO of Rapt AI, said in a press briefing, "The AI models we are seeing today are so large and most importantly are so dynamic and unpredictable. The older tools for optimizing don't really fit at all. We observed these dynamics. Enterprises are throwing lots of money. Hiring a new set of talent in AI. It's one of these disruptive technologies. We have a scenario where CFOs and CIOs are asking where is the return. In some cases, there is tens of millions, hundreds of millions or billions of dollars spend on GPU-related infrastructure." Leeming said Anil Ravindranath, CTO of Rapt AI, saw the solution. And that involved deploying monitors to enable observations of the infrastructure. "We feel we have the right solution at the right time. We came out of stealth last fall. We are in a growing number of Fortune 100 companies. Two are running the code among cloud service providers," Leeming said. And he said, "We do have strategic partners but our conversations with AMD went extremely well. They are building tremendous GPUs, AI accelerators. We are known for putting the maximum amount of workload on GPUs. Inference is taking off. It's in production stage now. AI workloads are exploding. Their data scientists are running as fast as they can. They are panicking, they need tools, they need efficiency, they need automation. It's screaming for the right solution. Inefficiencies -- 30% GPU underutilization. Customers do want flexibility. Large customers are asking if you support AMD." Improvements that can take nine hours can be done in three minutes, he said. Ravindranath said in a press briefing the Rapt AI platform enables up to 10 times model run capacity at the same AI compute spending level, up to 90% cost savings, and zero humans in a loop and no code changes. For productivity, this means no more waiting for compute and time spent tuning infrastructure. Lemming said other techniques have been around for a while and haven't cut it. Run AI, a rival, overlaps in a competitive way somewhat. He said his company observes in minutes instead of hours and then optimizes the infrastructure. Ravindranath said Run AI is more like a scheduler but Rapt AI positions itself for unpredictable results and deals with it. "We run the model and figure it out, and that's a huge benefit for inference workloads. It should just automatically run," Ravindranath said. The benefits: lower costs, better GPU utilization The companies said that AMD Instinct GPUs, with their industry-leading memory capacity, combined with Rapt's intelligent resource optimization, helps ensure maximum GPU utilization for AI workloads, helping lower total cost of ownership (TCO). Rapt's platform streamlines GPU management, eliminating the need for data scientists to spend valuable time on trial-and-error infrastructure configurations. By automatically optimizing resource allocation for their specific workloads, it empowers them to focus on innovation rather than infrastructure. It seamlessly supports diverse GPU environments (AMD and others, whether in the cloud, on premises or both) through a single instance, helping ensure maximum infrastructure flexibility. The combined solution intelligently optimizes job density and resource allocation on AMD Instinct GPUs, resulting in better inference performance and scalability for production AI deployments. Rapt's auto-scaling capabilities further help ensure efficient resource use based on demand, reducing latency and maximizing cost efficiency. Rapt's platform works out-of-the-box with AMD Instinct GPUs, helping ensure immediate performance benefits. Ongoing collaboration between Rapt and AMD will drive further optimizations in exciting areas such as GPU scheduling, memory utilization and more, helping ensure customers are equipped with a future ready AI infrastructure. "At AMD, we are committed to delivering high-performance, scalable AI solutions that empower organizations to unlock the full potential of their AI workloads." said Negin Oliver, corporate vice president of business development for data center GPU business at AMD, in a statement. "Our collaboration with Rapt AI combines the cutting-edge capabilities of AMD Instinct GPUs with Rapt's intelligent workload automation, enabling customers to achieve greater efficiency, flexibility, and cost savings across their AI infrastructure."

AMD partners with Rapt AI to automate AI workload management on Instinct GPUs - SiliconANGLE

AMD partners with Rapt AI to automate AI workload management on Instinct GPUs Advanced Micro Devices Inc. said today it's collaborating with a startup called Rapt AI Inc. to improve artificial intelligence training and inference performance on its Instinct brand of graphics processing units. Rapt AI is the creator of an intelligent platform that uses AI smarts to automate workload management on high-end GPUs, helping to maximize performance and scale, simplify application deployment and reduce the cost overhead of AI applications. According to the companies, many enterprises are struggling to get a handle on their AI applications. The challenge stems from the fact that customers must rely on huge clusters of GPUs to support their most complex workloads, but many struggle to manage these resources effectively. As such, there's an urgent need for more efficient resource allocation to avoid performance bottlenecks for GPU workloads. "As more organizations move to production AI, maximizing infrastructure efficiency and cost effectiveness becomes paramount," said Rapt AI Chief Technology Officer Anil Ravindranath. Rapt AI's software is designed to work with AMD Instinct accelerators such as the MI300X, MI325X and the upcoming MI350 GPUs, which are alternatives to Nvidia Corp.'s better known H100, H200 and new "Blackwell" AI accelerators. By using Rapt AI's automation software to intelligently manage fleets of AMD GPUs, companies can expect to squeeze the maximum performance out of their silicon for any kind of AI workload, ensuring they fully utilize those resources to lower the total cost of ownership. The software also helps to simplify the deployment of AI applications in both on-premises and cloud environments. According to Rapt AI, it allows organizations to save hours of time experimenting with different infrastructure configurations by automatically setting up the most optimal workload balance, even in diverse compute clusters made up of multiple kinds of GPUs. The result will be improved inference and training performance and increased scalability for production AI deployments, with Rapt AI's unique auto-scaling software optimizing resource allocation based on application demand. AMD's collaboration with Rapt AI means that the software will work perfectly, out-of-the-box, with all AMD Instinct GPUs, helping customers to realize immediate performance benefits with simple deployment. Moreover, the companies plan to collaborate in future to enable further optimizations in areas such as GPU scheduling, memory utilization and more, continuously boosting performance to ensure customers have access to the most optimal and cost-effective AI infrastructure. Rapt AI Chief Executive Charlie Leeming said that by working more closely with AMD, it can develop more intricate performance optimizations, increasing the benefits for joint customers. "This joint solution is set to transform AI infrastructure management, driving better performance, cost efficiency and faster time-to-value for our mutual customers," he promised.

Rapt AI and AMD Partner to Boost AI Workload Management on Instinct GPUs

Rapt AI announced a strategic collaboration with AMD to redefine AI infrastructure management. This alliance aims to improve AI inference and training workload management and performance on AMD Instinct™ GPUs, offering customers a scalable and cost-effective solution for deploying AI applications. As AI adoption accelerates, organizations are grappling with resource allocation, performance bottlenecks, and complex GPU management. By integrating Rapt's intelligent workload automation platform with AMD Instinct MI300X, MI325X and upcoming MI350 series GPUs, this collaboration delivers a scalable, high-performance, and cost-effective solution that enables customers to maximize AI inference and training efficiency across on-premises and multi-cloud infrastructures.