Cloudera Launches AI Inference Service with NVIDIA NIM to Accelerate GenAI Development and Deployment

Cloudera Launches AI Inference Service with NVIDIA NIM to Boost GenAI Development

Cloudera launched Cloudera AI Inference powered by NVIDIA NIM microservices, part of the NVIDIA AI Enterprise platform. As one of the industry's first AI inference services to provide embedded NIM microservice capability, Cloudera AI Inference uniquely streamlines the deployment and management of large-scale AI models, allowing enterprises to harness their data's true potential to advance GenAI from pilot phases to full production.

Cloudera AI Inference service boosts scalable AI with Nvidia NIM - SiliconANGLE

How Cloudera AI Inference is accelerating scalable AI with NVIDIA NIM microservices for enhanced model deployments As artificial intelligence drives faster insights and real-time decision-making across the enterprise, the Cloudera AI Inference service, designed to operationalize machine learning at scale, is gaining traction. To boost large language model performance and the private deployment of models, the Cloudera AI Inference service uses Nvidia NIM microservices and accelerated computing, according to Priyank Patel (pictured), vice president of artificial intelligence and machine learning at Cloudera Inc. "What we are integrating is the software stack that the Nvidia team has built out, something called NIM -- NIM microservices," Patel stated. "It's an integrated hardware-software layer that sits above their [graphics processing units]. We learned more of what goes into the NIM, and that really formed the basis of the Cloudera AI Inference service. It's the model serving offering from Cloudera that works anywhere on public clouds as well as on-premises and fundamentally enables our customers and enterprises to have private endpoints for AI to be able to build and run AI privately." Patel spoke with theCUBE Research's Bob Laliberte and co-host Rebecca Knight at the Cloudera Evolve24 event during an exclusive broadcast on theCUBE, SiliconANGLE Media's livestreaming studio. They discussed how the Cloudera AI Inference service enables fast deployment of models. (* Disclosure below.) Given that data is growing exponentially, broadening AI solutions with products such as the Cloudera AI Inference service is important. The solution helps enhance user experience, scalability and operational efficiency, Patel pointed out. "AI with Cloudera is about us building the best platform for our customers to build their AI applications with," he noted. "AI in Cloudera is about us infusing AI within our platform without our customers ever needing to know about it, and that means there are dozens of teams internally within our organization who are building the copilots, the assistants and the capabilities that would ease the regular day-to-day user of the Cloudera platform. Cloudera manages a significant amount of data estate both on-premise and the cloud." Making developers' lives easier is top of mind for enterprises. As a result, AI fits into the picture since it transforms developers' work through enhanced collaboration, improved productivity and automated code generation, according to Patel. "When we started out two years ago, the core competence of actually building these AI systems was with the data science teams, the AI teams [and] the machine-learning teams because that's the technology evolution of these deep learning networks," he said. "As it has progressed to now, we see and ... internally use the term gen AI builders, intentionally not calling them developers [or] scientists because we think that there is a simplification of the skill set and up-leveling of skill set that has gone through in the industry." Here's the complete video interview, part of SiliconANGLE's and theCUBE Research's coverage of the Cloudera Evolve24 event:

Cloudera Unveils AI Inference Service with Embedded NVIDIA NIM Microservices to Accelerate GenAI Development and Deployment

Cloudera's AI Inference service boosts LLM performance speeds by 36x using NVIDIA accelerated computing and NVIDIA NIM microservices , providing enhanced performance, robust security, and scalable flexibility for enterprises Combined capability brings together companies' differentiators in a single offering: Cloudera's trusted data as the foundation for trusted AI with NVIDIA accelerated computing and the NVIDIA AI Enterprise software platform to deploy secure and performant AI applications privately on Cloudera Cloudera, the only true hybrid platform for data, analytics, and AI, today launched Cloudera AI Inference powered by NVIDIA NIM microservices, part of the NVIDIA AI Enterprise platform. As one of the industry's first AI inference services to provide embedded NIM microservice capability, Cloudera AI Inference uniquely streamlines the deployment and management of large-scale AI models, allowing enterprises to harness their data's true potential to advance GenAI from pilot phases to full production. Recent data from Deloitte reveals the biggest barriers to GenAI adoption for enterprises are compliance risks and governance concerns, yet adoption of GenAI is progressing at a rapid pace, with over two-thirds of organizations increasing their GenAI budgets in Q3 this year. To mitigate these concerns, businesses must turn to running AI models and applications privately - whether on premises or in public clouds. This shift requires secure and scalable solutions that avoid complex, do-it-yourself approaches. Cloudera AI Inference protects sensitive data from leaking to non-private, vendor-hosted AI model services by providing secure development and deployment within enterprise control. Powered by NVIDIA technology, the service helps to build trusted data for trusted AI with high-performance speeds, enabling the efficient development of AI-driven chatbots, virtual assistants, and agentic applications impacting both productivity and new business growth. The launch of Cloudera AI Inference comes on the heels of the company's collaboration with NVIDIA, reinforcing Cloudera's commitment to driving enterprise AI innovation at a critical moment, as industries navigate the complexities of digital transformation and AI integration. Developers can build, customize, and deploy enterprise-grade LLMs with up to 36x faster performance using NVIDIA Tensor Core GPUs and nearly 4x throughput compared with CPUs. The seamless user experience integrates UI and APIs directly with NVIDIA NIM microservice containers, eliminating the need for command-line interfaces (CLI) and separate monitoring systems. The service integration with Cloudera's AI Model Registry also enhances security and governance by managing access controls for both model endpoints and operations. Users benefit from a unified platform where all models -- whether LLM deployments or traditional models -- are seamlessly managed under a single service. Additional key features of Cloudera AI Inference include: Advanced AI Capabilities: Utilize NVIDIA NIM microservices to optimize open-source LLMs, including LLama and Mistral, for cutting-edge advancements in natural language processing (NLP), computer vision, and other AI domains.Hybrid Cloud & Privacy: Run workloads on prem or in the cloud, with VPC deployments for enhanced security and regulatory compliance.Scalability & Monitoring: Rely on auto-scaling, high availability (HA), and real-time performance tracking to detect and correct issues, and deliver efficient resource management.Open APIs & CI/CD Integration: Access standards-compliant APIs for model deployment, management, and monitoring for seamless integration with CI/CD pipelines and MLOps workflows.Enterprise Security: Enforce model access with Service Accounts, Access Control, Lineage, and Auditing features.Risk-Managed Deployment: Conduct A/B testing and canary rollouts for controlled model updates. "As enterprises rapidly scale their AI capabilities, the need for trusted data and seamless integration becomes more critical than ever. With GenAI advancing rapidly in India, businesses are increasingly focused on balancing innovation with strong governance and compliance to fully harness its potential. Our partnerships with NVIDIA and integration with Snowflake exemplify Cloudera's commitment to empowering organizations to innovate at scale in a secure manner," says Mayank Baid, Regional Vice President, India & South Asia, Cloudera. "By embedding NVIDIA's NIM microservices into Cloudera AI Inference, we are offering unprecedented performance and flexibility for AI applications, while ensuring secure and efficient deployment of large-scale AI models to protect sensitive data. At the same time, extending our Open Data lakehouse interoperability Snowflake enables enterprises to leverage an open, unified hybrid data lakehouse powered by Apache Iceberg. Together, these innovations accelerate the journey from data to insight, enabling businesses to drive meaningful outcomes with trusted, compliant AI solutions." "Enterprises are eager to invest in GenAI, but it requires not only scalable data but also secure, compliant, and well-governed data," said industry analyst, Sanjeev Mohan. "Productionizing AI at scale privately introduces complexity that DIY approaches struggle to address. Cloudera AI Inference bridges this gap by integrating advanced data management with NVIDIA's AI expertise, unlocking data's full potential while safeguarding it. With enterprise-grade security features like service accounts, access control, and audit, organizations can confidently protect their data and run workloads on prem or in the cloud, deploying AI models efficiently with the necessary flexibility and governance." "We are excited to collaborate with NVIDIA to bring Cloudera AI Inference to market, providing a single AI/ML platform that supports nearly all models and use cases so enterprises can both create powerful AI apps with our software and then run those performant AI apps in Cloudera as well," said Dipto Chakravarty, Chief Product Officer at Cloudera. "With the integration of NVIDIA AI, which facilitates smarter decision-making through advanced performance, Cloudera is innovating on behalf of its customers by building trusted AI apps with trusted data at scale." "Enterprises today need to seamlessly integrate generative AI with their existing data infrastructure to drive business outcomes," said Kari Briski, vice president of AI software, models and services at NVIDIA. "By incorporating NVIDIA NIM microservices into Cloudera's AI Inference platform, we're empowering developers to easily create trustworthy generative AI applications while fostering a self-sustaining AI data flywheel". These new capabilities will be unveiled at Cloudera's premier AI and data conference, Cloudera EVOLVE NY, taking place Oct. 10. Click here to learn more about how these latest updates deepen Cloudera's commitment, elevating enterprise data from pilot to production with GenAI

Cloudera Teams With Nvidia To Create New AI Inference Service

The new Cloudera AI Inference Service leverages Nvidia NIM microservices to boost the development of large-scale AI models that can tap into the huge volumes of data stored on the Cloudera Data Platform. Data platform provider Cloudera is taking a deeper dive into AI with a new service, powered by Nvidia's NIM microservices, for deploying and managing large-scale AI models. The new Cloudera AI Inference service, which will make its formal debut at the Cloudera EVOLVE NY event in New York City on Thursday, will make it easier for businesses and organizations to fully leverage their data to advance generative AI projects into full production. "We focus a large part of our energy on getting to 'AI ready,'" said Abhas Ricky, Cloudera chief strategy officer, in an interview with CRN in which outlined his company's role in the current wave of AI development and adoption and how the new AI Inference service expands the company's AI efforts. [Related: Meeting The Exploding Demand For Data: The 2024 CRN Big Data 100] Cloudera's flagship system, the Cloudera Data Platform (CDP), provides a number of data management capabilities including operational database, data engineering, data warehouse, data flow, data stream processing and machine learning functions. The rise of AI and generative AI is creating new demands for trusted data and Cloudera has taken steps to fill those needs. In June, for example, the company acquired the Verta Operational AI Platform from Verta.ai in a move Cloudera said would deepen its AI technology portfolio and expertise with Verta AI and machine learning talent and technology. Ricky said Cloudera's AI strategy is based on three pillars: Ensuring that customers can run scaled AI workloads on GPUs on private clouds; ensuring that clients can leverage any open-source or proprietary model; and providing the necessary tooling that allows customers to work with such capabilities as enterprise search, semantic querying, retrieval augmented generation and more. In March Cloudera announced an expanded collaboration with Nvidia, dubbed Cloudera Powered by Nvidia, through which the two companies planned to integrate Nvidia NIM microservices (part of the Nvidia AI Enterprise software platform) into Cloudera Machine Learning. Ricky said Cloudera AI Inference, powered by embedded NIMS microservices, streamlines the deployment and management of large-scale AI models and allows organizations to serve data on the Cloudera platform to large language models (LLMs) to advance their GenAI projects. Using Cloudera AI Inference developers can build, customize and deploy enterprise-grade LLMs with up to 36x faster performance using Nvidia Tensor Core GPUs and nearly 4x throughput compared to CPU systems, according to the company. Overcoming Data Security Hurdles While many businesses and organizations are launching AI projects, including AI-driven chatbots and virtual assistants, concerns about data compliance and governance have slowed many efforts. And Ricky noted that some 70 to 75 percent of all data assets are on private cloud systems. With Cloudera AI Inference, sensitive data doesn't need to be pushed to a third-party AI model with all the inherent risks of that data leaking out, according to Ricky. Cloudera is building enterprise-grade security and governance around data and model development, deployment and access. According to Cloudera's detailed description of the new service, AI Inference integrates user interfaces and APIs directly with NVIDIA NIM microservice containers, eliminating the need for command-line interfaces and separate monitoring systems. The service's integration with Cloudera's AI Model Registry enhances security and governance by managing access controls for both model endpoints and operations. Users benefit from a unified platform where all models - whether LLM deployments or traditional models - are seamlessly managed under a single service. Cloudera AI Inference utilizes Nvidia NIM microservices to optimize open-source LLMs, including Llama and Mistral, according to the company. Workloads can be run on premise or in the cloud with virtual private cloud deployments for enhanced security and regulatory compliance. And users can rely on auto-scaling, high-availability and real-time performance tracking to detect and correct issues and maintain efficient resource management. Of importance to channel partners, Ricky said Cloudera AI Inference gives the company's systems integrator and ISV partners greater opportunity to build generative AI applications, agents and other software that tap into data in the Cloudera platform. "Enterprises today need to seamlessly integrate generative AI with their existing data infrastructure to drive business outcomes," said Kari Briski, vice president of AI software, models and services at NVIDIA, in a statement. "By incorporating NVIDIA NIM microservices into Cloudera's AI Inference platform, we're empowering developers to easily create trustworthy generative AI applications while fostering a self-sustaining AI data flywheel." Cloudera AI Inference has been in tech preview since June and is now generally available.