Nvidia Launches NeMo Microservices to Boost Enterprise AI Agent Development

Nvidia launches NeMo software tools to help enterprises build custom AI agents for tasks

Chip giant Nvidia on Wednesday announced the general availability of tools to develop "agentic" artificial intelligence for enterprises. Called NeMo microservices, the software tools, which are part of Nvidia's AI Enterprise software portfolio, offer several functions that customize and repeatedly optimize the functioning of AI agents for a variety of tasks, including call centers and software development. Also: The 4 types of people interested in AI agents - and what businesses can learn from them In a media briefing, Nvidia's head of generative AI for enterprise, Joey Conway, framed the NeMo software as a way to use AI agents as "digital employees." "Our view of where we see things going is that there are over a billion knowledge workers across many industries, geographies, and locations," said Conway. "And our view is that digital employees, or AI agents, will be able to help enterprises get more work done in these various domains and scenarios." The early implementations of the AI agents have demonstrated measurable productivity gains, said Conway. For example, Amdocs, a maker of software used by phone companies, has used NeMo microservices to create billing agents, sales agents, and network agents. The billing agent, which handles customers' calls about their phone bills, was able to resolve more inquiries, including a 50% increase in what's called "first-call resolution," said Conway. Also: Crawl, then walk, before you run with AI agents, experts recommend Conway's remarks about agents as digital employees echo a persistent theme from the past year: the idea of AI code as corporate "workers" that can take over corporate processes and be managed just like employees. Nvidia has been offering NeMo software for over five years in a variety of forms, with the overarching goal of speeding up companies' development of AI models. Also: 5 ways to boost your team's productivity - without relying on generative AI Along the way, the company in 2022 began offering NeMo pre-built AI models as an on-demand cloud offering. The microservices followed in October of last year. Components of NeMo include two microservices that have already been available: Curator and Retriever. Curator is used by developers to build "pipelines" that clean and refine data sets used to train or fine-tune AI models. Retriever takes data sources and extracts elements that will be used by the model, such as text, graphics, and chart elements. Also: 93% of IT leaders will implement AI agents in the next two years Three additional components work with Curator and Retriever: Customizer, Evaluator, and Guardrails. The Customizer microservice takes output from Curator and combines it with techniques for post-training, or fine-tuning, to "teach these models new skills," as Conway put it. The Evaluator is a sort of push-button version of AI benchmark tests, which run the model through testing after it has been through Customizer, to evaluate whether the model "actually improved and gained new skills." Also: As AI agents multiply, IT becomes the new HR department Guardrails is meant to operate at runtime with the AI agent to improve "compliance protection" with respect to "safety and security measures" for an enterprise. The intention with NeMo is that models pass repeatedly through the various microservices to be updated and gain new abilities, what Nvidia refers to as a "flywheel." The NeMo microservices are paired with Nvidia's infrastructure software for deployment of agents, called NIM, an acronym for Nvidia Inference Microservices. A NIM is an AI model in an application container that runs on a container manager, such as Kubernetes, and is accessed by developers via an API. Also: I've tried lots of AI image generators, and Nvidia and MIT's is the one to beat for speed The NeMo software will greatly simplify many of the tasks of training, post-training, evaluating, and revising that developers have to do if they work directly with Python code and AI frameworks, said Nvidia's Conway. "The focus for NeMo microservices is being able to build these microservices so that the rest of the ecosystem can get started much faster," said Conway. "From our experience, we've seen that these can be quite complicated," he said, referring to developing AI models and agents, and deploying them. Also: 5 ways to manage your team more effectively in the AI-enabled enterprise "Previously, many of our advanced customers had to rely on various open-source libraries, which are often best effort and not always correct," he added. "We've been able to take all of that software, put it under NeMo Evaluator, add the latest techniques, and then simplify the interaction so it's a few simple API calls." Get the morning's top stories in your inbox each day with our Tech Today newsletter.

Nvidia NeMo microservices to embed AI agents in workflows

Smarter agents, continuous updates, and the eternal struggle to prove ROI As Nvidia releases its NeMo microservices to embed AI agents into enterprise workflows, research has found that almost half of businesses are seeing only minor gains from their investments in AI. NeMo microservices are a set of tools, some already available, which developers can use to build AI agents capable of integrating with existing applications and services to automate tasks, and manage the lifecycle of agents to keep them updated as necessary with the latest information. "There are over a billion knowledge workers across many industries, geographies, and locations, and our view is that digital employees or AI agents will be able to help enterprises get more work done in this variety of domains and scenarios," said Joey Conway, Nvidia's senior director of generative AI software for enterprise. NeMo microservices are also included in the overarching Nvidia AI Enterprise suite of developer tools. The components include NeMo Curator for gathering enterprise data, which gets passed to NeMo customizer, described by Conway as a microservice that "takes the latest state-of-the-art training techniques and teaches models new skills and new knowledge so we can ensure that the models powering the agents stay up to date." NeMo Evaluator is intended to check that the model powering the agent actually has improved instead of regressing, while NeMo Guardrails try to keep the agent on topic so it operates as intended and avoids safety and security pitfalls. Nvidia envisions these microservices working in a circular pipeline, taking new data and user feedback, using this to improve the AI model, then redeploying it. Nvidia refers to this as a "data flywheel," although we can't help feeling that this misunderstands what an actual flywheel does. Conway described NeMo microservices as "essentially like a Docker container." The orchestration relies on Kubernetes, with additional features such as Kubernetes Operators to help. "We have some software today to help with the data preparation and curation. There will be a lot more coming there," he said. Nvidia claims broad software support for its new AI toolkit, including enterprise platforms such as SAP, ServiceNow, and Amdocs; AI software stacks like DataRobot and Dataiku; plus other tools such as DataStax and Cloudera. It also supports models from Google, Meta, Microsoft, Mistral AI, and Nvidia itself. Examples where NeMo microservices are already being put to work include Amdocs, which is laboring on three types of agents for its telecoms operator customers, Nvidia said. These comprise a billing agent, a sales agent, and a network agent. The billing agent focuses on query resolution, while the sales agent works on personalized offers and customer engagement as part of deal closure. The network agent will analyze logs and network information across geographic regions and countries to proactively identify service issues. Developers can download NeMo microservices from the Nvidia NGC catalog, or deploy it as part of Nvidia AI Enterprise suite. Also being released today is research published in the UK which claims that businesses are spending an average of £321,000 ($427,000) on AI in a bid to improve customer experience, though many are yet to see significant gains. It found 44 percent of business leaders indicated that AI has, so far, only delivered a slight improvement. Despite this, nearly all respondents (93 percent) claimed their AI investment has delivered a good return on investment (ROI). The research was commissioned by Storyblok, provider of CMS software for marketers and developers, which said that businesses need to look beyond surface-level implementations and integrate AI in a way that drives meaningful transformation. It found the most popular use cases for AI among UK business leaders are website content creation, customer service, marketing analysis, translation services, and marketing content creation. ®

Enterprises Onboard AI Teammates Faster With NVIDIA NeMo Tools to Scale Employee Productivity

Now generally available, NeMo microservices integrate with partner platforms as building blocks for creating AI agents that get more done using business intelligence and world-class reasoning models, including NVIDIA Llama Nemotron. An AI agent is only as accurate, relevant and timely as the data that powers it. Now generally available, NVIDIA NeMo microservices are helping enterprise IT quickly build AI teammates that tap into data flywheels to scale employee productivity. The microservices provide an end-to-end developer platform for creating state-of-the-art agentic AI systems and continually optimizing them with data flywheels informed by inference and business data, as well as user preferences. With a data flywheel, enterprise IT can onboard AI agents as digital teammates. These agents can tap into user interactions and data generated during AI inference to continuously improve model performance -- turning usage into insight and insight into action. Without a constant stream of high-quality inputs -- from databases, user interactions or real-world signals -- an agent's understanding can weaken, making responses less reliable and agents less productive. Maintaining and improving the models that power AI agents in production requires three types of data: inference data to gather insights and adapt to evolving data patterns, up-to-date business data to provide intelligence, and user feedback data to advise if the model and application are performing as expected. NeMo microservices help developers tap into these three data types. NeMo microservices speed AI agent development with end-to-end tools for curating, customizing, evaluating and guardrailing the models that drive their agents. With NeMo microservices, developers can build data flywheels that boost AI agent accuracy and efficiency. Deployed through the NVIDIA AI Enterprise software platform, NeMo microservices are easy to operate and can run on any accelerated computing infrastructure, on premises or in the cloud, with enterprise-grade security, stability and support. The microservices have become generally available at a time when enterprises are building large-scale multi-agent systems, where hundreds of specialized agents -- with distinct goals and workflows -- collaborate to tackle complex tasks as digital teammates, working alongside employees to assist, augment and accelerate work across functions. This enterprise-wide impact positions AI agents as a trillion-dollar opportunity -- with applications spanning automated fraud detection, shopping assistants, predictive machine maintenance and document review -- and underscores the critical role data flywheels play in transforming business data into actionable insights. NVIDIA partners and industry pioneers are using NeMo microservices to build responsive AI agent platforms so that digital teammates can help get more done. Working with Arize and Quantiphi, AT&T has built an advanced AI-powered agent using NVIDIA NeMo, designed to process a knowledge base of nearly 10,000 documents, refreshed weekly. The scalable, high-performance AI agent is fine-tuned for three key business priorities: speed, cost efficiency and accuracy -- all increasingly critical as adoption scales. AT&T boosted AI agent accuracy by up to 40% using NeMo Customizer and Evaluator by fine-tuning a Mistral 7B model to help deliver personalized services, prevent fraud and optimize network performance. BlackRock is working with NeMo microservices for agentic AI capabilities in its Aladdin tech platform, which unifies the investment management process through a common data language. Teaming with Galileo, Cisco's Outshift team is using NVIDIA NeMo microservices to power a coding assistant that delivers 40% fewer tool selection errors and achieves up to 10x faster response times. Nasdaq is accelerating its Nasdaq Gen AI Platform with NeMo Retriever microservices and NVIDIA NIM microservices. NeMo Retriever enhanced the platform's search capabilities, leading to up to 30% improved accuracy and response times, in addition to cost savings. NeMo microservices support a broad range of popular open models, including Llama, the Microsoft Phi family of small language models, Google Gemma, Mistral and Llama Nemotron Ultra, currently the top open model on scientific reasoning, coding and complex math benchmarks. Meta has tapped NVIDIA NeMo microservices through new connectors for Meta Llamastack. Users can access the same capabilities -- including Customizer, Evaluator and Guardrails -- via APIs, enabling them to run the full suite of agent-building workflows within their environment. "With Llamastack integration, agent builders can implement data flywheels powered by NeMo microservices," said Raghotham Murthy, software engineer, GenAI, at Meta. "This allows them to continuously optimize models to improve accuracy, boost efficiency and reduce total cost of ownership." Leading AI software providers such as Cloudera, Datadog, Dataiku, DataRobot, DataStax, SuperAnnotate, Weights & Biases and more have integrated NeMo microservices into their platforms. Developers can use NeMo microservices in popular AI frameworks including CrewAI, Haystack by deepset, LangChain, LlamaIndex and Llamastack. Enterprises can run AI agents on NVIDIA-accelerated infrastructure, networking and software from leading system providers including Cisco, Dell, Hewlett Packard Enterprise and Lenovo. Consulting giants including Accenture, Deloitte and EY are building AI agent platforms for enterprises using NeMo microservices. Developers can download NeMo microservices from the NVIDIA NGC catalog. The microservices can be deployed as part of NVIDIA AI Enterprise with extended-life software branches for API stability, proactive security remediation and enterprise-grade support.

Nvidia announces general availability of NeMo tools for building AI agents - SiliconANGLE

Nvidia announces general availability of NeMo tools for building AI agents Nvidia Corp. today announced the general availability of NeMo microservices, a set of tools designed to assist developers get artificial intelligence agents up faster by tapping into AI inference and information systems at scale. Agents have become a focal point for creating "digital teammates" capable of driving workforce productivity for knowledge and service workers by taking orders, discovering information and doing proactive work. Unlike AI chatbots, agents can take autonomous actions with little or no human oversight but they need data to make accurate and efficient decisions as part of their reasoning. This can be particularly true for proprietary knowledge, which might be locked behind company firewalls or when using rapidly changing real-time information. "Without a constant stream of high-quality inputs -- from databases, user interactions or real-world signals -- an agent's understanding can weaken, making responses less reliable, which makes agents less productive," said Joey Conway, senior director of generative AI software for enterprise at Nvidia. To help developers rapidly build and deploy agents, Nvidia is releasing NeMo microservices, including Customizer, Evaluator, Guardrails, Retriever and Curator. They are designed to ease enterprise AI engineers' experience building agentic AI experiences when scaling and accessing data. Customizer assists with large language model fine-tuning by providing up to 1.8 times higher training throughput. It provides an application programming interface that allows developers to curate models rapidly so they can fit a dataset before they deploy it. Evaluator simplifies the evaluation of AI models and workflows based on custom and industry benchmarks with just five API calls. Guardrails runs atop an AI model or agent to keep it from behaving in a way that is either unsafe or out of bounds. It can provide additional compliance with 1.4x efficiency and only a half-second more latency. Retriever, announced at GTC 2025, allows developers to build agents that can extract data from systems and accurately process it, enabling them to build complex AI data pipelines such as retrieval-augmented generation. "NeMo microservices are easy to operate and can run on any accelerated computing infrastructure, both on-premises and the cloud, while providing enterprise-grade security, stability and support," added Conway. Nvidia designed the NeMo tools so that developers with general AI knowledge can access them via API calls to get AI agents up and running. Right now enterprises are beginning to build complex multi-agent systems where hundreds of expert agents collaborate to achieve unified goals while working alongside human teammates. NeMo microservices support a large number of popular open AI models, including Meta Platforms Inc.'s Llama, Microsoft Phi family of small language models, Google LLC's Gemma and Mistral. Nvidia's Llama Nemotron Ultra, currently ranking as the top open model on scientific reasoning, coding and complex math benchmarks, is also accessible through the microservices. Numerous leading AI service providers, including Cloudera Inc., Datadog Inc., Dataiku, DataRobot Inc., DataStax Inc., SuperAnnotate AI Inc. and Weights & Biases Inc., have included NeMo microservices in their platforms. Developers can start using these microservices in their processes today through popular AI frameworks such as CrewAI, Haystack by Deepset, LangChain, LlamaIndex and Llamastack. Using the new NeMo microservices, Nvidia partners and tech companies have built AI agent platforms and onboarded digital teammates to get more work done. For example, AT&T Inc. used NeMo Customizer and Evaluator to increase AI agent accuracy by fine-tuning a Mistral 7B model for personalized services, preventing fraud and optimizing network performance. And BlackRock Inc. is working with the microservices in its Aladdin tech platform to unify investment management through a common data language.

Nvidia's NeMo tools aim to fix weak AI returns for businesses

Nvidia has released its NeMo microservices, a set of tools designed to help developers embed AI agents into enterprise workflows. The move comes as research reveals that nearly half of businesses are seeing only minor gains from their AI investments. NeMo microservices are part of Nvidia's AI Enterprise suite and enable developers to build AI agents that can integrate with existing applications and services to automate tasks. The components include NeMo Curator for data gathering, NeMo Customizer for model training, NeMo Evaluator to check model improvement, and NeMo Guardrails to prevent safety and security issues. According to Joey Conway, Nvidia's senior director of generative AI software for enterprise, NeMo microservices work in a circular pipeline, using new data and user feedback to improve the AI model, which is then redeployed. Conway described NeMo microservices as "essentially like a Docker container," with orchestration relying on Kubernetes. Nvidia claims its AI toolkit has broad software support, including enterprise platforms like SAP, ServiceNow, and Amdocs, as well as AI software stacks like DataRobot and Dataiku. Examples of NeMo microservices in use include Amdocs, which is developing three types of agents for telecoms operators: a billing agent, a sales agent, and a network agent. Developers can download NeMo microservices from the Nvidia NGC catalog or deploy it as part of the Nvidia AI Enterprise suite. The release of NeMo microservices coincides with research published in the UK, which found that businesses are spending an average of £321,000 ($427,000) on AI to improve customer experience. Nvidia starts manufacturing AI chips for the US in Texas The research, commissioned by Storyblok, found that 44% of business leaders reported that AI has delivered only a slight improvement, although 93% claimed their AI investment has delivered a good return on investment. The most popular use cases for AI among UK business leaders include website content creation, customer service, marketing analysis, translation services, and marketing content creation.