Nvidia releases Nemotron 3 Nano Omni with 30B params for edge AI agents and multimodal tasks

Nvidia releases Nemotron 3 Nano Omni: open multimodal model with 30B params, 3B active, for edge AI agents

Nvidia released Nemotron 3 Nano Omni on Tuesday, an open-weight multimodal AI model that unifies vision, audio, and language understanding in a single architecture designed to power autonomous AI agents on edge devices. The model has 30 billion parameters but activates only three billion per forward pass through a mixture-of-experts design, a ratio that allows it to run on a single GPU while matching or exceeding the multimodal capabilities of models several times its size. Nvidia claims nine times higher throughput than comparable open multimodal models with equivalent interactivity, 2.9 times faster single-stream reasoning on multimodal tasks, and roughly nine times greater effective system capacity for video reasoning. The model tops six benchmarks across document intelligence, video understanding, and audio comprehension. It processes text, images, audio, video, documents, charts, and graphical interfaces as inputs and produces text as output, meaning a single model can replace the patchwork of specialised vision, speech, and document-processing models that most enterprise AI deployments currently stitch together. The release, available on Hugging Face under Nvidia's Open Model Agreement with full commercial use rights, represents the most aggressive move yet by the company that sells the infrastructure for AI into the market for the AI itself. Nemotron 3 Nano Omni uses a hybrid Mamba-Transformer architecture with 23 Mamba-2 selective state-space layers, 23 mixture-of-experts layers with 128 experts routing to six per token plus a shared expert, and six grouped-query attention layers. The vision encoder, C-RADIOv4-H, handles variable-resolution images with 16-by-16 patches scaling from 1,024 to 13,312 visual patches per image. The audio encoder, Parakeet-TDT-0.6B-v2, processes speech and environmental audio. Video processing uses three-dimensional convolutions to capture motion between frames rather than treating video as a sequence of still images. The base text model was pretrained on 25 trillion tokens and supports a 256,000-token context window. The architectural choices reflect a specific design philosophy: maximise capability per active parameter rather than total parameters, because edge deployment is constrained not by model size at rest but by compute per inference step. The three-billion active parameters at inference mean the model can run on hardware announced at Nvidia's GTC 2026 developer conference, including the DGX Spark and DGX Station workstations, without requiring the multi-GPU clusters that power larger models in data centres. The mixture-of-experts approach is not new, but its application to a multimodal model at this scale is. Most open multimodal models either use a single dense architecture, which requires all parameters to be active on every inference step, or use separate specialist models stitched together in a pipeline, which introduces latency at each handoff. Nemotron 3 Nano Omni does neither. It routes each token to six of 128 experts within a unified model, meaning vision tokens, audio tokens, and text tokens all flow through the same architecture but activate different expertise depending on the modality. The result is a model that can process a video feed, a spoken instruction, and a document simultaneously without the inter-model latency that makes pipeline architectures unsuitable for real-time agent applications. For enterprise deployments, this collapses the operational complexity of maintaining separate vision, speech, and language models with separate inference endpoints, monitoring, and versioning into a single model serving a single endpoint. Nvidia has spent the AI boom selling infrastructure: GPUs, networking, and the CUDA software ecosystem that locks developers into its hardware. The Nemotron model family, which has been downloaded more than 50 million times in the past year, represents a parallel strategy in which Nvidia also provides the models that run on that infrastructure. The logic is circular but powerful: Nvidia's models are optimised for Nvidia's hardware, and Nvidia's hardware is optimised for Nvidia's models, creating a full-stack ecosystem that competes with the model-plus-cloud offerings from Google, Amazon, and Microsoft. The case for small, domain-specific language models has been made across education, healthcare, and enterprise, and Nemotron 3 Nano Omni extends that argument to multimodal applications: rather than calling a massive cloud model for every vision or audio task, enterprises can run a compact model locally that handles the full perceptual stack. Early enterprise adoption includes Foxconn, Palantir, Aible, ASI, Eka Care, and H Company, with Dell, DocuSign, Infosys, Oracle, and Zefr evaluating the model for production deployment. The use cases, factory-floor visual inspection, document processing, voice agent applications, and screen understanding for computer-use agents, reflect the market Nvidia is targeting: not consumer AI assistants but industrial AI agents that need to see, hear, and read in real time on local hardware. The model is available as an Nvidia NIM microservice, through Amazon SageMaker JumpStart, and on OpenRouter, with deployment options including vLLM, SGLang, Ollama, llama.cpp, and TensorRT-LLM. The breadth of deployment options is itself a competitive statement: Nvidia is making the model runnable everywhere, on every framework, to maximise adoption and deepen the dependency on Nvidia's broader ecosystem. Open-source AI models designed for agentic reasoning are arriving from multiple directions simultaneously. DeepSeek's V4-Pro and V4-Flash, released last week, use a hybrid attention architecture optimised for long-horizon agentic tasks. Meta's Llama models dominate the open-weight text space. Google's Gemini models handle multimodal tasks at cloud scale. OpenAI's GPT models remain the commercial benchmark. What distinguishes Nemotron 3 Nano Omni is not any single capability but the combination: multimodal perception across vision, audio, and text in a single model, with mixture-of-experts efficiency that enables edge deployment, released as open weights with commercial licensing. No other model currently offers all four properties together. The closest comparators, Google's Gemini Nano for on-device and Meta's Llama for open weights, each lack at least one element: Gemini Nano is not open-weight, and Llama's multimodal capabilities do not include audio processing in a unified architecture. The competitive implications extend beyond the model itself. If Nvidia's open models become the default for edge AI agent deployment, the company captures value at every layer of the stack: the GPU that runs inference, the software framework that optimises it, and now the model itself. Competitors who build on Nvidia's models deepen their dependency on Nvidia's hardware. Competitors who build their own models still need Nvidia's GPUs to train them. The agentic AI era is accelerating across the industry, and Nvidia's strategy is to be indispensable at every layer rather than dominant at one. Nemotron 3 Nano Omni is not Nvidia's answer to GPT-4o. It is Nvidia's argument that the future of AI agents will be built on small, efficient, open models running on Nvidia hardware at the edge, rather than large, proprietary models running on someone else's cloud. Whether that argument holds depends on whether the enterprises building the next generation of autonomous systems prefer local control over cloud convenience, and whether a model with three billion active parameters can do the work that currently requires models with hundreds of billions. The benchmarks say it can. The market will decide whether the benchmarks are right.

Nvidia introduces Nemotron 3 Nano Omni with vision and speech for powerful agentic AI use - SiliconANGLE

Nvidia introduces Nemotron 3 Nano Omni with vision and speech for powerful agentic AI use Nvidia Corp. today launched a powerful reasoning artificial intelligence model that unifies text, vision and speech, capable of acting as the "brains" of faster, smarter agentic AI applications. Dubbed Nemotron 3 Nano Omni, and weighing in at around 30 billion parameters, the new state-of-the-art model uses mixture-of-experts architecture to deliver extremely low latency and provides high flexibility and control. Nvidia combined vision and audio encoders with its 30B-AD3B hybrid MoE architecture to eliminate the need for separate perception modules, allowing its AI model to unify everything into one. The company said this allowed the model to improve efficiency at scale and provide up to nine times faster throughput than other open omni models on the market. "To build useful agents, you can't wait seconds for a model to interpret a screen," said Gautier Cloix, chief executive of H Company. "By building on Nemotron 3 Nano Omni, our agents can rapidly interpret full HD screen recordings -- something that wasn't practical before." The result is a lower cost and higher scalability. With its smaller size, it can also be compressed enough to run on higher-end consumer hardware and execute efficiently on enterprise cloud deployments. The company said it is designed to run alongside other proprietary cloud models or other Nvidia Nemotron open models, such as Nemotron 3 Super for high-frequency execution or Super for complex planning. The new model allows for rapid understanding of documents, computer displays, voice activity, video and more. This makes it the perfect interface for working with people and bridging to more complex machine states. It can take conversational replies from a user and quickly turn it around into reasoning. Nvidia said the Nemotron family - including Ultra, Super and Nano - has seen over 50 million downloads in the past year. The Omni variant extends the family's capabilities into the multimodal and agentic domains.

Nvidia launches Nemotron 3 Nano Omni multimodal AI model

Nvidia unveiled Nemotron 3 Nano Omni, an open multimodal AI model that integrates vision, audio, and language capabilities into a single architecture. The model aims to address the issues of fragmented pipelines in enterprise AI systems by processing multiple input types, including text, images, audio, and video, and generating text as output. Nvidia stated that it combines the knowledge capacity of larger models while reducing computational costs. Constructed on a 30-billion-parameter hybrid mixture-of-experts architecture, Nemotron 3 Nano Omni activates approximately 3 billion parameters per inference. This architecture consolidates components, including a Parakeet speech encoder for audio and a C-RADIOv4-H vision encoder, enhancing the model's performance. Nvidia claims that the model provides up to 9x higher throughput compared to similar open omni models. It achieves around 3x greater throughput with 2.75x lower compute power for video reasoning tasks, supporting a 256K-token context window and topping six leaderboards for complex document intelligence and media understanding. Foxconn, Palantir, and H Company have adopted the model. Gautier Cloix, CEO of H Company, stated, "Utilizing the Nemotron 3 Nano Omni allows our agents to swiftly analyze full HD screen recordings, a capability that was previously unfeasible." Additionally, companies such as Dell, Oracle, and Infosys are currently evaluating the model. The model is accessible on platforms including Hugging Face, OpenRouter, Amazon SageMaker JumpStart, Vultr, and over 25 partner platforms. Nvidia released Nemotron 3 Nano Omni with open weights, datasets, and training recipes for developer customization. This model represents a key component in Nvidia's broader Nemotron 3 family, which includes Super and Ultra models designed for heavier workloads and has recorded over 50 million downloads in the past year.

NVIDIA Launches Nemotron-3 Nano Omni, marking a new era of truly multimodal AI

NVIDIA has launched Nemotron-3 Nano Omni, a unified AI that combines sight, sound, and language in one system. The model marks a shift toward faster, more intuitive, and human-like AI capable of understanding real-world context in real time. In the ever-accelerating world of artificial intelligence, the ambition has long been clear: build machines that experience the world the way humans do through sight, sound, and language, all at once. For years, that vision remained fragmented, stitched together through a clunky chain of specialised tools. Each handled a single "sense," passing outputs down a digital conveyor belt. On April 28, 2026, NVIDIA took a decisive step toward ending that fragmentation. With the launch of Nemotron-3 Nano Omni, the company introduced something more cohesive: a unified intelligence that doesn't just process inputs, but perceives environments. From Graphics pioneer to AI powerhouse To grasp the weight of this shift, it helps to understand the company behind it. Once synonymous with gaming GPUs, NVIDIA has evolved into the backbone of the AI economy. Under CEO Jensen Huang, it has transformed into a full-stack AI giant designing the chips that power machine learning while simultaneously building the software ecosystems that train it. Today, NVIDIA isn't just participating in the AI revolution; it's actively architecting it. The Journey to "Omni' The Nemotron series has mirrored the broader trajectory of AI itself. The earlier Nemotron-2 models excelled in text coding, reasoning, and mathematics while remaining confined to language. By late 2025, the Nemotron-3 introduced "agentic reasoning", allowing AI to plan and execute multi-step tasks with surprising autonomy. But even then, perception remained siloed. The Nano Omni changes that. It fuses reasoning with native multimodal capabilities, effectively giving the model "eyes" and "ears" alongside "brain". Instead of outsourcing vision and audio to separate systems, it processes everything within a single continuous loop. The result is not just faster AI, but more intuitive AI. Ending the "Lost in Translation" Era Previously, analysing something as simple as a video required multiple steps: extracting frames, transcribing audio, and feeding both into a language model. Each step introduced latency and the risk of losing nuance. Nano Omni eliminated that friction. It interprets video, audio, and imagery simultaneously, capturing tone, gesture, and context in real time. A conversation is no longer just words; it becomes an interplay of expression, timing, and intent. This integrated understanding can make outputs dramatically faster and significantly more accurate. Practical Help for the Everyday Professional The real value of this technology is how it saves time for people across various sectors: A shift bigger than speed What Nemotron-3 Nano Omni ultimately represents is a philosophical shift. AI is no longer something users must adapt to learning prompts, structure queries, and simplify communication. Instead, technology is adapting to us. By unifying perception, NVIDIA is reducing the cognitive load of interacting with machines. Professionals are freed from acting like intermediaries, translating real-world problems into machine-readable steps, and can instead focus on decision-making, creativity, and strategy. The Bottom Line This isn't just another model release. It's the quiet dismantling of a long-standing limitation in AI design. With Nano Omni, NVIDIA is pushing the industry toward a future where machines don't just compute, they comprehend. And in doing so, it's redefining productivity itself: not as doing more work, but as doing more meaningful work.

NVIDIA's New 30B Nemotron Model Tested : Mixture of Experts (MoE)

The NVIDIA Nemotron 3 Nano Omni features a 30-billion-parameter Mixture of Experts (MoE) architecture, designed to process diverse input formats such as video, audio, images, PDFs and text. According to All About AI, a recent evaluation highlighted the model's ability to deliver accurate outputs across multiple tasks, including audio transcription, image description and structured text extraction from PDFs. One test involved a React Vite-based application with drag-and-drop functionality, demonstrating how the model handles multimodal inputs with precision and efficiency. Dive into this deep dive to understand how the Nemotron 3 Nano Omni performs in practical applications, from chat-based reasoning to text-to-image generation. Learn about its low-latency cloud processing and open source adaptability, as well as its limitations in handling complex contextual reasoning. This breakdown provides a clear view of the model's capabilities and challenges, helping you evaluate its potential for your specific use cases. The Nemotron 3 Nano Omni is the latest addition to NVIDIA's Nemotron series, which is focused on pushing the boundaries of multimodal AI. Its 30B MoE architecture dynamically allocates computational resources, making sure optimal performance across a wide range of tasks. The model is open source, allowing developers to customize and integrate it into diverse projects. It supports both local inference on compatible hardware and cloud-based deployment, making it accessible to a broad audience, from individual developers to enterprise-level users. This flexibility, combined with its robust architecture, positions the Nemotron 3 Nano Omni as a versatile tool for tackling complex data processing challenges. Its open source nature also encourages innovation, allowing users to adapt the model to their specific needs. A defining feature of the Nemotron 3 Nano Omni is its ability to seamlessly process multiple input formats. This capability makes it a valuable asset for industries that rely on diverse data types. The model excels in converting unstructured data into structured outputs, simplifying workflows and enhancing productivity. Key functionalities include: These capabilities highlight the model's potential to streamline data processing tasks across various domains, from media and education to enterprise-level document management. Here are additional guides from our expansive article library that you may find useful on NVIDIA. To evaluate the Nemotron 3 Nano Omni, a test application was developed using the React Vite framework. This application featured a drag-and-drop interface, allowing users to upload files for processing. The outputs included audio transcriptions, image descriptions, and text extracted from PDFs. The testing process demonstrated the model's versatility and ease of integration into real-world applications. Developers can use its multimodal capabilities to create user-friendly tools that enhance workflows and improve user experiences. The drag-and-drop functionality, combined with the model's ability to handle diverse input formats, underscores its practicality for both individual and enterprise-level projects. The Nemotron 3 Nano Omni delivers impressive performance across several critical metrics. In cloud-based environments, it processes inputs with minimal latency, making sure fast and reliable results. Its accuracy in tasks such as transcription and image description is particularly noteworthy, often producing outputs that require little to no post-processing. The model's reasoning capabilities were also tested through a chat application. It handled complex queries effectively, providing coherent and contextually relevant responses. This ability to process and respond to intricate questions positions the Nemotron 3 Nano Omni as a reliable tool for applications requiring advanced reasoning and decision-making. Another standout feature of the Nemotron 3 Nano Omni is its tool-calling capability. During testing, the model was integrated with OpenCode to execute tool-based tasks efficiently. For example, a single-file HTML application was created to demonstrate text-to-image generation using the GPT-2 Image API. The integration process was smooth and the model executed tasks without compromising performance. This functionality opens up new possibilities for automation and advanced application development. By allowing seamless interaction with external tools, the Nemotron 3 Nano Omni can support complex workflows that require multiple systems to work in tandem. This makes it particularly valuable for developers looking to build sophisticated, AI-driven solutions. The versatility of the Nemotron 3 Nano Omni makes it suitable for a wide range of applications across various industries. Some promising use cases include: These applications underscore the model's potential to transform industries that rely heavily on data-driven processes, offering solutions that are both efficient and scalable. While the Nemotron 3 Nano Omni offers numerous strengths, it is not without its limitations. Certain reasoning tasks, particularly those requiring deep contextual understanding or long-term memory, remain challenging. Additionally, minor bugs were observed in the test application's interface, which could impact the overall user experience. These limitations highlight areas for improvement as the model continues to evolve. Addressing these challenges will be crucial for maximizing its potential and making sure its effectiveness across a broader range of applications. The NVIDIA Nemotron 3 Nano Omni is a powerful multimodal AI model that sets a new standard for processing diverse input formats. Its robust capabilities in transcription, image description and reasoning, combined with its speed and accuracy, make it an invaluable tool for developers and businesses alike. While there are areas for refinement, its potential for applications in automation, content generation and multimodal workflows is undeniable. As AI technology continues to advance, the Nemotron 3 Nano Omni stands out as a promising solution for addressing complex, data-driven challenges. Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, Geeky Gadgets may earn an affiliate commission. Learn about our Disclosure Policy.

What Is NVIDIA Nemotron 3 Nano Omni? The open multimodal model built for agentic AI

Most AI agent systems today are a patchwork. Need to process a screen recording? One model. Transcribe audio from a customer call? Another. Parse a PDF? A third. Each handoff between models adds latency, fragments context, and introduces fresh opportunities for error. NVIDIA's new Nemotron 3 Nano Omni is built to fix exactly that. Also read: Pompeii AI reconstruction: What a Vesuvius victim looked like nearly 2,000 years ago Launched on April 28, 2026, Nemotron 3 Nano Omni is a single omni-modal reasoning model, having the capabilities of vision, audio and language understanding wrapped into one model. Unlike other models, which utilize separate perception models to handle different forms of data, it processes all forms of input - text, images, audio, video, documents, graphs and graphical user interfaces at once as "eyes and ears" for the agent. The performance speak for itself. According to NVIDIA, Nemotron 3 Nano Omni outperforms the competition by providing 9x greater throughput compared to any other open-source omni-modal model of the same quality of interactivity. This means the difference between an interactive and a laggy agent. The model operates on a 30B-A3B mixture-of-experts architecture using Conv3D and EVS components, working with a context of up to 256K. It tops six different leaderboards in complex document intelligence, video understanding and audio reasoning tasks. Also read: Microsoft's Sovereign AI cloud push and its India significance explained NVIDIA positions the model across three primary use cases. For computer use agents, it powers the visual perception loop - H Company's computer use agent, for instance, runs native 1920×1080 resolution inputs using Nemotron 3 Nano Omni, enabling real-time reasoning over full HD screen recordings. For document intelligence, it can reason across mixed-media inputs - PDFs, charts, screenshots - coherently, without losing the thread between visual structure and text. For audio-video workflows in customer service or compliance, it maintains unified context across what was said, shown, and written. It comes pre-trained with open weights, datasets, and methods. And it is now available from Hugging Face, OpenRouter, and build.nvidia.com through NVIDIA NIM Microservices. This model works on local devices such as NVIDIA Jetson, all the way up to data center scale, which is useful for companies with localized data restrictions. This model was initially used by Palantir, Foxconn, Docusign, and Infosys. H Company and Aible are already shipping their products built on this model. The Nemotron 3 series surpassed 50 million downloads in the past year. Omni is its most advanced iteration yet.