Nvidia Unveils Nemotron-Nano-9B-v2: A Compact AI Powerhouse with Toggle-On Reasoning

Nvidia releases a new small, open model Nemotron-Nano-9B-v2 with toggle on/off reasoning

Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders. Subscribe Now Small models are having a moment. On the heels of the release of a new AI vision model small enough to fit on a smartwatch from MIT spinoff Liquid AI, and a model small enough to run on a smartphone from Google, Nvidia is joining the party today with a new small language model (SLM) of its own, Nemotron-Nano-9B-V2, which attained the highest performance in its class on selected benchmarks and comes with the ability for users to toggle on and off AI "reasoning," that is, self-checking before outputting an answer. While the 9 billion parameters are larger than some of the multimillion parameter small models VentureBeat has covered recently, Nvidia notes it is a meaningful reduction from its original size of 12 billion parameters and is designed to fit on a single Nvidia A10 GPU. As Oleksii Kuchiaev, Nvidia Director of AI Model Post-Training, said on X in response to a question I submitted to him: "The 12B was pruned to 9B to specifically fit A10 which is a popular GPU choice for deployment. It is also a hybrid model which allows it to process a larger batch size and be up to 6x faster than similar sized transformer models." For context, many leading LLMs are in the 70+ billion parameter range (recall parameters refer to the internal settings governing the model's behavior, with more generally denoting a larger and more capable, yet more compute intensive model). The model handles multiple languages, including English, German, Spanish, French, Italian, Japanese, and in extended descriptions, Korean, Portuguese, Russian, and Chinese. It's suitable for both instruction following and code generation. Nemotron-Nano-9B-V2 and its pre-training datasets available right now on Hugging Face and through the company's model catalog. A fusion of Transformer and Mamba architectures It's based on Nemotron-H, a set of hybrid Mamba-Transformer models that form the foundation for the company's latest offerings. While most popular LLMs are pure "Transformer" models, which rely entirely on attention layers, they can become costly in memory and compute as sequence lengths grow. Instead, Nemotron-H models and others using the Mamba architecture developed by researchers at Carnegie Mellon University and Princeton, also weave in selective state space models (or SSMs), which can handle very long sequences of information in and out by maintaining state. These layers scale linearly with sequence length and can process contexts much longer than standard self-attention without the same memory and compute overhead. A hybrid Mamba-Transformer reduces those costs by substituting most of the attention with linear-time state space layers, achieving up to 2-3× higher throughput on long contexts with comparable accuracy. Other AI labs beyond Nvidia such as Ai2 have also released models based on the Mamba architecture. Toggle on/of reasoning using language Nemotron-Nano-9B-v2 is positioned as a unified, text-only chat and reasoning model trained from scratch. The system defaults to generating a reasoning trace before providing a final answer, though users can toggle this behavior through simple control tokens such as /think or /no_think. The model also introduces runtime "thinking budget" management, which allows developers to cap the number of tokens devoted to internal reasoning before the model completes a response. This mechanism is aimed at balancing accuracy with latency, particularly in applications like customer support or autonomous agents. Benchmarks tell a promising story Evaluation results highlight competitive accuracy against other open small-scale models. Tested in "reasoning on" mode using the NeMo-Skills suite, Nemotron-Nano-9B-v2 reaches 72.1 percent on AIME25, 97.8 percent on MATH500, 64.0 percent on GPQA, and 71.1 percent on LiveCodeBench. Scores on instruction following and long-context benchmarks are also reported: 90.3 percent on IFEval, 78.9 percent on the RULER 128K test, and smaller but measurable gains on BFCL v3 and the HLE benchmark. Across the board, Nano-9B-v2 shows higher accuracy than Qwen3-8B, a common point of comparison. Nvidia illustrates these results with accuracy-versus-budget curves that show how performance scales as the token allowance for reasoning increases. The company suggests that careful budget control can help developers optimize both quality and latency in production use cases. Trained on synthetic datasets Both the Nano model and the Nemotron-H family rely on a mixture of curated, web-sourced, and synthetic training data. The corpora include general text, code, mathematics, science, legal, and financial documents, as well as alignment-style question-answering datasets. Nvidia confirms the use of synthetic reasoning traces generated by other large models to strengthen performance on complex benchmarks. Licensing and commercial use The Nano-9B-v2 model is released under the Nvidia Open Model License Agreement, last updated in June 2025. The license is designed to be permissive and enterprise-friendly. Nvidia explicitly states that the models are commercially usable out of the box, and that developers are free to create and distribute derivative models. Importantly, Nvidia does not claim ownership of any outputs generated by the model, leaving responsibility and rights with the developer or organization using it. For an enterprise developer, this means the model can be put into production immediately without negotiating a separate commercial license or paying fees tied to usage thresholds, revenue levels, or user counts. There are no clauses requiring a paid license once a company reaches a certain scale, unlike some tiered open licenses used by other providers. That said, the agreement does include several conditions enterprises must observe: * Guardrails: Users cannot bypass or disable built-in safety mechanisms (referred to as "guardrails") without implementing comparable replacements suited to their deployment. * Redistribution: Any redistribution of the model or derivatives must include the Nvidia Open Model License text and attribution ("Licensed by Nvidia Corporation under the Nvidia Open Model License"). * Compliance: Users must comply with trade regulations and restrictions (e.g., U.S. export laws). * Trustworthy AI terms: Usage must align with Nvidia Trustworthy AI guidelines, which cover responsible deployment and ethical considerations. * Litigation clause: If a user initiates copyright or patent litigation against another entity alleging infringement by the model, the license automatically terminates. These conditions focus on legal and responsible use rather than commercial scale. Enterprises do not need to seek additional permission or pay royalties to Nvidia simply for building products, monetizing them, or scaling their user base. Instead, they must make sure deployment practices respect safety, attribution, and compliance obligations. Positioning in the market With Nemotron-Nano-9B-v2, Nvidia is targeting developers who need a balance of reasoning capability and deployment efficiency at smaller scales. The runtime budget control and reasoning-toggle features are meant to give system builders more flexibility in managing accuracy versus response speed. Their release on Hugging Face and Nvidia's model catalog indicates that they are meant to be broadly accessible for experimentation and integration. Nvidia's release of Nemotron-Nano-9B-v2 showcase a continued focus on efficiency and controllable reasoning in language models. By combining hybrid architectures with new compression and training techniques, the company is offering developers tools that seek to maintain accuracy while reducing costs and latency.

Meet Nemotron Nano AI model from NVIDIA: What does it do better?

Nemotron Nano redefines AI performance, balancing speed, cost, and advanced reasoning capabilities NVIDIA's Nemotron Nano AI model is redefining the possibilities for small language models (SLMs), bringing cutting-edge AI to resource-constrained devices like PCs, workstations, and edge hardware. With versions like the Llama-3.1-Nemotron-Nano-8B-v1 and the Nemotron-Nano-9B-v2, this compact yet powerful model is turning heads for its efficiency and performance. But what exactly makes Nemotron Nano stand out in a crowded field of AI models? From blazing-fast performance to innovative reasoning capabilities, here's a deep dive into why this model is a game-changer. Also read: NVIDIA brings Blackwell to the cloud, announces GeForce Now's biggest upgrade yet at Gamescom 2025 Imagine running a sophisticated AI model on a single NVIDIA RTX GPU or an edge device without needing a sprawling data center. That's the promise of Nemotron Nano. With 8 to 9 billion parameters, it's designed for low-latency, cost-effective deployment on devices like the NVIDIA A10G, H100, or even consumer-grade GPUs. Unlike larger models that demand massive computational resources, Nemotron Nano brings enterprise-grade AI to the edge, making it a game-changer for industries where real-time processing is critical. Take customer support, for example. Businesses can deploy Nemotron Nano to power chatbots that respond instantly to customer queries, all while running on a single GPU. This efficiency translates to lower operational costs and the ability to scale AI solutions without breaking the bank. "Nemotron Nano is about bringing AI to where the data lives," says an NVIDIA spokesperson. "It's about making AI accessible and practical for real-world applications." The Nemotron Nano v2, with its hybrid Mamba-Transformer architecture, is a speed demon. By using only four attention layers, it achieves up to six times higher throughput than similarly sized models like Qwen3-8B on a single NVIDIA A10G GPU in bfloat16 precision. This makes it ideal for applications where every millisecond counts, such as real-time customer interactions or autonomous agent workflows. For developers, this speed means more than just faster responses. It's about enabling AI to handle high volumes of queries without lag, whether it's processing thousands of customer inquiries or analyzing data streams on the fly. In benchmarks, Nemotron Nano's throughput sets it apart, making it a go-to choice for enterprises looking to balance performance and cost. One of Nemotron Nano's most innovative features is its "reasoning on/off" toggle, controlled via tokens like /think or /no_think. This allows developers to fine-tune the model's behavior based on the task at hand. Need a quick answer to a simple question? Turn reasoning off for lightning-fast responses. Facing a complex math problem or coding challenge? Activate reasoning mode for step-by-step logic that rivals larger models. This flexibility shines in benchmarks. Nemotron Nano scores an impressive 72.1% on AIME25 (math), 97.8% on MATH500, 64.0% on GPQA (general knowledge), and 71.1% on LiveCodeBench (coding). These numbers put it neck-and-neck with models many times its size, proving that small doesn't mean less capable. Developers can even adjust the "thinking budget" to control how many tokens are used for reasoning, optimizing performance for edge devices where resources are tight. For tasks beyond text, the Llama Nemotron Nano VL (8B) takes things to the next level. This multimodal vision-language model leads the OCRBench V2 leaderboard for document understanding, excelling at extracting and analyzing information from complex documents like PDFs, charts, tables, and diagrams. Whether it's parsing financial reports, medical records, or legal contracts, Nemotron Nano VL delivers precision on a single GPU, making it a boon for industries like finance, healthcare, and law. Picture a hospital using Nemotron Nano VL to scan and summarize patient records in seconds, or a financial firm extracting key data from dense quarterly reports. Its ability to handle optical character recognition (OCR), text spotting, and table extraction with high accuracy makes it a versatile tool for automating document-heavy workflows. Nemotron Nano isn't just for English-speaking users. It supports a wide array of languages, including German, Spanish, French, Italian, Japanese, Korean, Portuguese, Russian, and Chinese. This makes it a powerful ally for global enterprises looking to deploy AI across diverse markets. Whether it's a customer support bot in Tokyo or a document processor in São Paulo, Nemotron Nano delivers consistent performance across languages, breaking down barriers in international workflows. Also read: NVIDIA becomes first company ever to hit $4 trillion mark, spurred by AI NVIDIA's commitment to openness sets Nemotron Nano apart. Released under the permissive NVIDIA Open Model License, it allows immediate commercial use with minimal restrictions, provided safety guardrails and compliance are maintained. NVIDIA also shares most of the pretraining dataset - 6.6 trillion tokens, including web crawl, math, code, and multilingual Q&A - giving developers unprecedented transparency and the ability to customize the model for specific needs. Integration with NVIDIA's ecosystem, like the NeMo framework for model customization and NIM microservices for scalable deployment, makes Nemotron Nano a plug-and-play solution for enterprises. "We're not just giving you a model; we're giving you the tools to make it your own," says an NVIDIA engineer. This openness, combined with enterprise-grade performance, positions Nemotron Nano as a favorite for businesses looking to innovate without proprietary constraints. Nemotron Nano's strengths lie in its efficiency, speed, and versatility, but it's not without limitations. Its 8B-9B parameter size means it can't match the raw power of larger models like Nemotron Ultra or Llama-3.1-70B for the most complex tasks. And while it's optimized for NVIDIA GPUs, organizations using non-NVIDIA hardware may face compatibility challenges. Still, for edge AI, document processing, multilingual applications, and tasks requiring fast, accurate reasoning, Nemotron Nano is hard to beat. Its ability to deliver high performance on modest hardware makes it a democratizing force in AI, bringing advanced capabilities to businesses and developers who might otherwise be priced out. As AI moves from the cloud to the edge, models like Nemotron Nano are paving the way. Its blend of speed, efficiency, and intelligence makes it a standout in NVIDIA's AI portfolio, offering a glimpse into a future where powerful AI runs on the devices we already own. Whether it's powering smarter chatbots, automating document analysis, or solving complex problems on the fly, Nemotron Nano is proving that big things can come in small packages. For developers and enterprises ready to harness AI at the edge, Nemotron Nano is more than a model, it's a catalyst for innovation. As NVIDIA continues to refine and expand its capabilities, one thing is clear: Nemotron Nano is setting a new standard for what small language models can achieve.