NVIDIA's Open-Source AI Model Nemotron-70B Outperforms GPT-4 and Claude 3.5

Nvidia's New Free, Open Source AI Model Reportedly Outperforms OpenAI's GPT-4o, Anthropic's Claude 3.5 Sonnet - Meta Platforms (NASDAQ:META), NVIDIA (NASDAQ:NVDA)

Last week, Nvidia Corp. NVDA quietly introduced a new AI model, Llama-3.1-Nemotron-70B-Instruct, that has reportedly outperformed its competitors in benchmark tests. What Happened: Nvidia's latest AI model has shown remarkable efficiency and performance, despite having fewer parameters. The Nemotron-70B model, built on Meta Platforms Inc.'s META Llama 3.1 framework, has surpassed larger models in benchmark tests, scoring 85.0 in Arena Hard, 57.6 in AlpacaEval 2 LC, and 8.98 in GPT-4-Turbo MT-Bench. The model's superior performance in these tests indicates its ability to produce human-like responses in general queries and coding applications. See Also: Bank Earnings Propel Wall Street To Record Peaks, Oil Plunges Below $70, Gold Shines At All-Time Highs: This Week In The Market The Jensen Huang-led company has also made the Nemotron-70B model open-source, releasing it on the AI community platform, Hugging Face. This move allows developers to modify the model to suit their needs, further improving research and development in AI applications. The model is now available for preview on Nvidia's official site, making it more accessible to the public. Subscribe to the Benzinga Tech Trends newsletter to get all the latest tech developments delivered to your inbox. Why It Matters: This latest AI model launch underscores NVIDIA's growing influence in the AI software space, a shift from its traditional focus on high-performance GPUs. The company's emphasis on efficiency and accessibility suggests a strategic change towards making advanced AI more applicable to developers and the general AI community. Earlier this month, Nvidia CEO praised ChatGPT-parent OpenAI, as one of the most influential companies of today's era in a Bg2 Pod episode. Moreover, the latest announcement comes amid production challenges for its Blackwell chips, which are not expected to be available before early 2025. Check out more of Benzinga's Consumer Tech coverage by following this link. Read Next: AI Power Demand Skyrockets, Nvidia CEO Lauds OpenAI, And Tesla's Optimus Robots Assisted By Humans: This Week In AI Disclaimer: This content was partially produced with the help of Benzinga Neuro and was reviewed and published by Benzinga editors. Photo courtesy: Shutterstock Market News and Data brought to you by Benzinga APIs

Nvidia's new open-source AI model beats GPT-4o on benchmarks

Nvidia unceremoniously launched a new artificial intelligence model on Oct 15 that's purported to outperform state-of-the-art AI systems including GPT-4o and Claude-3. According to a post on the X.com social media platform from the Nvidia AI Developer account, the new model, dubbed Llama-3.1-Nemotron-70B-Instruct, "is a leading model" on lmarena.AI's Chatbot Arena. Nemotron Llama-3.1-Nemotron-70B-Instruct is, essentially, a modified version of Meta's open-source Llama-3.1-70B-Instruct. The "Nemotron" portion of the model's name encapsulates Nvidia's contribution to the end result. The Llama "herd" of AI models, as Meta refers to them, are meant to be used as open-source foundations for developers to build on. In the case of Nemotron, Nvidia took up the challenge and developed a system designed to be more "helpful" than popular models such as OpenAI's ChatGPT and Anthropic's Claude-3. Nvidia used specially curated datasets, advanced fine-tuning methods, and its own state-of-the-art AI hardware to turn Meta's vanilla model into what might be the most "helpful" AI model on the planet. "I asked it a few coding questions I usually ask to compare LLMs and got some of the best answers from this one. lol, holy shit." Benchmarking When it comes to determining which AI model is "the best," there's no clear-cut methodology. Unlike, for example, measuring the ambient temperature with a mercury thermometer, there isn't a single "truth" that exists when it comes to AI model performance. Developers and researchers have to determine how well an AI model performs the same as humans are evaluated: through comparative testing. Related: AI 'mind uploads' could allow the dead to trade forever AI benchmarking involves giving different AI models the same queries, tasks, questions, or problems and then comparing the usefulness of the results. Often, due to the subjectivity of what is and isn't considered useful, human proctors are used to determine a machine's performance through blind evaluations. In Nemotron's case, it appears that Nvidia is claiming the new model outperforms existing state-of-the-art models such as GPT-4o and Claude-3 by a fairly wide margin. The image above depicts the ratings on the automated "Hard" test on the Chatbot Arena Leaderboards. While Nvidia's Llama-3.1-Nemotron-70B-Instruct doesn't appear to be listed anywhere on the boards, if the developer's claim that it scored an 85 on this test is valid, it would be the de facto top model in this particular section. What makes the achievement perhaps even more interesting is that Llama-3.1-70B is Meta's middle-tier open-source AI model. There's a much larger version of Llama-3.1, the 405B version (where the number refers to how many billion parameters the model was tuned with). By comparison, GPT-4o is estimated to have been developed with over one trillion parameters.

NVIDIA Unveils "Industry Leading" Open-Source Llama-3.1-Nemotron-70B-Instruct LLM , Surpassing OpenAI's GPT-4o In AI-Focused Benchmarks

NVIDIA decided to drop one of the industry's biggest "Llama-3.1-Nemotron-70B-Instruct" LLM, surpassing OpenAI GPT-4o & Anthropic's Claude 3.5 Sonnet. Team Green is pushing up the gears when it comes to innovating the AI segment in ways deemed impossible, and after apparently dominating the "AI hardware" segment, the firm is now looking towards showing its magic in open-source LLM models, collaborating with Meta. The newest Llama-3.1-Nemotron-70B-Instruct LLM from NVIDIA hasn't seen much mainstream coverage yet, but based on the initial information available along with benchmarks, the new LLM from Team Green might turn out as industry-leading. NVIDIA says that the Llama-3.1-Nemotron-70B-Instruct LLM is designed solely to make AI responses much more specific and aligned with human preference, especially in terms of factual correctness and coherent problem-solving. The model is said to be trained based on Meta's Llama-3.1-70B-Instruct Base, which is yet again a creation of Meta designed for 70 billion parameters. With NVIDIA's fine-tuning, the Llama-3.1-Nemotron-70B-Instruct specifically targets the "SteerLM Regression Reward Modelling." The post is diving into a bit of technicality, but given a marvel of such kind, I mean, it does deserve it. So, the SteerLM Regression Reward Modelling involves defining a reward function that guides the LLM's learning process by using regression models to refine datasets to generate a clearer response. This makes data quality and model complexity much more refined, ultimately allowing NVIDIA to generate responses close to the user's requirements. Interestingly, based on the Llama-3.1-Nemotron-70B-Instruct LLM model card present at HuggingFace, this particular model manages to solve the "strawberry" problem, which traditional AI models were unable to solve, where it involved counting the R's in the word. This isn't just the only achievement, as the upcoming details might surprise readers more. NVIDIA's Llama-3.1-Nemotron-70B-Instruct LLM has achieved leading ranking at numerous benchmarks, notably Arena Hard, an automatic evaluation tool for instruction-tuned LLMs, and here's how the overall scores stack up. Don't get into the specific figures for now, but the critical element to note here is that the Llama-3.1-Nemotron-70B-Instruct has managed to surpass mainstream LLMs in the industry, such as OpenAI's GPT-4o, which is a significant milestone, given how big of an impact NVIDIA's fine-tuning has on the Llama-3.1-70B-Instruct Base. We haven't seen how the LLM performs in specific situations, such as complex coding tasks, or even inferencing-focused problems, but the initial benchmarks do reveal that NVIDIA's newest LLM is well-equipped. Well, if you are eager to access the Llama-3.1-Nemotron-70B-Instruct LLM, you can either get it from NVIDIA's "NIM" platform here, or there is a compatible version available at HuggingFace, which you can check out here. Overall, Team Green is on its way to becoming dominant in the AI industry, conquering mainstream segments.

Nvidia just dropped a new AI model that crushes OpenAI's GPT-4 -- no big launch, just big results

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Nvidia quietly unveiled a new artificial intelligence model on Tuesday that outperforms offerings from industry leaders OpenAI and Anthropic, marking a significant shift in the company's AI strategy and potentially reshaping the competitive landscape of the field. The model, named Llama-3.1-Nemotron-70B-Instruct, appeared on the popular AI platform Hugging Face without fanfare, quickly drawing attention for its exceptional performance across multiple benchmark tests. Nvidia reports that their new offering achieves top scores in key evaluations, including 85.0 on the Arena Hard benchmark, 57.6 on AlpacaEval 2 LC, and 8.98 on the GPT-4-Turbo MT-Bench. These scores surpass those of highly regarded models like OpenAI's GPT-4o and Anthropic's Claude 3.5 Sonnet, catapulting Nvidia to the forefront of AI language understanding and generation. Nvidia's AI gambit: From GPU powerhouse to language model pioneer This release represents a pivotal moment for Nvidia. Known primarily as the dominant force in graphics processing units (GPUs) that power AI systems, the company now demonstrates its capability to develop sophisticated AI software. This move signals a strategic expansion that could alter the dynamics of the AI industry, challenging the traditional dominance of software-focused companies in large language model development. Nvidia's approach to creating Llama-3.1-Nemotron-70B-Instruct involved refining Meta's open-source Llama 3.1 model using advanced training techniques, including Reinforcement Learning from Human Feedback (RLHF). This method allows the AI to learn from human preferences, potentially leading to more natural and contextually appropriate responses. With its superior performance, the model has the potential to offer businesses a more capable and cost-efficient alternative to some of the most advanced models on the market. The model's ability to handle complex queries without additional prompting or specialized tokens is what sets it apart. In a demonstration, it correctly answered the question "How many r's are in strawberry?" with a detailed and accurate response, showcasing a nuanced understanding of language and an ability to provide clear explanations. What makes these results particularly significant is the emphasis on "alignment," a term in AI research that refers to how well a model's output matches the needs and preferences of its users. For enterprises, this translates into fewer errors, more helpful responses, and ultimately, better customer satisfaction. How Nvidia's new model could reshape business and research For businesses and organizations exploring AI solutions, Nvidia's model presents a compelling new option. The company offers free hosted inference through its build.nvidia.com platform, complete with an OpenAI-compatible API interface. This accessibility makes advanced AI technology more readily available, allowing a broader range of companies to experiment with and implement advanced language models. The release also highlights a growing shift in the AI landscape toward models that are not only powerful but also customizable. Enterprises today need AI that can be tailored to their specific needs, whether that's handling customer service inquiries or generating complex reports. Nvidia's model offers that flexibility, along with top-tier performance, making it a compelling option for businesses across industries. However, with this power comes responsibility. Like any AI system, Llama-3.1-Nemotron-70B-Instruct is not immune to risks. Nvidia has cautioned that the model has not been tuned for specialized domains like math or legal reasoning, where accuracy is critical. Enterprises will need to ensure they are using the model appropriately and implementing safeguards to prevent errors or misuse. The AI arms race heats up: Nvidia's bold move challenges tech giants Nvidia's latest model release signals just how fast the AI landscape is shifting. While the long-term impact of Llama-3.1-Nemotron-70B-Instruct remains uncertain, its release marks a clear inflection point in the competition to build the most advanced AI systems. By moving from hardware into high-performance AI software, Nvidia is forcing other players to reconsider their strategies and accelerate their own R&D. This comes on the heels of the company's introduction of the NVLM 1.0 family of multimodal models, including the 72-billion-parameter NVLM-D-72B. These recent releases, particularly the open-source NVLM project, have shown that Nvidia's AI ambitions go beyond just competing -- they are challenging the dominance of proprietary systems like GPT-4o in areas ranging from image interpretation to solving complex problems. The rapid succession of these releases underscores Nvidia's ambitious push into AI software development. By offering both multimodal and text-only models that compete with industry leaders, Nvidia is positioning itself as a comprehensive AI solutions provider, leveraging its hardware expertise to create powerful, accessible software tools. Nvidia's strategy seems clear: it's positioning itself as a full-service AI provider, combining its hardware expertise with accessible, high-performance software. This move could reshape the industry, pushing rivals to innovate faster and potentially sparking more open-source collaboration across the field. As developers test Llama-3.1-Nemotron-70B-Instruct, we're likely to see new applications emerge across sectors like healthcare, finance, education, and beyond. Its success will ultimately depend on whether it can turn impressive benchmark scores into real-world solutions. In the coming months, the AI community will closely watch how Llama-3.1-Nemotron-70B-Instruct performs in real-world applications beyond benchmark tests. Its ability to translate high scores into practical, valuable solutions will ultimately determine its long-term impact on the industry and society at large. Nvidia's deeper dive into AI model development has intensified the competition. If this is the beginning of a new era in artificial intelligence, it's one where fully integrated solutions may set the pace for future breakthroughs.

NVIDIA Llama 3.1 Nemotron 70b is Outperforming GPT-4o and Claude 3.5

NVIDIA's introduction of the Llama 3.1 Nemotron 70-billion-parameters instruct model marks a significant advancement in the field of artificial intelligence. This open-source model has not only matched but also surpassed the performance of prominent closed-source models like OpenAI's GPT-4o and Claude 3.5 Sonnet, signaling a shift in the AI landscape. The success of the Nemotron 70b underscores the growing influence and potential of open-source AI initiatives within the industry. Imagine a world where the most advanced AI technologies are not locked behind corporate walls but are accessible to everyone, fostering innovation and collaboration. This is the vision that NVIDIA brings to life with the Llama 3.1 Nemotron 70-billion-parameters instruct model. In a landscape dominated by closed-source giants like GPT-4o and Claude 3.5 Sonnet, the Nemotron 70b not only competes but surpasses these models, marking a pivotal moment in AI development. This breakthrough suggests that open-source models are not just catching up -- they are leading the way, opening up a wealth of possibilities for researchers, developers, and enthusiasts worldwide. The secret sauce behind the Nemotron 70b's success lies in its innovative approach to AI training and development. By using advanced techniques such as reinforcement learning and sophisticated reward models, NVIDIA has crafted an AI that aligns more closely with human expectations and preferences. Imagine an AI that not only understands your queries but responds in a way that feels intuitive and contextually appropriate. While we won't provide more insight into all the technical details just yet, it's clear that the Nemotron 70b is setting a new standard for what open-source AI can achieve. The Llama 3.1 Nemotron 70b represents a new standard in AI development, demonstrating that open-source models can compete with and even outperform their proprietary counterparts. This achievement highlights several key advantages of open-source AI: By using these strengths, the Nemotron 70b offers a compelling alternative to closed-source solutions, potentially accelerating the pace of AI advancement across various domains. The exceptional performance of the Nemotron 70b can be attributed to several sophisticated development techniques: Reinforcement Learning Post-Training: This approach allows the model to continue learning from real-world interactions after its initial training phase, enhancing its adaptability and performance in diverse scenarios. Advanced Reward Models: The implementation of sophisticated reward models enables the AI to align its responses more closely with human expectations and preferences. This alignment is crucial for producing outputs that are not only accurate but also relevant and contextually appropriate. Discover other guides from our vast content that could be of interest on Open-source AI model. Two pioneering reward modeling techniques contribute significantly to the Nemotron 70b's capabilities: 1. Bradley Terry Model: This statistical approach evaluates pairs of responses to determine which is superior, allowing for fine-grained optimization of the model's output quality. 2. Regression-Style Scoring: By assigning numeric scores based on specific criteria such as helpfulness, accuracy, and relevance, this method provides a more nuanced framework for improving the model's performance. These advanced reward modeling techniques work in tandem to refine the Nemotron 70b's responses, resulting in more coherent, contextually appropriate, and useful outputs across a wide range of applications. A critical factor in the Nemotron 70b's success is the utilization of the HelpSteer 2 data set. This innovative training resource combines: By integrating these elements, HelpSteer 2 provides a comprehensive framework for training, allowing the Nemotron 70b to learn nuanced patterns and make more informed decisions. This approach enhances the model's ability to handle complex queries and generate high-quality responses across various domains. The Nemotron 70b has demonstrated exceptional performance in several key benchmarks: Reward Bench: Outperforming competitors in this metric, which evaluates the model's ability to generate responses aligned with human preferences. Arena Hard Auto: Excelling in this challenging benchmark, showcasing the model's advanced reasoning capabilities and adaptability to complex scenarios. These results highlight the Nemotron 70b's potential for tackling intricate tasks with precision and reliability, making it suitable for a wide range of applications in research, industry, and beyond. The rapid progress demonstrated by the Nemotron 70b and similar open-source models signals a promising future for AI development. As these models continue to evolve, they hold significant potential for: However, it's important to acknowledge that challenges remain. The Nemotron 70b, like other AI models, still faces difficulties with certain types of reasoning tasks. Ongoing research and development efforts are focused on addressing these limitations through techniques such as: Prompt Engineering: Refining the way queries are presented to the model to optimize its performance in specific scenarios. Continuous Learning: Implementing mechanisms for the model to update and improve its knowledge base over time. Task-Specific Fine-Tuning: Adapting the model for specialized applications while maintaining its general capabilities. NVIDIA's Nemotron 70b represents a significant milestone in open-source AI development. By demonstrating performance that rivals and even surpasses established closed-source models, it sets a new benchmark for what's possible in the realm of accessible, community-driven AI innovation. As the field continues to evolve, open-source models like the Nemotron 70b are poised to play an increasingly crucial role in shaping the future of artificial intelligence, driving progress through collaboration, transparency, and shared knowledge. Jump over to the Hugging Face website for more information on the latest AI model from NVIDIA.

New NVIDIA AI Model To Help Developers Improve Prompt Responses - MEDIANAMA

Disclaimer: This content generated by AI & may have errors or hallucinations. Edit before use. Read our Terms of use Nvidia has released a preview of its new AI model, named Llama-3.1-nemotron-70b-instruct on its website. The company states that the Large Language Model (LLM) is "customized by NVIDIA to improve the helpfulness of LLM-generated responses to user queries." Nvidia intended the AI model to help other AI developers customise the responses of their models across different applications and domains. Built on Meta's Llama 3.1 framework, Nvidia's newest AI model seems to outclass the latest offerings from OpenAI and Anthropic. According to HelpSteer2, an open-source "Helpfulness Dataset" that helps models become more helpful, factually correct, and coherent, the model was able to reach scores like a AlpacaEval 2 LC of 57.6, Arena Hard of 85.0 and a GPT-4-Turbo MT-Bench of 8.98. AlpacaEval tests the ability of models to follow general user instructions while Arena Hard is an evaluation tool for instruction-tuned LLMs, which are AI models that follow instructions. GPT-4-Turbo MT-Bench is a tool that measures the ability of AI models to engage in coherent, informative, and engaging conversations. This makes such as Llama-3.1-nemotron-70b-instruct the highest scoring AI model on these benchmarks, putting it over GPT-4o and Claude 3.5 Sonnet.