Mistral AI Unveils Ministral 3B and 8B: Pioneering On-Device AI Models

Mistral introduces Ministral 3B and 8B on-device AI computing models - SiliconANGLE

Mistral introduces Ministral 3B and 8B on-device AI computing models Mistral AI, a Paris-based artificial intelligence startup, today introduced two new AI large language models, Ministral 3B and 8B, designed for on-device and edge computing thanks to their small size. The company called this new model family "les Ministraux," for its rating in the sub-10 billion-parameter category, which makes them small enough to run on platforms such as smartphones, tablets and internet of things devices. Mistral said the new frontier models can be tuned for common use cases including specialist tasks and work as AI agents via function-calling capabilities. "Our most innovative customers and partners have increasingly been asking for local, privacy-first inference for critical applications such as on-device translation, internet-less smart assistants, local analytics, and autonomous robotics," the company said in the announcement. "Les Ministraux were built to provide a compute-efficient and low-latency solution for these scenarios." These smaller AI models can be used to moderate larger models, such as Mistral Large, as intermediaries in multi-step workflows to handle input parsing, task routing and application calling to reduce costs. The company said that both models support a context length of up to 128,000 tokens, which puts them in line with OpenAI's GPT-4 Turbo for how much data can be input. Ministral 8B also comes with a special "sliding window attention pattern," which allows faster and more memory-efficient deployment. The release of Ministral 3B and 8B comes a year after the release of Mistral 7B, an LLM that the company touted as a significant advancement in model architecture. The 8B and 3B regards the number of parameters in both models, 8 billion and 3 billion, and the company says that the smallest model, Ministral 3B, already outperforms Mistral 7B in most benchmarks. According to benchmarks, pre-trained Ministral 3B beat Google LLC's Gemma 2 2B and Meta Platforms Inc. Llama 3.2 3B models in the Multi-task Language Understanding evaluation with a score of 60.9 compared to 52.4 and 56.2 respectively. Ministral 8B also outperformed Llama 8B with a 65.0 compared to 64.7.

Les Ministraux: Ministral 3B and 8B models bring GenAI to the edge

In a world dominated by bloated AI models that live in the cloud, Mistral AI is flipping the script. The French startup just unleashed two new models -- Ministral 3B and 8B -- that are designed to run on edge devices. Mistral's new offerings, dubbed "Les Ministraux," might sound like a French art-house film, but these models are poised to shake up the AI world. With just 3 billion and 8 billion parameters respectively, the Ministraux family is all about efficiency. Forget those resource-hogging AI models that require a data center to function. "Our most innovative customers and partners have increasingly been asking for local, privacy-first inference for critical applications," Mistral explained. Here's where it gets really spicy: both the 3B and 8B models can handle a context window of 128,000 tokens. That's the equivalent of a 50-page book. For comparison, even OpenAI's GPT-4 Turbo caps out around the same token count, and that's no small feat. With this kind of capacity, the Ministraux models don't just outperform their predecessor, the Mistral 7B -- they're also eating Google's Gemma 2 2B and Meta's Llama models for breakfast. According to Mistral's own benchmarks, the 3B model scored 60.9 in the Multi-task Language Understanding evaluation, leaving competitors like Llama 3.2 3B and Gemma 2 2B trailing at 56.2 and 52.4, respectively. Not bad for a "smaller" model, right? While everyone else in the AI world is scrambling to make bigger, badder models that guzzle energy, Mistral is playing a different game. By running on local devices, Les Ministraux cut out the need for massive cloud servers and, in doing so, offer a much more eco-friendly option. It's a move that aligns perfectly with the increasing pressure on tech companies to be more environmentally conscious. AI might be the future, but nobody wants that future to come at the cost of the planet. There's also the privacy angle. With everything running locally, your data stays on your device, which is a huge win for industries like healthcare and finance that are increasingly under the microscope for how they handle sensitive information. You can think of it as AI without the snooping -- a refreshing change in a world where every app seems to be collecting more data than the NSA. But let's not kid ourselves -- Mistral isn't just doing this for the greater good. The company, co-founded by alumni from Meta and Google's DeepMind, is in the business of making waves. They've already raised $640 million in venture capital and are laser-focused on building AI models that not only rival the likes of OpenAI's GPT-4 and Anthropic's Claude but also turn a profit in the process. And while making money in the generative AI space is about as easy as winning the lottery, Mistral isn't backing down. In fact, they started generating revenue this past summer, which is more than you can say for a lot of their competitors. By offering Ministral 8B for research purposes and making both models available through their cloud platform, La Platforme, Mistral is positioning itself as the cool kid on the AI block -- open enough to attract developers, but smart enough to monetize its tech through strategic partnerships. It's a hybrid approach that mirrors what open-source giants like Red Hat did in the Linux world, fostering community while keeping the cash registers ringing.

Mistral AI Unveils New Models for On-Device AI Computing

Ideal for privacy-first applications like on-device translation, local analytics, and offline smart assistants. French company Mistral AI has announced the launch of its latest models, Ministral 3B and Ministral 8B, on the first anniversary of the release of Mistral 7B. The AI company says these new state-of-the-art models are specifically designed for on-device computing and edge use cases, setting a new benchmark in the sub-10 billion parameter category. The company refers to them as "les Ministraux." Also Read: SAP Expands Its Partnership with Mistral AI to Broaden Customer Choice Ministral 3B and 8B offer enhanced capabilities in knowledge, commonsense reasoning, and function-calling efficiency. They can be used or fine-tuned for a variety of uses, from orchestrating agentic workflows to creating specialist task workers. Both models support a context length of up to 128k (currently 32k on vLLM), enabling them to handle complex tasks with ease. Notably, the Ministral 8B features a special interleaved sliding-window attention pattern for faster and more memory-efficient inference, the company said on Wednesday. Targeting local, privacy-first inference for critical applications, these models are ideal for use cases such as on-device translation, offline smart assistants, local analytics, and autonomous robotics. "Les Ministraux were built to provide a compute-efficient and low-latency solution for these scenarios. From independent hobbyists to global manufacturing teams, les Ministraux deliver for a wide variety of use cases," the company explained. Furthermore, when used alongside larger models like Mistral Large, the les Ministraux models serve as efficient intermediaries for function-calling in multi-step workflows, enabling low-latency input parsing and task routing. "They can be fine-tuned to handle input parsing, task routing, and calling APIs based on user intent across multiple contexts with extremely low latency and cost," the company said. Also Read: BNP Paribas Advances AI Integration into Banking with Over 750 Use Cases in Production According to the company, performance benchmarks indicate that les Ministraux consistently outperform their peers across various tasks, as Mistral re-evaluated all models using its internal framework for fair comparison.

Mistral releases new AI models optimized for edge devices

French AI startup Mistral has released its first generative AI models designed to be run on edge devices, like laptops and phones. The new family of models, which Mistral is calling "Les Ministraux," can be used or tuned for a variety of applications, from text generation to working in conjunction with more capable models to complete tasks. There's two Les Ministraux models available -- Ministral 3B and Ministral 8B -- both of which have a context window of 128,000 tokens, meaning they can ingest roughly the length of a 50-page book. "Our most innovative customers and partners have increasingly been asking for local, privacy-first inference for critical applications such as on-device translation, internet-less smart assistants, local analytics, and autonomous robotics," Mistral writes in a blog post. "Les Ministraux were built to provide a compute-efficient and low-latency solution for these scenarios." The Les Ministraux models are available for download -- but only for research purposes. In point of fact, Mistral is only allowing developers to use Ministral 8B for research -- not Ministral 3B, for which it's requiring a commercial license. Moreover, Mistral's requiring developers and companies interested in Les Ministraux self-deployment setups to contact it for a commercial license. Otherwise, developers can use Ministral 3B and Ministral 8B on its cloud platform -- and other clouds with whom Mistral has partnered with "shortly," the company says. Ministral 8B costs 10 cents per million output/input tokens (~750,000 words), while Minstral 3B costs 4 cents per million output/input tokens.

Mistral AI Launches Ministral 3B and 8B Models for Edge Computing

Benchmarks indicate that the Ministral models outperform competitors, including Gemma 2 2B and Llama 3.2 3B and Llama 3.1 8B Mistral AI has announced the launch of two new models, Ministral 3B and Ministral 8B, on the first anniversary of its Mistral 7B model. These new models focus on on-device computing and edge applications, enhancing capabilities in areas such as knowledge reasoning and function-calling. Ministral models can handle up to 128k context length and offer a unique sliding-window attention pattern for efficient inference, especially in resource-constrained environments. They aim to meet the demand for local and privacy-first inference in applications like on-device translation, smart assistants, local analytics, and robotics. The models serve as intermediaries for larger models, improving task routing and API calling across various contexts. Benchmarks indicate that the Ministral models outperform competitors, including Gemma 2 2B and Llama 3.2 3B and Llama 3.1 8B Both models are available for commercial use, with pricing set at $0.04 per million tokens for Ministral 3B and $0.1 for Ministral 8B. The model weights for the 8B Instruct model will be available for research use. Last month, Mistral AI launched Pixtral 12B, a model capable of processing both images and text. With approximately 12 billion parameters, it employs vision encoding to interpret images alongside text. A day after Meta released Llama 3.1, Mistral AI also launched Mistral Large 2, the latest generation of its flagship model, offering substantial improvements in code generation, mathematics, and multilingual support. The model introduces advanced function-calling capabilities and is available on la Plateforme.

Mistral AI's new language models bring AI power to your phone and laptop

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Mistral AI, a rising star in the artificial intelligence arena, launched two new language models on Wednesday, potentially reshaping how businesses and developers deploy AI technology. The Paris-based startup's new offerings, Ministral 3B and Ministral 8B, are designed to bring powerful AI capabilities to edge devices, marking a significant shift from the cloud-centric approach that has dominated the industry. These compact models, collectively dubbed "les Ministraux," are surprisingly capable despite their small size. Ministral 3B, with just 3 billion parameters, outperforms Mistral's original 7 billion parameter model on most benchmarks. Its larger sibling, Ministral 8B, boasts performance rivaling models several times its size. The significance of this release extends far beyond technical specifications. By enabling AI to run efficiently on smartphones, laptops, and IoT devices, Mistral is opening doors to applications previously considered impractical due to connectivity or privacy constraints. This shift towards edge computing could make advanced AI capabilities more accessible, bringing them closer to end-users and addressing privacy concerns associated with cloud-based solutions. Consider a scenario where a factory robot needs to make split-second decisions based on visual input. Traditionally, this would require sending data to a cloud server for processing, introducing latency and potential security risks. With Ministral models, the AI can run directly on the robot, enabling real-time decision-making without external dependencies. This edge-first approach also has profound implications for personal privacy. Running AI models locally on devices means sensitive data never leaves the user's possession. This could significantly impact applications in healthcare, finance, and other sectors where data privacy is paramount. It represents a fundamental shift in how we think about AI deployment, potentially alleviating concerns about data breaches and unauthorized access that have plagued cloud-based systems. Mistral's timing aligns with growing concerns about AI's environmental impact. Large language models typically require significant computational resources, contributing to increased energy consumption. By offering more efficient alternatives, Mistral is positioning itself as an environmentally conscious choice in the AI market. This move aligns with a broader industry trend towards sustainable computing, potentially influencing how companies approach their AI strategies in the face of growing climate concerns. The company's business model is equally noteworthy. While making Ministral 8B available for research purposes, Mistral is offering both models through its cloud platform for commercial use. This hybrid approach mirrors successful strategies in the open-source software world, fostering community engagement while maintaining revenue streams. By nurturing a developer ecosystem around their models, Mistral is creating a robust foundation against larger competitors, a strategy that has proven effective for companies like Red Hat in the Linux space. The AI landscape is becoming increasingly crowded. Tech giants like Google and Meta have released their own compact models, while OpenAI continues to dominate headlines with its GPT series. Mistral's focus on edge computing could carve out a distinct niche in this competitive field. The company's approach suggests a future where AI is not just a cloud-based service, but an integral part of every device, fundamentally changing how we interact with technology. However, challenges remain. Deploying AI at the edge introduces new complexities in model management, version control, and security. Enterprises will need robust tooling and support to effectively manage a fleet of edge AI devices. This shift could spawn an entirely new industry focused on edge AI management and security, similar to how the rise of cloud computing gave birth to a plethora of cloud management startups. Mistral seems aware of these challenges. The company is positioning its new models as complementary to larger, cloud-based systems. This approach allows for flexible architectures where edge devices handle routine tasks, while more complex queries are routed to more powerful models in the cloud. It's a pragmatic strategy that acknowledges the current limitations of edge computing while still pushing the boundaries of what's possible. The technical innovations behind les Ministraux are equally impressive. Ministral 8B employs a novel "interleaved sliding-window attention" mechanism, allowing it to process long sequences of text more efficiently than traditional models. Both models support context lengths of up to 128,000 tokens, translating to about 100 pages of text -- a feature that could be particularly useful for document analysis and summarization tasks. These advancements represent a leap forward in making large language models more accessible and practical for everyday use. As businesses grapple with the implications of this technology, several key questions emerge. How will edge AI impact existing cloud infrastructure investments? What new applications will become possible with always-available, privacy-preserving AI? How will regulatory frameworks adapt to a world where AI processing is decentralized? The answers to these questions will likely shape the trajectory of the AI industry in the coming years. Mistral's release of compact, high-performing AI models signals more than just a technical evolution -- it's a bold reimagining of how AI will function in the very near future. This move could disrupt traditional cloud-based AI infrastructures, forcing tech giants to rethink their dependence on centralized systems. The real question is: in a world where AI is everywhere, will the cloud still matter?