IBM Unveils Granite 3.1: A Leap Forward in Enterprise AI with Enhanced Performance and Capabilities

IBM's new enterprise AI models are more powerful than anything from OpenAI or Google

IBM is zooming along with new open-source Granite Large Language Models (LLM) releases every few months. Granite 3.1 is the latest generation model, building upon the success of Granite 3.0. The model offers enhanced capabilities and performance optimized for business applications. Also: Gemini Advanced users can now access Google's most experimental model The family of Granite 3.1 models boasts an impressive 128K token context window, a substantial increase from their predecessors. This expansion allows the models to process and understand much larger amounts of text -- equivalent to approximately 85,000 English words -- enabling more comprehensive analysis and generation tasks. By comparison, OpenAI's ChatGPT 3, which ignited the AI revolution, could handle only 2,000 tokens. Big Blue claims its new Granite 8B Instruct model outperforms its rivals, such as Google Gemma 2, Meta Llama 3.1, and Qwen 2.5, on HuggingFace's OpenLLM Leaderboard benchmarks. Also: Want generative AI LLMs integrated with your business data? You need RAG The Granite 3.1 family includes dense models and Mixture of Experts (MoE) variants. IBM states its Granite 2B and 8B models are text-only dense LLMs trained on over 12 trillion data tokens. The dense models are designed to support tool-based use cases and for retrieval augmented generation (RAG), streamlining code generation, translation, and bug fixing. The MoE models are trained on over 10 trillion tokens of data. IBM claims these models are ideal for deployment in on-device applications with low latency. The new release also introduces image-in/text-out functionality, broadening the models' applicability for businesses working with graphics. Granite 3.1 also offers improved foreign language proficiency. Alongside English, it can now work with a dozen languages, including German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Simplified Chinese. In short, if you have an international business, IBM has an LLM you can use. As before, the new Granite LLMs are licensed under the Apache 2 open-source license. IBM also maintains its commitment to openness by disclosing the training datasets and providing comprehensive documentation on model development and performance. Also: Generative AI brings new risks to everyone. Here's how you can stay safe If you're worried about intellectual property issues and AI, IBM offers an uncapped indemnity for third-party IP claims related to the use of its models. You may not be concerned about such potential legal problems yet, but I assure you that you will be. Granite 3.1 models are available on IBM's Watsonx platform; cloud service providers like Google Vertex AI; and AI platforms including Hugging Face, NVIDIA (as NIM microservices), Ollama, and Replicate. The release of Granite 3.1 is poised to accelerate AI adoption in enterprise settings. By combining these models with proprietary data using techniques like IBM's InstructLab, businesses can potentially achieve task-specific performance rivaling larger models at a fraction of the cost. As the AI landscape evolves rapidly, IBM's Granite 3.1 represents a significant step forward in providing enterprises with powerful, efficient, and trustworthy AI tools to drive innovation and solve complex business challenges.

​​IBM wants to be the enterprise LLM king with its new open-source Granite 3.1 models

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More IBM is staking its claim at the top of the open-source AI leaderboard with its new Granite 3.1 series out today. The Granite 3.1 large language models (LLMs) offer enterprise users extended context length of 128K tokens, new embedding models, integrated hallucination detection and improved performance. According to IBM, the new Granite 8B Instruct model tops open source rivals of the same size including Meta Llama 3.1, Qwen 2.5 and Google Gemma 2. IBM ranked its models across a series of academic benchmarks included in the OpenLLM Leaderboard. The new models are part of accelerated release cadence of IBM's Granite open source models. Granite 3.0 was just released in October. At the time, IBM claimed that it has a $2 billion book of business related to generative AI. With the Granite 3.1 update, IBM is focusing on packing more capability into smaller models. The basic idea is that smaller models are easier for enterprises to run and are more cost efficient to operate. "We've also just boosted all the numbers -- all the performance of pretty much everything across the board has improved," David Cox, VP for AI models at IBM Research, told VentureBeat. "We use Granite for many different use cases, we use it internally at IBM for our products, we use it for consulting, we make it available to our customers and we release it as open source, so we have to be kind of good at everything." Why performance and smaller models matter for enterprise AI There are any number of ways an enterprise can evaluate the performance of an LLM with benchmarks. The direction that IBM is taking is running models through the gamut of academic and real world tests. Cox emphasized that IBM tested and trained its models to be optimized for enterprise use cases. Performance isn't just about some abstract measure of speed, either; rather, it is a somewhat more nuanced measure of efficiency. One aspect of efficiency that IBM is aiming to push forward is helping users spend less time to get desired results. "You should spend less time fiddling with prompts," said Cox. "So, the stronger a model is in an area, the less time you have to spend engineering prompts." Efficiency is also about model size. The larger a model, the more compute and GPU resources it typically requires, which also means more cost. "When people are doing minimum viable prototype kind of work, they often jump to very large models, so you might go to a 70 billion parameter model or a 405 billion parameter model to build your prototype," said Cox. "But the reality is that many of those are not economical, so the other thing we've been trying to do is drive as much capacity as possible into the smallest package possible." Context matters for enterprise agentic AI Aside from the promise of improved performance and efficiency, IBM has dramatically expanded Granite's context length. With the initial Granite 3.0 release, the context length was limited to 4k. In Granite 3.1, IBM has extended that to 128k, allowing for the processing of much longer documents. The extended context is a significant upgrade for enterprise AI users, both for retrieval augmented generation (RAG) as well as agentic AI. Agentic AI systems and AI agents often need to process and reason over longer sequences of information, such as larger documents, log traces or extended conversations. The increased 128k context length allows these agentic AI systems to have access to more contextual information, enabling them to better understand and respond to complex queries or tasks. IBM is also releasing a series of embedding models to help accelerate the process of converting data into vectors. The Granite-Embedding-30M-English model can achieve performance of 0.16 seconds per query, which IBM claims is faster than rival options including Snowflake's Arctic. How IBM has improved Granite 3.1 to serve enterprise AI needs So how did IBM manage to improve its performance for Granite 3.1? It wasn't any one specific thing, but rather a series of process and technical innovations, Cox explained. IBM has developed increasingly advanced multi-stage training pipelines, he said. This has allowed the company to extract more performance from models. Also, a critical part of any LLM training is data. Rather than just focusing on increasing the quantity of training data, IBM has put a strong emphasis on improving the quality of data used to train the Granite models. "It's not a quantity game," said Cox. "It's not like we're going to go out and get 10 times more data and that's magically going to make models better." Reducing hallucination directly in the model A common approach to reducing the risk of hallucinations and errant outputs in LLMs is to use guardrails. Those are typically deployed as external features alongside an LLM. With Granite 3.1, IBM is directly integrating hallucination protection into the model. The Granite Guardian 3.1 8B and 2B models now include a function-calling hallucination detection capability. "The model can natively do its own guardrailing, which can give different opportunities to developers to catch things," said Cox. He explained that performing hallucination detection in the model itself optimizes the overall process. Internal detection means fewer inference calls, making the model more efficient and accurate. How enterprises can use Granite 3.1 today and what's next The new Granite models are all now freely available as open source to enterprise users. The models are also available via IBM's Watsonx enterprise AI service and will be integrated into IBM's commercial products. The company plans on keeping an aggressive pace for updating the Granite models. Looking forward, the plan for Granite 3.2 is to add multimodal functionality that will debut in early 2025. "You're gonna see us over the next few point releases, adding more of these kinds of different features that are differentiated leading up to the stuff that we'll announce at the IBM Think conference next year," said Cox.