Liquid AI Unveils Groundbreaking LFM Models: A New Era in AI Architecture

Liquid LFM 40B: Redefining Transformer AI Architecture

Liquid AI has unveiled its groundbreaking Liquid Foundation Models (LFMs), signaling a significant leap forward in AI architecture. These innovative models seamlessly integrate the strengths of Transformer and Mamba models, establishing a new standard for performance while minimizing memory usage and optimizing inference efficiency. LFMs are carefully crafted to cater to a wide array of applications, including those running on edge devices, ensuring adaptability and scalability across various hardware platforms. The core of LFMs lies in their hybrid architecture, which ingeniously combines the robust capabilities of Transformers with the innovative features of Mamba models. This synergistic approach enables LFMs to: By using the strengths of both architectures, LFMs provide a powerful and flexible solution that adapts to the unique demands of various AI applications. One of the standout features of LFMs is their optimization for multiple hardware platforms, including industry leaders such as NVIDIA, AMD, and Qualcomm. This cross-platform compatibility allows seamless deployment of LFMs across different systems without the need for extensive infrastructure modifications. The benefits of this flexibility are particularly evident in: LFMs' adaptability to various hardware platforms sets them apart as a versatile solution that caters to the diverse needs of businesses and organizations. Here are a selection of other articles from our extensive library of content you may find of interest on the subject of AI transformers : LFMs shine in tasks that require general and expert knowledge, logical reasoning, and handling long context. Their architecture is optimized to excel in applications that demand high-level cognitive processing and decision-making capabilities. This makes LFMs ideal for: By using the strengths of LFMs in these areas, businesses can unlock new possibilities for AI-driven solutions that enhance productivity, customer engagement, and decision-making processes. Liquid AI is committed to the ongoing development and improvement of LFMs. The company plans to continuously update and scale these models, further enhancing their capabilities and expanding their applicability. The focus on memory efficiency and effective context length optimization will remain at the forefront of LFM development, ensuring they stay at the cutting edge of AI innovation. As LFMs evolve, they hold the potential to redefine the landscape of AI deployment and performance optimization. Their ability to deliver efficient and effective results across a wide range of applications positions them as a transformative force in the AI industry. Liquid Foundation Models represent a significant milestone in the evolution of AI architecture. By combining the strengths of Transformers and Mamba models, LFMs offer a versatile and efficient solution that caters to the diverse needs of businesses and organizations. Their cross-platform compatibility, cognitive prowess, and continuous evolution make them a compelling choice for enterprises seeking to harness the power of AI. As the AI landscape continues to evolve, Liquid AI's LFMs are poised to lead the way, setting new benchmarks for performance, efficiency, and adaptability. Embracing this innovative architecture opens doors to a future where AI seamlessly integrates into various aspects of our lives, driving progress and transforming industries.

MIT spinoff Liquid debuts non-transformer AI models and they're already state-of-the-art

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Liquid AI, a startup co-founded by former researchers from the Massachusetts Institute of Technology (MIT)'s Computer Science and Artificial Intelligence Laboratory (CSAIL), has announced the debut of its first multimodal AI models. Unlike most others of the current generative AI wave, these models are not based around the transformer architecture outlined in the seminal 2017 paper "Attention Is All You Need." Instead, Liquid states that its goal "is to explore ways to build foundation models beyond Generative Pre-trained Transformers (GPTs)" and with the new LFMs, specifically building from "first principles...the same way engineers built engines, cars, and airplanes." It seems they've done just that -- as the new LFM models already boast superior performance to other transformer-based ones of comparable size such as Meta's Llama 3.1-8B and Microsoft's Phi-3.5 3.8B. Known as the "Liquid Foundation Models (LFMs)," these models currently come in three different sizes and variants: The "B" in their name stands for billion and refers the number of parameters -- or settings -- that govern the model's information processing, analysis, and output generation. Generally, models with a higher number of parameters are more capable across a wider range of tasks. Already, Liquid AI says the LFM 1.3B version outperforms Meta's new Llama 3.2-1.2B and Microsoft's Phi-1.5 on many leading third-party benchmarks including the popular Massive Multitask Language Understanding (MMLU) consisting of 57 problems across science, tech, engineering and math (STEM) fields, "the first time a non-GPT architecture significantly outperforms transformer-based models." All three are designed to offer state-of-the-art performance while optimizing for memory efficiency, with Liquid's LFM-3B requiring only 16 GB of memory compared to the more than 48 GB required by Meta's Llama-3.2-3B model (shown in the chart above). Maxime Labonne, Head of Post-Training at Liquid AI, took to his account on X to say the LFMs were "the proudest release of my career :)" and to clarify that the core advantage of LFMs: their ability to outperform transformer-based models while using significantly less memory. The models are engineered to be competitive not only on raw performance benchmarks but also in terms of operational efficiency, making them ideal for a variety of use cases, from enterprise-level applications specifically in the fields of financial services, biotechnology, and consumer electronics, to deployment on edge devices. However, importantly for prospective users and customers, the models are not open source. Instead, users will need to access them through Liquid's inference playground, Lambda Chat, or Perplexity AI. How Liquid is going 'beyond' the generative pre-trained transformer (GPT) In this case, Liquid says it used a blend of "computational units deeply rooted in the theory of dynamical systems, signal processing, and numerical linear algebra," and that the result is "general-purpose AI models that can be used to model any kind of sequential data, including video, audio, text, time series, and signals" to train its new LFMs. Last year, VentureBeat covered more about Liquid's approach to training post-transformer AI models, noting at the time that it was using Liquid Neural Networks (LNNs), an architecture developer at CSAIL that seeks to make the artificial "neurons" or nodes for transforming data, more efficient and adaptable. Unlike traditional deep learning models, which require thousands of neurons to perform complex tasks, LNNs demonstrated that fewer neurons -- combined with innovative mathematical formulations -- could achieve the same results. Liquid AI's new models retain the core benefits of this adaptability, allowing for real-time adjustments during inference without the computational overhead associated with traditional models, handling up to 1 million tokens efficiently, while keeping memory usage to a minimum. A chart from the Liquid blog shows that the LFM-3B model, for instance, outperforms popular models like Google's Gemma-2, Microsoft's Phi-3, and Meta's Llama-3.2 in terms of inference memory footprint, especially as token length scales. While other models experience a sharp increase in memory usage for long-context processing, LFM-3B maintains a significantly smaller footprint, making it highly suitable for applications requiring large volumes of sequential data processing, such as document analysis or chatbots. Liquid AI has built its foundation models to be versatile across multiple data modalities, including audio, video, and text. With this multimodal capability, Liquid aims to address a wide range of industry-specific challenges, from financial services to biotechnology and consumer electronics. Accepting invitations for launch event and eyeing future improvements Liquid AI says it is is optimizing its models for deployment on hardware from NVIDIA, AMD, Apple, Qualcomm, and Cerebras. While the models are still in the preview phase, Liquid AI invites early adopters and developers to test the models and provide feedback. Labonne noted that while things are "not perfect," the feedback received during this phase will help the team refine their offerings in preparation for a full launch event on October 23, 2024, at MIT's Kresge Auditorium in Cambridge, MA. The company is accepting RSVPs for attendees of that event in-person here. As part of its commitment to transparency and scientific progress, Liquid says it will release a series of technical blog posts leading up to the product launch event. The company also plans to engage in red-teaming efforts, encouraging users to test the limits of their models to improve future iterations. With the introduction of Liquid Foundation Models, Liquid AI is positioning itself as a key player in the foundation model space. By combining state-of-the-art performance with unprecedented memory efficiency, LFMs offer a compelling alternative to traditional transformer-based models.

Liquid AI debuts new LFM-based models that seem to outperform most traditional LLMs - SiliconANGLE

Liquid AI debuts new LFM-based models that seem to outperform most traditional LLMs Artificial intelligence startup and MIT spin-off Liquid AI Inc. today launched its first set of generative AI models, and they're notably different from competing models as they're built on a fundamental new architecture. The new models are being called "Liquid Foundation Models" or LFMs, and they're said to deliver impressive performance that's on a par with, or even superior to, some of the best large language models available today. The Boston-based startup was founded by a team of researchers from the Massachusetts Institute of Technology, including Ramin Hasani, Mathias Lechner, Alexander Amini, and Daniela Rus. They're said to be pioneers in the concept of "liquid neural networks", which is a class of AI models that's quite different from the Generative Pre-trained Transformer-based models we know and love today, such as OpenAI's GPT series and Google LLC's Gemini models. The company's mission is to create highly capable and efficient general-purpose models that can be used by organizations of all sizes. To do that, it's building LFM-based AI systems that can work at every scale, from the network edge to enterprise-grade deployments. According to Liquid, its LFMs represent a new generation of AI systems that are designed with both performance and efficiency in mind. They use minimal system memory while delivering exceptional computing power, the company explains. They're grounded in dynamical systems, numerical linear algebra and signal processing, which makes them ideal for handling various types of sequential data, including text, audio, images, video and signals. Liquid AI first made headlines in December when it raised $37.6 million in seed funding. At the time, it explained that its LFMs are based on a newer, Liquid Neural Network architecture that was originally developed at MIT's Computer Science and Artificial Intelligence Laboratory. LNNs are based on the concept of artificial neurons, or nodes for transforming data. Whereas traditional deep learning models need thousands of neurons to perform computing tasks, LNNs can achieve the same performance with significantly fewer. It does this by combining those neurons with innovative mathematical formulations, enabling it to do much more with less. The startup says its LFMs retain this adaptable and efficient capability, which enables them to perform real-time adjustments during inference without the enormous computational overheads associated with traditional LLMs. As such, they can handle up to one million tokens efficiently without any noticeable impact on memory usage. Liquid AI is kicking off with a family of three models at launch, including LFM-1B, which is said to be a dense model with 1.3 billion parameters, designed for resource-constrained environments. Slightly more powerful is LFM-3B, which has 3.1 billion parameters and is said to be ideal for edge deployments, such as mobile applications, robots and drones. Finally, there's LFM-40B, which is a vastly more powerful Mixture-of-Experts model with 40.3 billion parameters, designed to deployed on cloud servers in order to handle the most complex use cases. The startup reckons its new models have already shown "state-of-the-art results" across a number of important AI benchmarks, and it believes they are shaping up to be formidable competitors to existing generative AI models like ChatGPT. Whereas traditional LLMs see a sharp increase in memory usage when performing long-context processing, the LFM-3B model notably maintains a much smaller memory footprint (see above) which makes it an excellent choice for applications that require large amounts of sequential data to be processed. Example use cases might include chatbots and document analysis, the company said. In terms of their performance, the LFMs delivered some impressive results, with LFM-1B outperforming transformer-based models in the same size category. Meanwhile, LFM-3B stands up well against models such as Microsoft Corp.'s Phi-3.5 and Meta Platforms Inc.'s Llama family. As for LFM-40B, its efficiency is such that it can even outperform larger models while maintaining an unmatched balance between performance and efficiency. Liquid AI said the LFM-1B model put in an especially dominating performance on benchmarks such as MMLU and ARC-C, setting a new standard for 1B-parameter models. The company is making its models available in early access via platforms such as Liquid Playground, Lambda - via its Chat and application programming interfaces - and Perplexity Labs. That will give organizations a chance to integrate its models into various AI systems and see how they perform in various deployment scenarios, including edge devices and on-premises. One of the things it's working on now is optimizing the LFM models to run on specific hardware built by Nvidia Corp., Advanced Micro Devices Inc., Apple Inc, Qualcomm Inc. and Cerebras Computing Inc., so users will be able to squeeze even more performance out of them by the time they reach general availability. The company says it will release a series of technical blog posts that take a deep dive into the mechanics of each model ahead of their official launch. In addition, it's also encouraging red-teaming, inviting the AI community to test its LFMs to the limit, to see what they can and cannot yet do.