UAE's TII Unveils Falcon 3: A Powerful Family of Small Language Models Challenging Open-Source Leaders

UAE's TII Announces 'Powerful' Small AI Model Falcon 3

The small model is available in multiple variants up to 10 billion parameters, and has been released on Hugging Face. Technology Innovation Institute (TII), a research institute from Abu Dhabi, UAE, has unveiled a new family of small language models titled Falcon 3. The models range from one billion parameters to 10 billion parameters in both base and instruct versions. Falcon is available as an open-source model under TII's Falcon License 2.0. The institute also released benchmark results comparing some of the other leading models in its category. Both Falcon 3 7B and 10B outperformed models like Qwen 2.5 7B and Llama 3.1 8B in several benchmarks. TII is a global research institution based in Abu Dhabi and funded by the Abu Dhabi government. It was established in May 2020 and focuses on research in AI, quantum computing, robotics, and cryptography. Falcon 3 employed a shared parameter technique called Grouped Query Attention (GQA) that reduces the memory demands, thereby leading to low latency during inference. "The initial training was followed by multiple stages to improve reasoning and math performance with high-quality data and context extension with natively long context data," read the announcement. The model was also trained in four languages, including English, Spanish, Portuguese, and French. All variants of the Falcon model are available for download on Hugging Face. In August, TII launched the Falcon Mamba 7B model. It outperformed Meta's Llama 3.1 8B, Llama 3 8B, and Mistral's 7B in benchmarks. In May, they launched Falcon 2, an 11B text and vision model. Are small language models finally delivering the promise? A few days ago, Microsoft announced the latest Phi-4 model. With just 14B parameters, the model outperformed much larger models like Llama 3.3 70B and GPT 4o on several benchmarks. There have also been discussions about the relevance of pre-training and a brute-force approach to improve the model by increasing its size. Ilya Sutskever, former OpenAI chief scientist, had his say on this debate in his presentation at NeurIPS 2024. "Pre-training as we know it will unquestionably end," he said, referring to the lack of available data. "We have but one internet. You could even go as far as to say that data is the fossil fuel of AI. It was created somehow, and now we use it," he added. He also speculated that the use of inference time computing and synthetic data for training is a key technique that may help researchers overcome the problem. That said, if small models leverage new and innovative techniques and deliver high performance on resource-constrained devices, the smartphone market in 2025 will be the one to watch out for.

UAE's Falcon 3 challenges open-source leaders amid surging demand for small AI models

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More The UAE government-backed Technology Innovation Institute (TII) has announced the launch of Falcon 3, a family of open-source small language models (SLMs) designed to run efficiently on lightweight, single GPU-based infrastructures. Falcon 3 features four model sizes -- 1B, 3B, 7B, and 10B -- with base and instruct variants, promising to democratize access to advanced AI capabilities for developers, researchers, and businesses. According to the Hugging Face leaderboard, the models are already outperforming or closely matching popular open-source counterparts in their size class, including Meta's Llama and category leader Qwen-2.5. The development comes at a time when the demand for SLMs, with fewer parameters and simpler designs than LLMs, is rapidly growing due to their efficiency, affordability, and ability to be deployed on devices with limited resources. They are suitable for a range of applications across industries, like customer service, healthcare, mobile apps and IoT, where typical LLMs might be too computationally expensive to run effectively. According to Valuates Reports, the market for these models is expected to grow, with a CAGR of nearly 18% over the next five years. What does Falcon 3 bring to the table? Trained on 14 trillion tokens -- more than double its predecessor Falcon 2 -- the Falcon 3 family employs a decoder-only architecture with grouped query attention to share parameters and minimize memory usage for key-value (KV) cache during inference. This enables faster and more efficient operations when handling diverse text-based tasks. At the core, the models support four primary languages -- English, French, Spanish, and Portuguese -- and come equipped with a 32K context window, allowing them to process long inputs, such as heavily worded documents. "Falcon 3 is versatile, designed for both general-purpose and specialized tasks, providing immense flexibility to users. Its base model is perfect for generative applications, while the instruct variant excels in conversational tasks like customer service or virtual assistants," TII notes on its website. According to the leaderboard on Hugging Face, while all four Falcon 3 models perform fairly well, the 10B and 7B versions are the stars of the show, achieving state-of-the-art results on reasoning, language understanding, instruction following, code and mathematics tasks. Among models under the 13B-parameter size class, Falcon 3's 10B and 7B versions outperform competitors, including Google's Gemma 2-9B, Meta's Llama 3.1-8B, Mistral-7B, and Yi 1.5-9B. They even surpass Alibaba's category leader Qwen 2.5-7B in most benchmarks -- such as MUSR, MATH, GPQA, and IFEval -- except for MMLU, which is the test for evaluating how well language models understand and process human language. Deployment across industries With the Falcon 3 models now available on Hugging Face, TII aims to serve a broad range of users, enabling cost-effective AI deployments without computational bottlenecks. With their ability to handle specific, domain-focused tasks with fast processing times, the models can power various applications at the edge and in privacy-sensitive environments, including customer service chatbots, personalized recommender systems, data analysis, fraud detection, healthcare diagnostics, supply chain optimization and education. The institute also plans to expand the Falcon family further by introducing models with multimodal capabilities. These models are expected to launch sometime in January 2025. Notably, all models have been released under the TII Falcon License 2.0, a permissive Apache 2.0-based license with an acceptable use policy that encourages responsible AI development and deployment. To help users get started, TII has also launched a Falcon Playground, a testing environment where researchers and developers can try out Falcon 3 models before integrating them into their applications.