Indian Startup Ziroh Labs Unveils Kompact AI: Running Large AI Models on CPUs

Indian Startup Unveils System to Run AI Without Advanced Chips

Ziroh Labs, an artificial intelligence startup operating in India, collaborated with researchers at the country's premier technology school to design an affordable system that it says can run large AI models without requiring advanced computing chips from the likes of Nvidia Corp. The company's framework, called Kompact AI, was developed in partnership with the Indian Institute of Technology, Madras. Ziroh Labs said the platform enables AI to run on the central processing units (CPUs) found in everyday computing devices as opposed to the coveted, and costly, graphics processing units (GPUs) that have been the linchpin of the artificial intelligence boom.

IIT Madras Launches Centre of AI Research, Unveils Kompact AI for CPU-Based AI

Kompact AI includes a model library spanning text, speech, vision and multimodal models, all tuned to run on CPUs. IIT Madras and the IITM Pravartak Technologies Foundation have partnered with California-based Ziroh Labs to establish the Centre of AI Research (CoAIR) to solve India's compute accessibility challenges using AI models optimised for CPUs and edge devices. The announcement was made during an event held at IIT Madras, where Ziroh Labs unveiled the first version of Kompact AI, an AI platform that enables foundational models to be built and served on CPUs, eliminating the need for GPUs. The collaboration is aligned with India's 'AI for All' mission. Kompact AI has been developed by Indian scientists and engineers and is already capable of running 17 optimised models, including DeepSeek, Qwen and Llama, on CPUs. These have been benchmarked for both performance and accuracy in collaboration with IIT Madras. "By making AI models executable on CPUs, we are unlocking the potential of AI without the heavy infrastructure costs typically associated with GPUs," Madhusudhanan Baskaran, principal consultant at IITM Pravartak Technologies Foundation, said. "Through the CoAIR, we will build specific, actionable AI models that can be deployed at scale, benefiting people in rural and underserved areas." V Kamakoti, director of IIT Madras, said the initiative reflects a nature-inspired approach."They provide a platform that uses customised and trained domain-specific models to provide accurate inferences on affordable conventional compute machines. This is a major step in arresting the possible AI divide between one who can afford the modern hyperscalar systems and one who cannot." Ziroh Labs demonstrated Kompact AI's capabilities at the event, which was attended by dignitaries including Turing awardee Whitfield Diffie, Sun Microsystems co-founder Scott McNealy, and others from academia and industry. The platform supports offline operation, making it suitable for deployment in remote regions with limited internet access. Hrishikesh Dewan and Igneta DSouza, co-founders of Ziroh Labs, stated, "For AI to benefit everyone, it must be accessible, affordable, and adaptable to real-world conditions. Kompact AI is designed for such needs and operates efficiently on CPUs, optimised for task-specific applications." Kompact AI includes a model library spanning text, speech, vision and multimodal models, all tuned to run on CPUs. It also provides a runtime environment -- Common AI-Language Runtime (ICAN) -- that supports over 10 programming languages for developers to work with these models. The platform specifically targets models with fewer than 50 billion parameters, optimising inference and fine-tuning without compromising quality. "Anything that can be computed on a GPU can also be calculated on a CPU," the release noted. Kompact AI avoids the common drawbacks of quantised and distilled models, which typically trade off accuracy for efficiency. Sowmya Narayanan Sadagopan, former director of IIIT Bangalore, noted, "By combining the availability of AI development and deployment tools using CPUs, Kompact AI proves India can do frugal innovation at the cutting edge of technology. Such a solution using far less electrical power is sustainable." Meanwhile, Apoorv Iyer, executive vice president at HCLTech, said the company will partner with Ziroh Labs in this initiative. "We congratulate Ziroh Labs on its AI CPU launch, which promises to significantly increase the accessibility of AI." CoAIR aims to support specific use cases such as agriculture, education, and rural healthcare by deploying smaller, task-specific models on conventional devices.

Indian Startup Unveils System to Run AI Without Advanced Chips

The team of researchers showed their product working on a laptop Ziroh Labs, an artificial intelligence startup operating in India, collaborated with researchers at the country's premier technology school to design an affordable system that it says can run large AI models without requiring advanced computing chips from the likes of Nvidia. The company's framework, called Kompact AI, was developed in partnership with the Indian Institute of Technology, Madras. Ziroh Labs said the platform enables AI to run on the central processing units (CPUs) found in everyday computing devices as opposed to the coveted, and costly, graphics processing units (GPUs) that have been the linchpin of the artificial intelligence boom. A growing number of AI developers have touted efficiency gains that let them use fewer chips in the months following the viral success of China's DeepSeek, which purportedly built a competitive AI model for a fraction of the cost of its US peers. Ziroh Labs' approach primarily focuses on the process of inference, or operating AI systems after they've been trained. Ziroh Labs said it can optimize leading AI models to run on personal computers. In a demonstration event this week, the team of researchers showed their product working on a laptop that uses a shelf-bought Intel Xeon processor and querying models such as Meta Platforms' Llama 2 and Alibaba Group Holding's Qwen2.5. Other tech firms have also used CPUs to handle some inference workloads. Ziroh Labs said its approach leads to high-quality results. The startup said its technology has been tested by US chipmakers Intel and Advanced Micro Devices. "It's going to have a very profound market impact in the years ahead," said William Raduchel, former chief strategy officer of Sun Microsystems and a tech adviser to the startup who spoke virtually at the event. As in other countries, developers in India have struggled to afford, and get access to, top-of-the-line Nvidia chips to help build and support AI products. The shortage of GPUs risks hindering the speed and scale of local AI research and deployment. "The AI divide is because only those with high-end expensive GPU-powered resources can access, develop and deploy powerful AI," said V. Kamakoti, director at the Indian Institute of Technology, Madras. "We're demonstrating that you don't need a revolver to kill a mosquito."

Indian startup unveils system to run AI without advanced chips

India's Ziroh Labs, in collaboration with IIT Madras, has developed Kompact AI -- a system enabling large AI models to run on standard CPUs instead of costly GPUs. Tested by Intel and AMD, the platform aims to bridge the AI divide by making advanced AI more accessible and affordable for developers.Ziroh Labs, an artificial intelligence startup operating in India, collaborated with researchers at the country's premier technology school to design an affordable system that it says can run large AI models without requiring advanced computing chips from the likes of Nvidia Corp. The company's framework, called Kompact AI, was developed in partnership with the Indian Institute of Technology, Madras. Ziroh Labs said the platform enables AI to run on the central processing units (CPUs) found in everyday computing devices as opposed to the coveted, and costly, graphics processing units (GPUs) that have been the linchpin of the artificial intelligence boom. A growing number of AI developers have touted efficiency gains that let them use fewer chips in the months following the viral success of China's DeepSeek, which purportedly built a competitive AI model for a fraction of the cost of its US peers. Ziroh Labs' approach primarily focuses on the process of inference, or operating AI systems after they've been trained. Ziroh Labs said it can optimize leading AI models to run on personal computers. In a demonstration event this week, the team of researchers showed their product working on a laptop that uses a shelf-bought Intel Xeon processor and querying models such as Meta Platforms Inc.'s Llama 2 and Alibaba Group Holding Ltd.'s Qwen2.5. Other tech firms have also used CPUs to handle some inference workloads. Ziroh Labs said its approach leads to high-quality results. The startup said its technology has been tested by US chipmakers Intel Corp. and Advanced Micro Devices Inc. "It's going to have a very profound market impact in the years ahead," said William Raduchel, former chief strategy officer of Sun Microsystems and a tech adviser to the startup who spoke virtually at the event. As in other countries, developers in India have struggled to afford, and get access to, top-of-the-line Nvidia chips to help build and support AI products. The shortage of GPUs risks hindering the speed and scale of local AI research and deployment. "The AI divide is because only those with high-end expensive GPU-powered resources can access, develop and deploy powerful AI," said V. Kamakoti, director at the Indian Institute of Technology, Madras. "We're demonstrating that you don't need a revolver to kill a mosquito."

ETtech Explainer: How Ziroh Labs is trying to eliminate expensive GPUs

An academia-startup partnership is trying to eliminate expensive and complex GPUs, specialised circuits that boost intensive tasks such as AI, paring cost and driving accessibility.Home-grown bootstrapped AI startup Ziroh Labs, in collaboration with IIT Madras, last week launched Kompact, an AI platform that could help solve India's compute accessibility challenges by eliminating the need for scarce and expensive graphics processing units (GPU). While Kompact will potentially democratise AI in the country, some founders are waiting for more information about the initiative to see how far it can be a gamechanger. Annapurna Roy explains all about the initiative. What is Kompact AI and how does it work? The platform enables building and deploying high-performance AI efficiently on central processing units (CPUs) instead of on GPUs. Seventeen AI models including DeepSeek, Qwen and Llama have been optimised by Ziroh Labs to run efficiently on CPUs with their technical innovation. It enables inferencing and fine-tuning for models with less than 50 billion parameters, offering about three times better performance than the current CPU-based frameworks. The main goal is to enhance accessibility, Ziroh Labs cofounder and CEO Hrishikesh Dewan told ET, adding that most real-world use cases are within 50 billion parameters. Running generic larger models on CPUs is not the focus, he said. The platform will sign up at least 200 startups in the next two quarters -- free if needed -- for about a yearto have more validations and support use cases across healthcare, banking and finance, education. A pricing strategy will follow, although interested enterprises wanting to sign up now would have to pay. Dewan explained that existing distillation and quantisation methods may help AI run on CPUs, but they lead to reduced performance and accuracy while Kompact's new technique preserves this. Startups leveraging the platform can run their AI use cases on the cloud, on-premise or even on laptops, based on their needs and scale. Is it a game-changer? "Kompact AI actually claims that they're able to keep the performance of the model the same, without any compromise. If that is true, then it is indeed a breakthrough," said Pawan Prabhat, cofounder, Shorthills AI. He also said that many companies and researchers have been working on how to use CPUs instead of GPUs. It could potentially bring down overall costs of AI as all companies have lots of CPUs, he said, and this would benefit startups as well as large enterprises. Dewan said that CPU prices may also be driven down if more people demand them for AI over time. What are the challenges? AI CPUs are suitable for personal or limited use but scaling inference for large markets like India without compromising performance could be challenging, cautioned the founder of an AI model development startup. "Unless the data is seen or the models are actually used by a few companies and we have independent validation on the performance, I think that's one challenge we need to address,"said Prabhat.

IIT-Madras, Ziroh Labs Launch CPU-Based AI Platform to Bridge India's Compute Divide

New Delhi: In a move to tackle India's artificial intelligence compute accessibility gap, IIT-Madras and its technology arm, IITM Pravartak Technologies Foundation, have partnered with California-based deep-tech startup Ziroh Labs to launch a Centre of AI Research. At the heart of the initiative is 'Kompact AI' -- a cost-effective central processing unit (CPU)-based AI platform designed to eliminate dependence on costly graphic processing units (GPUs) and make high-performance AI more affordable and scalable, especially for low-resource settings. Kompact AI allows foundational models to be trained and deployed using conventional CPUs rather than GPUs, which are expensive and in short supply. It has been developed by scientists and engineers from India. The platform has already been used to optimise models such as DeepSeek, Qwen and Llama for CPU inferencing, with benchmarking support from IIT-Madras researchers. "This effort is certainly a major step in arresting the possible AI divide between one who can afford modern hyperscalar systems and one who cannot," said IIT-Madras director V Kamakoti. "This effort by Ziroh Labs and IITM Pravartak is a nature-inspired one wherein they provide a platform that uses customised and trained domain-specific models to provide accurate inferences on affordable conventional compute machines." The centre will focus on developing smaller, task-specific AI models that can be deployed in rural and underserved regions, where infrastructure for high-end computing is often lacking. Kompact AI, which can run without continuous internet connectivity, is being pitched as a solution for sectors such as agriculture, education and healthcare -- areas where low-cost, on-device AI could make the biggest difference. Highlighting the need for such a solution, Madhusudhanan B, principal consultant at IITM Pravartak, said: "By making AI models executable on CPUs, we are unlocking the potential of AI without the heavy infrastructure costs typically associated with GPUs. This is about taking technology beyond labs and into the hands of those who need it most." According to Ziroh Labs, Kompact AI delivers up to three times the performance of current CPU-based AI frameworks without compromising model accuracy -- something previous attempts at quantisation or distillation have struggled to achieve. "For AI to benefit everyone, it must be accessible, affordable, and adoptable to real-world conditions. Kompact AI is designed for such needs, and operates efficiently on CPUs, optimised for task-specific applications. This ensures that communities and enterprises with limited infrastructure can leverage the power of AI to create practical, impactful solutions," Ziroh Labs cofounders Hrishikesh Dewan and Igneta D'Souza said in a joint statement. Kompact AI includes a CPU-optimised model library supporting text, speech, vision and multimodal applications, and offers a developer runtime called ICAN (Common AI-Language Runtime) that supports over 10 programming languages. The company hopes this will attract a global developer community focused on building AI applications that don't rely on energy-intensive GPU stacks.

IIT Madras & IIT-Madras Pravartak Foundation partners with Ziroh Labs to establish the Centre of AI Research to solve AI for those with accessibility challenges

Kompact AI operates without constant internet connectivity, making it ideal for deployment in remote areas with limited or no access to reliable internet. Elaborating on the need for such developments, Prof. V. Kamakoti, Director, IIT Madras, said, "Nature has taught us that one can effectively acquire knowledge and subsequently infer in only a limited set of domains. Attempts to acquire everything under the universe are not sustainable and bound to fail over a period of time." Prof. V. Kamakoti added, "This effort by Ziroh Labs and IITM Pravartak is a nature-inspired one wherein they provide a platform that uses customized and trained domain-specific models to provide on affordable conventional compute machines. This effort is certainly a major step in arresting the possible AI divide between one who can afford the modern hyper scalar systems and one who cannot." Kompact AI can play a transformative role as it leverages multiple optimisations algorithmically and to execute models on the low end without any support required for external communication, such as the Internet. As a part of this upcoming CoAIR, IIT Madras will work on smaller AI models for specific use cases, leveraging Kompact AI to optimise execution, making high-performance AI more efficient, accessible, and relevant to real-world applications. Dr. Madhusudhanan B, Principal Consultant - IITM Pravartak Technologies Foundation, added, "At IITM Pravartak, our mission is to translate research innovations into tangible solutions that address real-world problems. Kompact AI is a game-changer in this regard. By making AI models executable on CPUs, we are unlocking the potential of AI without the heavy infrastructure costs typically associated with GPUs. Through the CoAIR, we will build specific, actionable AI models that can be deployed at scale, benefiting people in rural and underserved areas across India. This is about taking technology beyond the labs and putting it into the hands of those who need it the most." Speaking at the launch of Kompact AI, Prof Sadagopan, Former Director - IIIT-Bangalore, said, "By combining affordability/availability of AI development and deployment tools - using CPUs without excessive dependence on GPUs - Kompact AI proves once again that India can do frugal innovation at the cutting edge of technology. Even more interesting is the fact that such a solution using far less electrical power is sustainable. It is also aligned with India's AI Mission, "AI for All" - from India for the world." "We congratulate Ziroh Labs on its AI CPU launch which promises to significantly increase the accessibility of AI. HCLTech is excited to partner with Ziroh Labs on this exciting journey", said Apoorv Iyer, Executive Vice President and Global Head, GenAI Practice, HCLTech. According to Hrishikesh Dewan and Igneta DSouza Co-founders, Ziroh Labs said, "AI has the potential to transform every aspect of our lives, from improving healthcare outcomes to enhancing education and solving agricultural issues. However, for AI to benefit everyone, it must be accessible, affordable, and adaptable to real-world conditions. Kompact AI is designed for such needs and operates efficiently on CPUs, optimized for task-specific applications. This ensures that communities and enterprises with limited infrastructure can leverage the power of AI to create practical, impactful solutions. Our collaboration with IIT-Madras and IIT-M Pravartak Foundation will aim to address multiple challenges. We are grateful for this partnership, as teams from IITM have been pillars of support during the entire Kompact AI development journey." This AI platform aims to democratise AI by allowing developers across the globe to build, train and infer AI using CPUs commonly found in Cloud Data Centers and Edge Devices. Kompact AI enables AI development without violating any data privacy and data residency regulations across the globe.