In an effort to tackle the rising costs and sustainability challenges associated with generative AI, India-based GenAI research lab Bud Ecosystem has introduced an innovative new product, Bud Runtime. Designed to streamline the deployment of generative AI applications, Bud Runtime enables deployment on CPU-based infrastructure, offering a more affordable, scalable and sustainable option for organizations looking to harness the power of generative AI.
Since late 2023, Bud Ecosystem has been collaborating with companies like Intel, Microsoft, and Infosys to commoditize generative AI and make it easily accessible to organizations worldwide. Bud Runtime significantly reduces both capital and operational expenditures for organizations adopting generative AI, without compromising application performance. It enables developers, startups, companies, and research institutions to kickstart their generative AI initiatives at a cost as low as $200 per month.
In addition to supporting CPU-based inference, Bud Runtime also supports a broad range of hardware -- including GPUs, HPUs, TPUs, and NPUs -- from major vendors like Nvidia, Intel, AMD, and Huawei. One of the key innovations in Bud Runtime is its support for heterogeneous cluster parallelism, which allows organizations to utilize a mix of their existing hardware -- including CPUs, GPUs, HPUs, and other architectures -- for deploying generative AI workloads and easily scale as more compute resources become available. Enabling organizations to mitigate GPU shortages and lower operational costs of running generative AI applications. Bud Runtime is currently the only platform on the market offering this level of heterogeneous hardware parallelism and clustering.
"We began our GenAI journey in early 2023 and quickly encountered the high cost of GPUs. To tackle this, we built the first version of the Bud runtime to run smaller models on our existing infrastructure. Since then, we've evolved it to support mid-size models on CPUs and added compatibility with hardware from Nvidia, AMD, Intel, Huawei, and more -- thereby reducing costs and addressing hardware scarcity. As we saw others facing similar barriers, we decided to productise the technology to help startups, enterprises, and researchers adopt GenAI more efficiently." -- Jithin V.G, CEO, Bud Ecosystem, said.
Bud Ecosystem focuses on fundamental AI research, particularly in efficient transformer architectures for low-resource scenarios, decentralized models, hybrid inference, and AI inference optimization. The company also has published multiple research papers and released over 20 open-source models to its credit. Bud is also the only startup from India to have topped the Hugging Face LLM leaderboard for building a large language model on par with GPT-3.5 at the time.
For the past 18 months, Bud Ecosystem has been working with Intel to make production-ready GenAI inference possible on CPUs, especially their Xeon lineup. This collaboration was later extended to support Intel Gaudi accelerators as well. In addition to this partnership, the research lab has also joined hands with global technology companies like Microsoft, LTIM, and Infosys to help organizations around the world adopt Generative AI in a cost-effective and scalable way.
"Our mission is to democratize GenAI at scale by commoditizing it. This is only possible if we can use commodity hardware for GenAI at scale. To achieve this, we need to further enhance inference technology and develop better model architectures that require less parallel compute and memory bandwidth. Most of our research and engineering efforts are focused on this mission. We also intend to make these products and research available to everyone through permissive open-source projects. We have an exciting new open-source project coming up early next month."
It is a known fact that Generative AI has been making significant technological advancements of late. However, it remains very costly for companies to adopt. Only large companies are currently able to adopt and experiment with Generative AI. In addition, there is an ongoing scarcity of GPUs, which further limits accessibility. Only large companies are currently able to adopt and experiment with Generative AI. For those that do, projects often get stuck at the minimum viable product (MVP) stage and rarely progress to full production deployment. It is in this context that Bud Runtime proves itself beneficial for enterprises, as they look to usher in cost-effectiveness in the adoption of Gen AI.
