India's Balancing Act: Navigating Open and Closed Source GenAI Models

Not an open and shut case: India's balancing act between open and closed-source GenAI models

As India starts to emerge as the use-case capital of GenAI applications, tech leaders and startup founders believe both open and close-sourced technologies have a unique role to play in optimising costs, ensuring data sovereignty and delivering the best performance. ET's Himanshi Lohchab takes a deep dive on how India weighs in on pros and cons of open versus closed-source GenAI models.The Silicon Valley has lately been embroiled in a lobbying battle between big tech firms including Meta, Mistral and IBM on one side advocating for an "open science" approach to AI development that puts them at odds with rivals Anthropic, Microsoft and ChatGPT-maker OpenAI who are backing closed software. In fact, OpenAI is often trolled on social media for not being true to its name. But as India starts to emerge as the use-case capital of GenAI applications, tech leaders and startup founders believe both technologies have a unique role to play in optimising costs, ensuring data sovereignty and delivering the best performance. ET spoke to a cross section of startups and enterprises to understand the benefits and challenges of using open versus closed-source GenAI models. Cost of Deployment By definition, open-source software is where the source code is available to everyone in the public domain to use, modify, and distribute. Closed source, on the other hand, means the source code is restricted to private use and cannot be altered or built upon by users. In the context of GenAI, while open models are free and flexible, their cost of deployment may outrun the closed ones, industry experts say. "If you want to host an open model on premise, fine-tune it and customize it for organization-specific needs, there is a substantial infrastructure cost, LLMOps, lifecycle management and inferencing cost involved," said Arun Chandrasekaran, Gartner Distinguished VP Analyst. "Open-source models, while often free to access, can incur high deployment and maintenance costs due to technical requirements," said Rashid Khan, co-founder, of conversational AI startup Yellow.ai. Conversely, closed API models are often optimized, well-maintained, and continuously updated, which can save time and resources. "They provide a more turnkey solution with dedicated vendor support, ease of integration, and regular updates," he said. In fact, over the last 12 months, the pricing of closed LLM APIs has come down by 65-90%, which is helping startups to expand their margins, invest in research and development, improve performance, hire more talent as well as price their products competitively. "Deploying and fine-tuning an open-source model is a cost to the company and for initial stages of low volume or with some use cases that do not require 100% availability of the LLM, preferred approach is to go with paid APIs," said Ankur Dhawan, President - Product and Technology, edtech company upGrad. However, if an organization has the expertise and resources to manage the infrastructure optimally and technical skills to fine tune, they can eliminate the recurring costs associated with API access of closed models. For instance, Gnani.ai, which develops customer service chatbots and voice bots says the tokens consumed in a closed model often determine the costs. "In contrast, if you have a fine-tuned open model with your own proprietary data, you may have less to worry about regarding costs except for the deployment," said Ganesh Gopalan, Co-Founder and CEO, Gnani. ai. Performance AI researchers and scientists are divided over performance benchmarking. Stanford studies showed closed-source models still outperform their open-sourced counterparts. But common consensus suggests open models are catching up fast. The best closed models are getting better; and the best open models are getting better faster than them. "We had previously seen certain closed source models such as GPT-4o leading the charge in terms of quality, but open-source models are quickly catching up. For example, Meta's recently announced Llama 3.1 405B model is the same quality as other top models like GPT-4o and Claude Sonnet 3.5," said Baris Gultekin, Head of AI, Snowflake, US-based data cloud company which hosts top GenAI models. Meta, which is leading the open-source revolution with its Llama models, believes organizations must be more transparent about their evaluations. "We at Meta not only publish the benchmarks, but also the methodology that goes above and beyond what a lot of the other proprietary vendors do," said Ragavan Srinivasan, Vice President - Product Management, Meta. "...people aren't as open and transparent. There's just a scorecard. How are we evaluating this? Did you use 10 prompts? What are those 10 prompts? Are they all similar?" When it comes to critical applications, especially with agentic AI, controlling hallucinations and inaccuracies is the key priority for enterprises to preserve brand value. "It's about the trade-off between cost and quality," says Jonathan Frankle, Chief AI Scientist at Databricks, one among the highest valued AI companies. He added that cost takes many different forms. This could be speed or latency. "For coding assistants and real time chatbots, speed matters a lot." "What we've seen in the past few months is that this tradeoff has gotten better for everyone. For any given amount of money that you spend, you're going to get a better model," he said. Data sovereignty and privacy "When I speak with organizations, they often prefer open-source models," Snowflake's Gultekin said. "Companies want to bring AI closer to their data, ensuring that data security and privacy are upheld. Where the model runs is really important." For instance, banks are not comfortable with their data being exchanged to a public cloud where these closed models are accessible, says Gartner's Chandrasekaran. "But tomorrow if the same sits on-premise through some licensing arrangement, it could solve data security or privacy concerns," he added. Gnani.ai, which works with BFSI and healthcare organizations, said, for mission-critical applications, customers deploy on private clouds or even on-premise servers for customers in regulated industries in both US and India, Gopalan said. Open models are winning here. There's an intrinsic limitation to the amount of transparency one can provide with closed models, Databricks' Frankle said. "So, for me, as Databricks, I love the idea that I can sit down with a customer and walk them through what happens in our systems and how everything operates, and the model weights are available to them." "LLMs are black boxes, you can't exactly figure out why and how it does what it does," UpGrad's Dhawan said. "Hence, the decision on which one to use will depend on other factors like maintenance, availability, and cost of running the model. The trend is going to be similar to utilising the cloud for storing data as opposed to on-prem." Orchestration layer There is no one-size-fits-all. Therefore, organizations are experimenting with a multi-vendor strategy which orchestrates through cost, performance and security to offer the required outcomes. "A lot of CIOs or organizations are today looking at a multi-model strategy because they want to reduce vendor lock-in to any single provider. But the critical question is how I align the right model for right use-cases," Gartner's Chandrasekaran said. "So if you have a model garden, you need an orchestration tool in the backend, which not only optimizes my cost but also gives the best output through automated selection," he said. Microsoft Azure, the leading cloud provider which today hosts more than 1600 open and closed models, says "...factors like cost, efficiency, latency, and accuracy, make choosing the right model a critical step for enterprise adoption." "That's why Microsoft is committed to giving our customers flexibility in how they can develop and deploy customized AI solutions - either off the shelf or fine-tuned - through Azure AI," a company spokesperson said. Google, which has led the open-software revolution with Android OS powering millions of mobile devices, today, stands at a crucial intersection of open and closed systems when it comes to GenAI. It has a family of both open (Gemma) and closed (Gemini) models. In fact, back in 2019, Google scientists co-authored the seminal paper introducing the transformer architecture which today forms the basis of all GenAI models. It has also built key developer tools and infrastructure products like Transformer, TensorFlow and Jax needed for engineering these models. "Contributing openly to the research ecosystem is deeply embedded in our DNA...Decisions around whether to open source or release models to the public need to be made on a case-by-case basis," a Google spokesperson said. "It's important to carefully assess the balance of potential benefits and harms before releasing any AI systems to the general public."

India's generative AI startups look beyond building ChatGPT-like models

New Delhi: Sarvam and Adya.ai are two of India's newest startups targeting the red-hot generative artificial intelligence market, but they are also among a new breed of ventures looking beyond large language models like ChatGPT. The reason: industry-specific applications are easier to monetise and help avoid competition with AI giants. For Sarvam, seeking to be a full-stack generative AI firm, the approach is two-pronged. The startup, which raised $41 million in its first funding round from venture capital firms Lightspeed and Peak XV last year, is building vast large language models (LLMs) capable of understanding and processing human languages. It then uses these LLMs to build marketable products such as 'Agents' that are trained on domain-specific datasets, gaining expertise in fields such as healthcare, financial services and law. "In healthcare, for instance, a generative AI assistant can offer neonatal advice to pregnant women at a cost of around ₹1 -- no physical doctor's time can be this affordable," Pratyush Kumar, cofounder of Sarvam and adjunct faculty at Indian Institute of Technology (IIT), Madras, told Mint last month. "Instead of taking clientele away from doctors, in India, such domain-specific AI models can make healthcare more accessible to a wide population base." The approach Sarvam, Adya.ai and some of their peers are taking marks a sobering change after an initial AI euphoria to create general purpose generative AI models. Competing with Microsoft-owned OpenAI's ChatGPT or Google's Gemini will not only be expensive but tough given the headstart they have. AI applications solving smaller problems offer a better chance at success. "It's important for startups to solve specific, targeted problems," said Ankush Sabharwal, cofounder of CoRover, which has created BharatGPT. "Even within healthcare, the nature of services needed for metro markets is different from that in villages. Because of this, taking a domain-specific AI model approach is key -- building 'India-focused' models is also too large, and Big Tech is too big a competitor for this." Adya.ai, which raised $1.2 million in a pre-Series A funding round from the Indian Angel Network collective, is training its models to power ready-to-deploy AI assistants for e-commerce and retail companies to serve as customer service agents, said Shayak Mazumder, chief executive officer at the startup. "This is only the first domain that we're targeting, and we'll expand to more domains in future." According to Kashyap Kompella, technology analyst and founder of tech consultancy RPA2AI, building domain-specific AI applications and sub-models for enterprises "is the ideal sweet spot from development cost, market opportunity and monetization approaches." He cited parallels with the evolution of the IT industry where several companies built major expertise in applications deployed on platforms such as SAP and Salesforce. "The reason for this is that enterprise applications on such platforms are difficult to build, require experienced talent to be hired, and are long-term projects," he said. "The story is somewhat similar for generative AI." This approach is helping India's early-stage startups earn revenue from their generative AI projects. CoRover, for instance, expects annual revenue to close in on $5 million by FY25. Both Sarvam and Adya.ai also have paying enterprise clients. New LLM-based generative AI applications are seen as a natural evolution to early renditions of chat automation. Startups such as the Bengaluru-headquartered Yellow.ai and Pune-based E42.ai have built conversational agents for over five years now. With generative AI, the likes of Sarvam and CoRover are offering more interactive and less expensive AI agents in local Indian languages, which can help businesses expand customer support to multiple languages. Building sector-specific solutions, however, has its own challenges. CoRover's Sabharwal said, "We do not have enough targeted data for each and every domain," he said. "Because of this, we're running a platform where enterprise customers can choose among BharatGPT, OpenAI's GPT family and Google's Gemini as the foundational models, and build virtual assistants based on any model as deemed fit. The data for the specific domain is brought by the client itself." According to Sarvam's Kumar, while there is lack of data, especially in Indic languages, the problem has a solution, too. "There are various publicly accessible data sources that we use to collate data and operate AI models. The cost of running domain-specific AI models is not the problem," he said. "The only issue is that this will take some time to become mature, and be fully deployable within the domains with lesser and lesser margins for errors."