OpenAI's GPT-3.5 Turbo Update and India's AI Landscape: Balancing Innovation and Challenges

OpenAI's cheaper mini brings a mega issue for LLMs

Startup exectives said OpenAI's pricing could bring massive cost savings for scaling up simple use-cases, such as multilingual chatbots where a single query ranges between 15-25 tokens. OpenAI has also allowed calling multiple APIs (application programming interface), which means a user can pass contextual data from multiple sources at the same time to answer a complex query.OpenAI's pricing for its first-ever small language model GPT4o Mini--which is 60% cheaper than GPT 3.5 Turbo and 40% cheaper than Google's Gemini 1.5 Flash--has raised the competition bar among Indian makers of LLMs (large language models) who are building small and concise use-cases for organisations. Executives of startups said OpenAI's pricing could bring massive cost savings for scaling up simple use-cases, such as multilingual chatbots where a single query ranges between 15-25 tokens. OpenAI has also allowed calling multiple APIs (application programming interface), which means a user can pass contextual data from multiple sources at the same time to answer a complex query. "Besides chatbots, companies across various sectors can benefit from lower-priced yet advanced LLMs for multiple use cases. For instance, legal documentation summaries and compliance adherence in pharmacovigilance can leverage literature reviews," said Sreeraman Thiagarajan, founder and chief executive of Agrahyah Technologies, which works on top language and diffusion models. "Multilingual translation at scale is useful in education, book publishing and generating synthetic data for machine learning purposes. Large enterprises can use AI to analyse large volumes of unstructured data, such as social media conversations, to distil consumer insights," he said.

GenAI: why not play desi beats

The whole narrative around large language models (LLMs), referred to as foundation/frontier models - pertinent for GenAI - is currently driven by the models developed by the US tech industry such as ChatGPT and Gemini. The next biggest reason why Indic LLMs are needed is due to the geopolitical reasons. In the last 18 months, India has seen the first prototypes of at least half a dozen foundational/fine-tuned large language models (LLMs) such as Sarvam's OpenHathi and Ola's Krutrim. This even as their global counterparts like OpenAI's GPT-4o and Meta's Llama have started showing enterprise grade support for Hindi, Marathi, Kannada and other Indian languages across text and voice. This development, though inevitable, raises questions about the role of Indian LLMs and what problems they can solve. Building an Indic AI model from scratch is fraught with challenges. There is not enough training data, the cost of compute is humongous, and, most importantly, funding is not easy as there is a lack of clarity on use cases. Is there, however, a need for Indian LLMs? Absolutely, say experts. Indian LLMs The whole narrative around large language models, referred to as foundation/frontier models - pertinent for GenAI - is currently driven by the models developed by the US tech industry such as ChatGPT and Gemini. This may be concerning, experts said. "We have a huge reliance on the top 4-5 models, which are trained on Western data from North America and Western Europe, and, therefore, they embed cultural values from that region," said James Landay, cofounder, Stanford Institute for Human - centered Artificial Intelligence. He said countries like India and South Korea should ensure they have models of their own that represent their cultural values on top of which they want to build their models on. Vishnu Vardhan, founder of SML GenAI, which launched Indic LLM platform Hanooman, said this becomes important when developing sovereign GenAI applications in the areas of defence, cybersecurity and banking, that requires Indian cultural nuance, privacy and security. The next biggest reason why Indic LLMs are needed is due to the geopolitical reasons. Vivek Raghavan, cofounder of SarvamAI, said AI could become a geopolitical game in the future. "If tomorrow they (the West) decide that these models are not accessible, then what do you do?" he asked. This idea is perhaps what attracted investors to fund companies building Indic AI models. In December 2023, Sarvam AI raised $41 million from Lightspeed, Peak XV and Khosla Ventures. Krutrim became the first Indian AI unicorn when it raised $50 million in a round led by Matrix Partners at a valuation of $1 billion within months of its formation. All these companies, however, have had to work around various limitations to create their GenAI models. Tough road Some of the biggest challenges while building an LLM from India includes the fact that training data is small and fractionalised, given the spread of the Internet in India is relatively a recent phenomenon. The multiplicity of languages spoken in India also adds to the mammoth task. This is not the case for foreign frontier models that are trained on billions of parameters and primarily on English, making them more efficient, accurate and relatively reliable. For instance GPT3.5 has 175 billion parameters and its next iteration reportedly has close to 1.8 trillion parameters. "The important question is, what can you achieve with the model, not the size of the model," said Sarvam's Raghavan, who is also one of the chief architects of the Aadhaar project. Sarvam has built 7-13-billion parameter small language models, which Raghavan claimed is accurate at lower costs. "We are solving for a very specific section, which is the last-mile user. India is largely an application market, and, therefore, we want to keep the model's intent very narrow and focused," he said. IT services major Tech Mahindra, which launched the Indus Project, has created a 1.2-billion parameter model trained in Hindi and its 37 plus Indian dialects. "The most critical factor is to confine the data sets to the use case. The narrower you go, the narrower your need for data sets and, therefore, the need for the model to consume computation," said Kunal Purohit, president of next gen services at TechM. Sourcing data While OpenAI has not revealed the size of its training dataset, its GPT-3 model is believed to be trained on 45 terabytes of publicly available text. In the case of Indian languages, only 0.1% of text is available online and that is either literature or poor-quality public information, Sarvam's Raghavan said. Also, in most Indian languages, including Hindi and Kannada, the formal, written language is very different from the colloquial dialect people use in everyday conversation, experts said. "Therefore, textual data is not enough... Conversational data is critical for training Indic LLMs," said Pranav Mistry, founder of Two AI, backed by Reliance Jio and South Korea's Naver Corp. It launched ChatSUTRA trained in eight languages including Hindi, Gujarati, Japanese, Greek, and Arabic. Cost of compute It takes millions to train AI models in terms of money and resources. While Indian models are miniscule in size and do not require as much as their Western counterparts, they offer a different challenge. "If I say a sentence - for example, 'who is the prime minister of India' - that sentence in English may take, say, six tokens. But if I say 'Bharat ka pradhan mantri kaun hai' in Hindi, that same thing takes 18 tokens. So, it is three times more expensive," Sarvam's Raghavan explained. "If I finetune Llama, my cost is going to be three times more expensive in Hindi than English, and eight times more expensive in Korean than English," he said. Indian language models are using different approaches to bring that cost down, including the choice of GPU chips. Pricing competitiveness Ola and AI cloud Krutrim founder Bhavish Aggarwal recently made headlines when he announced that he is offering Krutrim AI cloud compute at as low as Rs 109 per hour for Nvidia A100 GPUs and at Rs 290/hour for the H100s. Hiranandani Group's data centre operator Yotta Data Services said it could offer high-power computation at between $1.5 and $5 per hour, which is among the lowest pricing in the world. ChatSUTRA claims that it costs $0.22 for a million tokens compared to GPT4o, which costs $15 for a million tokens. "If you see the cost differences, we can bring it down by 5-30 times," Two AI's Mistry said. However, even as Indian infrastructure and AI service companies offer globally competitive prices, the real challenge is finding takers. Lack of use cases Most of the enterprises have started piloting GenAI solutions using a mixture of OpenAI, Meta and Google. Indian companies have yet to see that kind of adoption. A Bengaluru-based founder said they were not sure if there really is a need for more LLMs in the country when the focus should be on building applications, and also given the limitations they operate in. Rohit Pandharkar, AI partner, EY said that initial enthusiasm around Indic LLMs came from the idea that not many of 140 crore Indians were English speakers and to cater to this investors saw the need to fund Indic LLM startups. But with global players now offering Indian language support, that theme has lost steam and large funding such as that of a Sarvam is unlikely to happen today, he added. However, Hanooman's Vardhan, who was cited earlier, said Indic models can do what ChatGPT is doing and they are much cheaper to implement, and there is a huge potential for Indian companies here. He said use cases will happen only when people work with companies to build different models as enterprises will not see return of investment if they are spending a lot on large frontier models. "This is where a lot of work is happening on bringing smaller, more efficient and precise models," Vardhan said. Sarvam said it is piloting many projects, but did not share specifics on what they are.