Groq CEO Advises India to Focus on AI Application and Fine-tuning, Not Foundational Models

India should just fine-tune the AI models that exist: Groq CEO Jonathan Ross

India must focus on building artificial intelligence applications and fine-tuning already existing models instead of spending top dollars on developing foundational models and AI chips, says Jonathan Ross, cofounder and chief executive of Groq, a Silicon Valley-based AI chip startup seen as a challenger to Nvidia. The company, which is betting on "inference economies", or generating insights from available data, is creating a peering network between India and its data centre in Saudi Arabia that has brought live 19,000 AI inferencing clusters within eight days, Ross tells ET's Himanshi Lohchab in an interview. Ross, who cofounded the Tensor Processing Unit (TPU) at Google, is mildly annoyed with people confusing Groq with Elon Musk's AI models 'Grok' and says his company has sent a 'cease and desist' notice to the Tesla boss, asking him to stop using the name. Edited excerpts: What is Groq's outlook on the India market? With AI, India will have a population advantage. All of a sudden, 1.5 billion people who already know how to use computers can just use their entrepreneurial talent to build things. Because now you don't need to learn software. India is in a unique position. Many other people around the world have built training hardware. Now those models are being given away for free. They're spending billions of dollars training them, giving them away for free. And if I were India, I would focus on taking those models that were made available for free. I would collect up enough of the roughly 25 languages that are spoken here. So that we could fine-tune the models that already exist. Make them better for those languages. And then, I would do inference. Should India build its own foundational model? There's no reason that you shouldn't do that as a backup plan. But other than the Googles, the Metas, the Microsofts, all the others have abandoned building their own models. Because every time they are ready to put it into production, there is a better model for free that they can just download. It's become a commodity. Usually, training will take six months. And there's a new model coming out every month. So why would you ever start from a zero and a disadvantage? Just take all the data that you have and train one of those existing models that's being given away for free. Fine-tuning is very different. It's much less expensive. Requires a lot less time. From a hardware perspective, we have less than 2% compute capacity of the world... Oftentimes, developing countries will have better infrastructure than the United States. You're not going to be stuck and saddled with the older technology that we're using in Silicon Valley. You get to do something brand new; you might end up with an advantage over everyone else. So, I would start by thinking how to build AI infrastructure not as a copy of the US? How do we do it for scale? How do we do it for economics? And then, how do we replicate this in the rest of the world? India is also aiming to develop its own AI chip. What are your thoughts on that? We're happy to build chips wherever the fabs are. We currently build our chips in the US and South Korea rather than Taiwan. If India had fabs, we'd build them here. But if you try and replicate everything that's been done all over the world, are you getting good use of the money? Why not try and win in an area rather than trying to replicate something that someone else has done? If you want to build your own fabs, you're probably signing up to $100-500 billion effort. Wouldn't you rather have $100-500 billion worth of chips that you could then use to make all of the Indian population productive and able to build their own businesses and compete in the world? We can build the chips in the US. You can build the tech companies here in India. That builds mutual interdependency, which builds peace and prosperity. Don't copy what China did and try to decouple. How do you see the relationship between the rise of agentic AI and the need for inferencing hardware, especially for an application-focused country like India? We have over 150,000 developers developing on console.grok.com in India today for free. We give more tokens away for free per day than GCP (Google Cloud Platform). For comparison, in the US, it's closer to 250,000. Now, the US is supposed to be in the lead on AI. But why is India catching up so quickly? That's because creating and using AI models are two different things. In fact, 95% of the people developing on Groq have never trained a model in their life. They just use it. And India should focus on what's next, which is using the models that others have created. Take the ones that exist. Fine-tune them for India. Small amount of effort. And then start building on top of it. India is a very cost-sensitive market and that's ideal for us. We want to bring the cost of intelligence to zero because the cheaper it gets, the more you're going to use. Just like the electricity in your home. We want to help India bring as much capacity online as possible. We just deployed 19,000 chips in Saudi Arabia with Aramco. And so now we're working on some peering arrangements (to directly exchange traffic between the networks in India and Saudi) to start using that capacity. Eventually, we'll probably even deploy some in India. How do you see competition from companies like Nvidia? We believe you should never ever build something that's competitive. By copying something that someone else has built, you're not solving an unsolved customer problem. Nvidia has built the ideal chips for training. We can't do better than that. Maybe we could match them, but what's the point? That's already a solved problem. We want to make the thinking (inferencing of models) fast. When you use OpenAI for reasoning, you often have to wait 40 seconds or even two minutes to get an answer. Just imagine if every time you did a Google search, you'd be standing in the street looking up an address and waiting for two minutes. That'd be useless. What we've done is we've built a system that is 20 times faster than a GPU. How often do people confuse your company for Elon Musk's Grok model? The confusion is real. We call dibs on Groq. We have the trademark, so we have legal protection. It is ours. When Elon announced his version with the K, we sent him a 'cease and desist'. He said that he named it after 'Hitchhiker's Guide to the Galaxy'. He got the book wrong. Apparently, he hadn't read the books because it's actually from a book called 'Stranger in a Strange Land'. He was literally just copying us and he had no idea what he was talking about. We recommended that he rename it Slartibartfast. It's actually one of the characters in the book that he mentioned. What does the DeepSeek moment mean for AI hardware? We've been talking about the Jevons paradox a year before Satya (Nadella) tweeted about it. The lower the cost of intelligence, the more people are going to buy. I actually think the amount of money spent on training is going to go up, but it follows the Jevons paradox. The cost to run an LLM hasn't really changed much in the last year. The cost for the equivalent amount of intelligence has actually gone down. Do you think shutting out China will hinder AI innovation going forward? India has banned TikTok, right? Why? Concerns of data security. So, the problem we had was that the DeepSeek model was being hosted in China very inexpensively, which meant that people would be sending their very sensitive data to China. And we decided to host the DeepSeek model on our hardware to prevent that data from going to China. Now, Groq is very clear on our data privacy. We do not retain people's data. We don't even have hard drives to store it on. Remember what they did was distillation of the OpenAI model. They did do innovation. But they also collected together a bunch of techniques that had already been developed but not been used by the open-source models yet. That was going to happen soon, but they got there first. So they were benefiting from what OpenAI did. How did you navigate fund shortages and how are you preparing for overcoming such situations in future? In Silicon Valley, the best companies start in the worst of financial times. Google started in 1998. Facebook started in 2003, but they had to weather the 2008 storm. Amazon was a little before the 2000 crash. Why is that? Well, I have a term that I call financial diabetes. When you get too much money, you don't learn how to spend it efficiently. Then you fail and you don't know how to earn it. One of the things that we do, we focused on high volume, low margin inference in part because we want to stay healthy. We don't want to be in the situation where we are making 70%-80% margins like some of the other companies in this space. When you make 70%-80% margins, you don't know how to operate more leanly. Our goal is to build out the inference platform for the world. We want to make sure that everyone has access to AI, that it's affordable, that no one is left out. To do that, we need to be very financially disciplined. Groq went through a couple of hard times. There was a point when we went to all our employees and said that we're running low on money. Would you be willing to trade your salaries for more equity? We called these Groq bonds. About 80% of employees participated. 50% went to the statutory minimum salary by law. I actually went below what was legal. I went to $1. Had we not done that, Groq wouldn't exist. Out of 150 employees, I think we only lost five at that time. We had less attrition when we told people that things were going to be difficult. People want to be part of a team. They want to feel trusted. When you are vulnerable with your employees, when you tell them the reality, when you bring them in and then they start trying to solve the problems with you, then they feel part of it, just like you are. They're not going to abandon you.

India Should Just Fine-tune the AI Models that Exist

India must focus on building artificial intelligence applications and fine-tuning already existing models instead of spending top dollars on developing foundational models and AI chips, says Jonathan Ross, cofounder and chief executive of Groq, a Silicon Valley-based AI chip startup seen as a challenger to Nvidia. India must focus on building artificial intelligence applications and fine-tuning already existing models instead of spending top dollars on developing foundational models and AI chips, says Jonathan Ross, cofounder and chief executive of Groq, a Silicon Valley-based AI chip startup seen as a challenger to Nvidia. The company, which is betting on bringing down the cost of AI by scaling the inferencing infrastructure, is creating a peering network between India and its data centre in Saudi Arabia that has brought live 19,000 AI inferencing clusters within eight days, Ross tells Himanshi Lohchab in an interview. Ross, who co-developed the Tensor Processing Unit (TPU) at Google, is mildly annoyed with people confusing Groq with Elon Musk's AI models 'Grok', and says his company has sent a 'cease and desist' notice to the Tesla boss, asking him to stop using the name. Edited excerpts: What is Groq's outlook on the India market? With AI, India will have a population advantage. All of a sudden, 1.5 billion people who already know how to use computers can just use their entrepreneurial talent to build things. Because now you don't need to learn software. India is in a unique position. Many other people around the world have built training hardware. Now those models are being given away for free. They're spending billions of dollars training them, giving them away for free. And if I were India, I would focus on taking those models that were made available for free. I would collect up enough of the roughly 25 languages that are spoken here. So that we could fine-tune the models that already exist. Make them better for those languages. And then, I would do inference. Should India build its own foundational model? There's no reason that you shouldn't do that as a backup plan. But other than the Googles, the Metas, the Microsofts, all the others have abandoned building their own models. Because every time they are ready to put it into production, there is a better model for free that they can just download. It's become a commodity. Usually, training will take six months. And there's a new model coming out every month. So why would you ever start from a zero and a disadvantage? Just take all the data that you have and train one of those existing models that's being given away for free. Fine-tuning is very different. It's much less expensive. Requires a lot less time. From a hardware perspective, we have less than 2% compute capacity of the world... Oftentimes, developing countries will have better infrastructure than the United States. You're not going to be stuck and saddled with the older technology that we're using in Silicon Valley. You get to do something brand new; you might end up with an advantage over everyone else. So, I would start by thinking how to build AI infrastructure not as a copy of the US? How do we do it for scale? How do we do it for economics? And then, how do we replicate this in the rest of the world? India is also aiming to develop its own AI chip. What are your thoughts on that? We're happy to build chips wherever the fabs are. We currently build our chips in the US and South Korea rather than Taiwan. If India had fabs, we'd build them here. But if you try and replicate everything that's been done all over the world, are you getting good use of the money? Why not try and win in an area rather than trying to replicate something that someone else has done? If you want to build your own fabs, you're probably signing up to $100-500 billion effort. Wouldn't you rather have $100-500 billion worth of chips that you could then use to make all of the Indian population productive and able to build their own businesses and compete in the world? We can build the chips in the US. You can build the tech companies here in India. That builds mutual interdependency, which builds peace and prosperity. Don't copy what China did and try to decouple. How do you see the relationship between the rise of agentic AI and the need for inferencing hardware, especially for an application-focused country like India? We have over 150,000 developers developing on console.grok.com in India today for free. We give more tokens away for free per day than GCP (Google Cloud Platform). For comparison, in the US, it's closer to 250,000. Now, the US is supposed to be in the lead on AI. But why is India catching up so quickly? That's because creating and using AI models are two different things. In fact, 95% of the people developing on Groq have never trained a model in their life. They just use it. And India should focus on what's next, which is using the models that others have created. Take the ones that exist. Fine-tune them for India. Small amount of effort. And then start building on top of it. India is a very cost-sensitive market and that's ideal for us. We want to bring the cost of intelligence to zero because the cheaper it gets, the more you're going to use. Just like the electricity in your home. We want to help India bring as much capacity online as possible. We just deployed 19,000 chips in Saudi Arabia with Aramco. And so now we're working on some peering arrangements (to directly exchange traffic between the networks in India and Saudi) to start using that capacity. Eventually, we'll probably even deploy some in India. How do you see competition from companies like Nvidia? We believe you should never ever build something that's competitive. By copying something that someone else has built, you're not solving an unsolved customer problem. Nvidia has built the ideal chips for training. We can't do better than that. Maybe we could match them, but what's the point? That's already a solved problem. We want to make the thinking (inferencing of models) fast. When you use OpenAI for reasoning, you often have to wait 40 seconds or even two minutes to get an answer. Just imagine if every time you did a Google search, you'd be standing in the street looking up an address and waiting for two minutes. That'd be useless. What we've done is we've built a system that is 20 times faster than a GPU. Other than (Google, Meta, Microsoft) all (cos) have abandoned building their own (AI) models. Because every time they are ready to put one into production, there is a better model for free ... It's become a commodity .