Baseten Secures $75M Funding to Boost AI Inference Performance and Efficiency

Baseten grabs $75M to crank up high-performance inference for AI workloads - SiliconANGLE

Baseten grabs $75M to crank up high-performance inference for AI workloads BaseTen Labs Inc., an artificial intelligence startup that's focused on high-performance inference for large language models and other AI applications, said today it has closed on a $75 million Series C funding round led by IVP and Spark Capital. The round, which also saw participation from Greylock, Conviction, South Park Commons, 01 Advisors and Lachy Groom, brings the startup's total amount raised to $135 million. Baseten has created an AI inference platform that's used by enterprises to run LLMs either in the cloud or on their own, on-premises infrastructure. For AI applications to scale, they need access to extremely fast and reliable inference, which refers to the process of querying models and computing a response. Yet for many organizations, this has proven challenging, with the required high-performance graphics processing units that power inference often hard to come by. Even the best-funded companies can struggle to access these resources, and if they can't find enough, the result will be poorly performing applications and, occasionally, even downtime. Add to that, the shortage of GPUs can also mean paying inflated costs. Instead of operating its own data centers, Baseten relies on public cloud infrastructure from providers such as Amazon Web Services Inc., Google Cloud and Microsoft Corp. By combining the resources of these cloud platforms, it says it can provide better access to GPUs. In addition, customers can also run the company's software in their own data centers. Baseten's platform provides everything required to get high-performance inference up and running, including a vast library of proprietary and open-source models, modern tooling and workflows for deploying, managing, scaling and maintaining LLMs in production, and the multi-cluster, multicloud infrastructure needed to scale across regions and AI model modalities. It also provides access to applied research, so customers can use the latest techniques and frameworks to squeeze better performance and cost-efficiency out of their AI applications, plus specialist AI engineers to assist customers in deploying those apps. In an interview with CNBC, Baseten's co-founder and Chief Executive Tuhin Srivastava said one of the main advantages his company provides is that it can guarantee access to GPU resources. While many companies do deploy their AI models alone, most will struggle to get enough GPUs in the right geographical location, he said. In addition, customers also experience frustration with last minute warnings that some of the GPUs they're using will be moved into maintenance mode, meaning they become unavailable at the drop of a hat. Baseten has enough resources to avoid these kinds of disruptions, Srivastava said. "In this market your number one differentiation is how fast you can move," he claimed. "That is the core benefit for our customers." In addition, Baseten can also save its customers tons of cash, claiming that the average customer sees its inference costs drop by around 40% once they start using its services, in addition to better performance. That might explain how the company has been able to grow its revenue by more than six-times in the last 12 months, though it didn't provide any concrete numbers regarding its sales. On the other hand, it did claim to have more than 100 enterprise customers on its books, including the crowdfunding platform Patreon Inc., the AI startup Writer Inc. and the video editing company Descript Inc. Srivastava said his customers prioritize the ability to bring high-quality AI products to market quickly, and choose Baseten to ensure that happens. "Speed, reliability, and cost-efficiency are non-negotiables, and that's where we devote 100% of our focus," he added. Spark Capital's general partner Will Reed said that if an AI product hasn't yet experienced problems with inference, it's because it hasn't managed to hit any real scale thus far. "Every successful AI project needs exceptional inference performance, or nobody wants to use it," he explained. "If you're betting the future of your product or company on that performance, choosing the right partner is make or break."

AI startup Baseten raises $75 million following DeepSeek's emergence

From left, Baseten founders Amir Haghighat, Tuhin Srivastava, Philip Howes and Pankaj Gupta. Baseten, a startup that runs artificial intelligence models for clients on their cloud infrastructure, has raised $75 million in funding, the company said Wednesday. The funding round values Baseten at $825 million and demonstrates that venture capitalists believe tech's AI boom stands to benefit a plethora of startups, not just those building large language models. In recent months, OpenAI, Anthropic and xAI have raised billions in funding, with much of the money going toward servers containing Nvidia graphics processing units, or GPUs. After companies finish training AI models on reams of data, they need to deploy those models somewhere at the inference stage, which is when models generate outputs in response to user queries. That's when Baseten comes in. Rather than run its own data centers, Baseten runs its software on data center equipment from cloud providers, including Amazon and Google. Customers can supply their own infrastructure with an enterprise tier. By drawing from multiple providers, Baseten offers access to more GPUs than a single cloud's current supply. "In this market, your No. 1 differentiation is how fast you can move. That is the core benefit for our customers," co-founder and CEO Tuhin Srivastava said. "You can go to production without worrying about reliability, security and performance." Companies can manage the deployment of their models without Baseten, but securing enough Nvidia chips in the right geographical areas can prove difficult, co-founder Amir Haghighat told CNBC. Cloud providers sometimes inform customers that some GPUs will be moved into maintenance mode and become unavailable within minutes. Baseten helps its clients handle those instances without interruptions, Srivastava said. After the January breakthrough of Chinese AI lab DeepSeek, which claimed its models were trained for a fraction of the costs as its U.S. counterparts, efficiency in AI has become more important than ever. Baseten was quick to add support for DeepSeek's R1 reasoning model that compares to OpenAI's o1. Baseten's website promises top-tier performance at a fraction of OpenAI's cost. There has been a lot of inbound from organizations looking at switching to DeepSeek, and Baseten has been busy trying to keep up, Srivastava said. "There are a lot of people paying millions of dollars per quarter to OpenAI and Anthropic that are thinking, 'How can I save money?'" he said. "And they've flocked." Baseten clients often see their inference costs fall 40% or more, while receiving better performance, in comparison with homegrown architectures, head of marketing Mike Bilodeau wrote in an email. The startup's revenue for the fiscal year that ended in January was six times more than it was in the prior year, Srivastava said, without providing a dollar figure. Founded in 2019 and based in San Francisco, Baseten has about 60 employees. Existing investors IVP and Spark Capital led the new round, with others participating. More than 100 enterprises are customers, along with hundreds of smaller companies, such as Descript, Patreon and Writer. Competitors include Salesforce-backed Together AI. Another challenge is that Baseten must compete with AI model companies and hedge funds for talent. "Having more money in somewhat of a weird economic environment, it does not hurt," Srivastava said.