General Compute secures $400M to challenge Nvidia with faster, cheaper AI inference chips

Why the first GPU financiers are turning to inference chips in a $400 million deal

General Compute, an AI inference cloud startup, has landed a $400 million loan from Upper90, a tech investment firm. It might be the first deal to put up inference-specific chips as collateral -- chips built to run already-trained AI models quickly and efficiently, rather than the more expensive chips used to build the models in the first place. The financing is the latest signal that markets are responding to concerns over the price of AI tools and tokens by turning to infrastructure that runs open-source models more cheaply than the newest LLMs from frontier labs. Founded by CEO Finn Puklowski, General Compute raised a $15 million seed round in May to build an inference neocloud around silicon from SambaNova, an Intel-backed chipmaker. (Neoclouds are purpose-built for AI workloads, unlike the general-purpose infrastructure offered by traditional hyperscalers like AWS or Azure.) The company's SN50 chips are designed for inference. They're power-efficient and don't require expensive water-cooling systems, which means they can be deployed more quickly than GPUs across a larger variety of data centers. General Compute says the new chips will provide 16 times faster inference than GPU-based clouds. The challenge is getting a lot of these chips, especially when you're a brand-new company. Upper90 co-founder and CEO Billy Libby, a former Goldman Sachs quantitative trader, had a playbook for this: In 2021, his firm financed GPU purchases by Crusoe, the energy-focused data center startup, which he believes was the first loan against the value of advanced chips. Traditional lenders eschewed such deals at the time because of the risks and uncertainties around GPU depreciation. But as CoreWeave made chips-backed loans into a business model and then the basis of a blockbuster IPO, this kind of financing has become common. "When we financed Nvidia GPUs as the first group to do that, the market was inefficient," Libby told TechCrunch. "We could really put together something as an early participant, and kind of get compensated for the risk." Now that GPUs are comparatively well understood and perhaps over-bought, Upper90 is turning to companies like General Compute to ride the next wave of the AI boom. "We think open source models are going to be important, and we went and looked for a player last year that was in inference," Libby said. "Everyone doesn't need a supercomputer, but they do need inference and AI." That thesis has been growing stronger, with companies that provide access to open models, like OpenRouter and Fireworks, raising new rounds at huge valuations. New models like Kimi's K3, recently just this week, have proven to compete with the latest releases from Anthropic and OpenAI on coding benchmarks. And new chipmakers like Groq and Cerebras have drawn interest from acquirers and public markets alike. General Compute's ability to access chips outside of Nvidia's ecosystem matters for the same reason. TensorWave, another AI infrastructure company, is making a similar bet on a partnership with AMD. As more alternatives to Nvidia emerge, compute providers that aren't locked into Nvidia deals may have an advantage in providing cost-efficient inference. "There are a bunch of chips that are starting to scale that have amazing [total cost of ownership], or that can operate much faster than Nvidia, but there's not too many buyers for them," Puklowski said. "By getting together with Upper90, this is not just, 'a cool startup got some money to buy some compute.' Like, this is the first signal of capital organizing itself and the fragmenting of Nvidia's monopolistic dominance."

Inference cloud operator General Compute raises $400M in debt financing

Artificial intelligence infrastructure startup General Compute Inc. today announced that it secured $400 million in debt financing. Upper90, the investment firm that is underwriting the round, will initially provide the company with $100 million. General Compute plans to draw down additional funds as customer demand increases. Unlike most AI infrastructure providers, General Compute is not using Nvidia Corp. graphics cards. The company has based its cloud platform on chips from Advanced Micro Devices Inc. and SambaNova Inc., a fellow startup that recently raised $1 billion in funding. AI models answer user prompts through a two-step workflow. They start by extracting the meaning of a prompt, a step known as the prefill phase. They then generate a response one token at a time in a process known as decoding. The decoding phase requires AI models to move a significant amount of data between the host chip's memory and processing circuits. SambaNova's accelerators are specifically optimized to speed up that workflow. As a result, they can perform decoding faster than some Nvidia graphics cards. The speed at which data moves between a chip's memory and processing modules depends on the physical distance between the circuits. The closer they are to each other, the less time it takes data to complete the trip. The memory and processing circuits in SambaNova's chips are placed immediately next to one another, which minimizes data travel times. General Compute uses SambaNova silicon to run decoding calculations. It performs the other step involved in inference workflows, the prefill phase, using AMD's MI300X graphics card. The chip contains 12 dies and 8 memory stacks that together contain 153 billion transistors. The MI300X and SambaNova's chips are designed to run in standard air-cooled server racks. As a result, General Compute can add capacity to its cloud without investing in the pricey liquid cooling equipment that most graphics cards require. That translates into faster infrastructure upgrades. General Compute enables customers to access its inference hardware through two services. The first offers managed versions of open-source language models. Developers can access the models through an application programming interface modeled after the one used by OpenAI Group PBC, which eases the learning curve. General Compute's second service provides dedicated infrastructure environments with more computing capacity. The company will use its debt financing to expand its cloud. According to a document on its website, General Compute has inked agreements that give it the option to purchase 15 megawatts worth of air-cooled rack capacity in co-location facilities. The company expects that infrastructure to be sufficient for its fourth quarter growth plans.