Modular Raises $250M to Challenge Nvidia's AI Software Dominance

A Former Apple Luminary Sets Out to Create the Ultimate GPU Software

At a certain point between building Apple's developer tools, leading a core part of Google's AI infrastructure team, and clashing with Elon Musk during a stint as Tesla's Autopilot chief, Chris Lattner's vision for his life's work started to come into focus. AI was taking over the world, and demand was growing for the chips that powered it. But the software stack for those chips was dominated by just a few big companies. Would developers be able to easily run their code across all the different chips dotting the AI landscape? Lattner's answer to that question is Modular, a software startup he founded in 2022 with his former Google colleague Tim Davis. Modular makes a unifying software layer that helps cloud businesses squeeze as much juice as possible out of GPUs and CPUs -- the high-powered chips that underpin generative AI. The startup has also built a new coding language, based on Python, that lets developers use a single language to build AI apps that run across multiple GPUs and CPUs. Modular's basic premise is that if a developer builds an app for one chip, they shouldn't have to jump through hoops in order to run it on another vendor's chip. But Modular's long-term goal is even more ambitious: to loosen the software choke hold that companies like Nvidia and AMD have on the industry, and become the de facto software for AI chips. "Our thesis is that the need for compute power is just exploding, but there is no unified compute platform," Lattner says. "Sovereign AI will be everywhere. There will be many Stargates. But there will be different types of chips optimized for different use cases, and there needs to be a unified layer for that." There are early signs that Modular's thesis bears out. AI giants like Nvidia, AMD, and Amazon have partnered with the startup to test the waters. The GPU cluster company SF Compute also worked with Modular to build what they claim is the world's cheapest API for large AI models. As of this week, Modular's developer platform now supports Apple Silicon GPUs, in addition to Nvidia and AMD chips. Building on this momentum, Modular just raised $250 million in venture capital funding, its third round of financing in three years, bringing its total valuation to $1.6 billion. The round was led by the Pittsburgh-based US Innovative Technology Fund. DFJ Growth also invested, along with existing investors General Catalyst, Greylock, and GV (formerly known as Google Ventures). "We've spent a bunch of time and energy trying to figure out what makes a startup in this space interesting, and with every company that has tried to build their own chip -- and even the big players, like AMD and Nvidia -- it all comes back to the software," says Dave Munichiello, managing partner at GV. "Chris convinced me that the software was the most interesting and valuable problem to address." It might be valuable -- but it's also extremely complicated. Part of that complication stems from Nvidia's closed ecosystem. Nvidia's chips make up the vast majority of the GPU market, but the company's 20-year-old proprietary software platform, CUDA, keeps developers locked in. AMD's software platform for high-performance computing, called ROCm, differs in that it's open source. This allows developers to more easily move code to different chips.

AI startup Modular raises $250 million, seeks to challenge Nvidia dominance

Sept 24 (Reuters) - AI startup Modular said on Wednesday it raised $250 million in a funding round valuing it at $1.6 billion, as it aims to challenge Nvidia's software stranglehold on the AI computing market. The funding, which nearly tripled the company's valuation from two years ago, was led by U.S. Innovative Technology fund. DFJ Growth and all existing investors, including GV, General Catalyst and Greylock also joined the round. Founded in 2022 by veteran engineers at Apple and Google, Modular's platform is designed to allow developers to run their AI applications across a variety of computer chips without having to rewrite code for each one. It now serves cloud providers such as Oracle and Amazon, as well as chipmakers Nvidia and AMD. Nvidia controls over 80% of the high-end AI chip market, partly thanks to its proprietary CUDA software, which locks over 4 million global developers into its ecosystem. Modular says it has adopted what it calls a "Switzerland" strategy to be a neutral software layer. Chris Lattner, co-founder and CEO of Modular, said the goal is not to defeat the market leader. "What we're focused on is not like pushing down Nvidia or crushing them. It's more about enabling a level playing field so that other people can compete," he said. It plans to sell the software directly to enterprises on a consumption basis and through revenue-sharing partnerships with cloud providers. The company's strategy has attracted investors betting on a multi-vendor future for AI hardware. Sam Fort, partner at DFJ Growth, compared Modular to "VMware for the AI era", which supported companies to work across CPUs. "Modular is trying to create the AI hypervisor that will allow you to port workloads across different vendors," Fort said. The company, with about 130 employees, plans to use the new capital to expand its engineering and go-to-market team. The funds will also help the company expand from its current focus on AI inference into the AI training market. Reporting by Krystal Hu in San Francisco; Editing by Lincoln Feast. Our Standards: The Thomson Reuters Trust Principles., opens new tab * Suggested Topics: * Business Krystal Hu Thomson Reuters Krystal reports on venture capital and startups for Reuters. She covers Silicon Valley and beyond through the lens of money and characters, with a focus on growth-stage startups, tech investments and AI. She has previously covered M&A for Reuters, breaking stories on Trump's SPAC and Elon Musk's Twitter financing. Previously, she reported on Amazon for Yahoo Finance, and her investigation of the company's retail practice was cited by lawmakers in Congress. Krystal started a career in journalism by writing about tech and politics in China. She has a master's degree from New York University, and enjoys a scoop of Matcha ice cream as much as getting a scoop at work.

Start-up Modular raises $250mn to challenge Nvidia's software dominance

A team of Silicon Valley veterans who previously worked at Apple and Google has raised $250mn to challenge Nvidia's dominance of the software tools used to programme artificial intelligence chips. Palo Alto-based Modular, which is building an alternative to Nvidia's Cuda software that has been vital to keeping customers locked to the chipmaker's AI products, announced it has received the new investment at a $1.6bn valuation on Wednesday. The funding will boost its efforts to build "Android for generative AI hardware", said the start-up's co-founder and chief executive Chris Lattner, likening its toolkit to the Google mobile operating system that boosted competition to Apple's iPhone. The new investment nearly triples Modular's valuation since its last round in 2023. The $250mn financing was led by US Innovative Technology, the fund headed by former Legendary Entertainment owner Thomas Tull, which has previously backed US defence tech group Anduril and AI cloud start-up Lambda. While Nvidia's Cuda software only works with the chipmaker's own hardware, Modular's system allows developers to create AI models and apps that can run on the latest graphics processing units from Nvidia, AMD and Apple's Macs, allowing them to switch between chips more easily. DFJ Growth, a major backer of SpaceX and xAI, is also joining the round alongside existing investors such as Alphabet's venture unit GV and General Catalyst. "Modular is addressing the most urgent challenge in AI: unifying the compute layer by enabling diversified processing hardware and software to operate cohesively," said Sam Fort, partner at DFJ Growth. But the start-up has an uphill struggle. Nvidia is also involved in a whirlwind of deals, such as its latest pledge to invest $100bn in OpenAI, which are seen as an attempt to ensure its chips and Cuda platform remain at the centre of the AI industry. Nvidia's GPUs today command an estimated 80 per cent of the booming market for AI data centre systems -- a dominance that has propelled the chipmaker to become the world's most valuable company. "The industry is locked into this Cuda world," said Lattner, who previously led Apple's developer tools team for several years and launched its Swift programming language. "This is a big challenge for developers, because people want choice. People want to be able to run and build AI, and put it on the hardware where it makes the most sense." Lattner co-founded Modular in 2022 alongside Tim Davis, after they worked together on Google's own custom AI chip, dubbed Tensor Processing Units, and other semiconductor projects at Google Brain, the Big Tech group's AI lab. Lattner argues that other industry attempts to build cross-compatible alternatives to Cuda -- including open source projects backed by the likes of OpenAI, Meta, Google and Microsoft, as well as efforts from chipmakers Intel and AMD -- have failed because each Big Tech company is trying to protect its own interests. The business charges corporate customers based on usage when they run AI tasks on its software on cloud computing platforms such as Amazon Web Services and Oracle. Several new AI cloud computing start-ups are also using Modular to run their data centres using multiple GPU suppliers, Davis said. Even though Modular presents a challenge to one of Nvidia's key differentiators, the chipmaker is also working as a partner with the start-up. "Nvidia has wonderful products, wonderful GPUs, but we want more competition to enable even better silicon," said Davis. "You shouldn't win because you have enormous market power, you should win because you have the best product."

Modular raises $250M to simplify AI deployment across hardware - SiliconANGLE

Artificial intelligence startup Modular Inc. said today it has raised $250 million in its third financing round, valuing the company at $1.6 billion. The round was led by Thomas Tull's US Innovative Technology fund, with DFJ Growth joining and with participation from all existing investors, including Google Ventures, General Catalyst and Greylock Ventures. The funding brings the total raised by the company to $380 million. Founded in 2022, Modular provides a platform that allows developers to run AI applications across different computer chips -- including central processing units, graphics processing units, application-specific integrated circuits and custom silicon -- without the need to rewrite or migrate code. Over the past three years, the company has built a software infrastructure layer and a specialized programming language designed to let enterprises deploy AI models across a mix of chips and servers. Modular's long-term goal is to provide AI deployment for enterprise users, addressing the current fragmented ecosystem that necessitates writing specialized code for each specific architecture. The company's platform is an enterprise-grade AI inference stack that abstracts away hardware. "When we founded Modular, we believed that the world needed a unified platform for AI, and today, that vision is more important than ever," said Chris Lattner, Chief Executive of Modular, in a statement. Nvidia Corp. currently dominates the AI accelerator market. Its Hopper and newer Blackwell architectures are estimated to power between 70% and 95% of AI data-center GPUs. That dominance is reinforced by CUDA, Nvidia's proprietary programming framework, which has become the de facto standard for AI development thanks to its powerful parallel computing capabilities. Challengers exist, most notably Advanced Micro Devices Inc., which produces Instinct AI accelerators and maintains the open-source ROCm software stack. But because so many developer tools and inference platforms are written for CUDA, migrating to ROCm is often difficult, leaving AMD at a disadvantage. That's where Modular sees its opportunity to loosen this vendor lock-in by giving enterprises more freedom to choose their hardware. Its platform already supports architectures from Nvidia, AMD and Apple's custom silicon. The company says its latest release delivers performance gains of 20% - 50% compared with leading frameworks such as vLLM and SGLang on next-generation accelerators, including Nvidia's B200 and AMD's MI355. This vision appears to be resonating: AMD, Nvidia, and Amazon.com Inc. have all joined as ecosystem partners. Modular has also teamed up with AI application developers such as Inworld AI to accelerate speech synthesis and the San Francisco Compute Company, which operates a GPU cluster marketplace. The company has grown to more than 130 employees, with headquarters in the San Francisco Bay Area. With the new funding, Modular plans to expand hiring across North America and Europe, scale its cloud platform, extend support for cloud and edge hardware, and broaden its focus from inference into AI training.

AI startup Modular raises $250 million, seeks to challenge Nvidia dominance - The Economic Times

The funding, which nearly tripled the company's valuation from two years ago, was led by U.S. Innovative Technology fund. DFJ Growth and all existing investors, including GV, General Catalyst and Greylock also joined the round.AI startup Modular said on Wednesday it raised $250 million in a funding round valuing it at $1.6 billion, as it aims to challenge Nvidia's software stranglehold on the AI computing market. The funding, which nearly tripled the company's valuation from two years ago, was led by U.S. Innovative Technology fund. DFJ Growth and all existing investors, including GV, General Catalyst and Greylock also joined the round. Founded in 2022 by veteran engineers at Apple and Google, Modular's platform is designed to allow developers to run their AI applications across a variety of computer chips without having to rewrite code for each one. It now serves cloud providers such as Oracle and Amazon, as well as chipmakers Nvidia and AMD. Nvidia controls over 80% of the high-end AI chip market, partly thanks to its proprietary CUDA software, which locks over 4 million global developers into its ecosystem. Modular says it has adopted what it calls a "Switzerland" strategy to be a neutral software layer. Chris Lattner, cofounder and CEO of Modular, said the goal is not to defeat the market leader. "What we're focused on is not like pushing down Nvidia or crushing them. It's more about enabling a level playing field so that other people can compete," he said. It plans to sell the software directly to enterprises on a consumption basis and through revenue-sharing partnerships with cloud providers. The company's strategy has attracted investors betting on a multi-vendor future for AI hardware. Sam Fort, partner at DFJ Growth, compared Modular to "VMware for the AI era", which supported companies to work across CPUs. "Modular is trying to create the AI hypervisor that will allow you to port workloads across different vendors," Fort said. The company, with about 130 employees, plans to use the new capital to expand its engineering and go-to-market team. The funds will also help the company expand from its current focus on AI inference into the AI training market.