OpenAI Develops First In-House AI Chip in Collaboration with Broadcom and TSMC

OpenAI Develops Its First "In-House" AI Chip, Collaborating With TSMC & Broadcom To Enhance Inferencing

OpenAI has reportedly built its first in-house custom AI chip in collaboration with Broadcom and TSMC, as the AI giant is looking to upscale its inferencing capabilities. OpenAI's First AI Chip Will Reportedly Target Inferencing Workloads, Expected To Debut With Cutting-Edge Performance OpenAI is probably one of the only firms in the industry that has presented massive ambitions when scaling AI compute power, whether through a network of "fabrication facilities" or even through developing in-house solutions that are more effective than existing counterparts. Sam Altman is known to experiment with the "AI hype," and now, according to a new report by Reuters, OpenAI has developed its first AI chip, collaborating with the chip designer Broadcom and none other than TSMC, the semiconductor giant. The report states that the firm's newest chip is targeted towards inferencing workloads, and while the industry is currently focused on model training and enhancements, the future definitely lies in bringing inference capabilities in the LLM models out there. Interestingly, it is revealed that OpenAI has dropped the idea of building a "network" of foundries and shifted the focus towards in-house chip design since the latter requires less financial resources and execution time. OpenAI is said to be building a hybrid model of "AI compute acquisition," which means that the firm is planning to expand its AI capabilities through the integration of existing architectures, such as those from NVIDIA and AMD, along with developing in-house solutions, to ensure diversity in workloads and reduce dependency on its partners. Since it has key relations with NVIDIA, OpenAI would certainly not offer its in-house chips in markets, although this isn't certain for now, and it depends on how successful OpenAI's first chip endeavor turns out. In terms of the actual details of the AI chip, the report doesn't mention specifics apart from the fact that OpenAI has managed to bring in TSMC onboard to cater to semiconductor needs, and the chip could debut in the industry by 2026, depending upon how OpenAI decides to proceed with it. It was reported previously that the OpenAI's chip might be based on TSMC's A16 Angstrom process, hence revealing that the ambition would target the higher-end market segment. OpenAI has reportedly secured a chip team of about 20 people, including individuals who have worked on Google's TPUs (Tensor Processing Units), so OpenAI's AI chip ambition is undoubtedly backed by the right people on the team. Given that we are in an era where every major tech giant is in pursuit of developing their well-equipped AI computing portfolio, it is certainly imminent that companies push towards developing in-house solutions to ease off the pressure on the AI supply chain, along with bringing in customization in existing workloads.

OpenAI to Launch First In-House AI Chip in 2026, Partners with Broadcom and TSMC

OpenAI is partnering with Broadcom and Taiwan Semiconductor Manufacturing Company (TSMC) to create its first in-house chip to support the company's artificial intelligence infrastructure, Reuters reported on Wednesday. The company has also chosen to diversify its chip supply by adding AMD's MI300X chips alongside NVIDIA's GPUs, which dominate the market. OpenAI had previously considered raising vast sums -- reportedly up to $7 trillion -- to build its own network of chip foundries. However, the cost and complexity of this ambitious project led the company to pause building such a network and instead focus on designing chips with industry partners. With a growing need for AI compute power, OpenAI joins industry leaders like Google and Amazon in pursuing a mix of internal chip design and external sourcing. Broadcom, an experienced partner in chip design and manufacturing, has helped OpenAI secure production capabilities with TSMC for its first custom chip, expected to launch in 2026. Analysts predict the chip, focused on inference tasks, will help OpenAI meet future needs as demand grows for applying AI models in real-time tasks rather than exclusively for training purposes. Currently, OpenAI heavily relies on NVIDIA's GPUs, which dominate the AI chip market with over 80% share. However, supply chain constraints and rising costs have led the company to explore partnerships with other chip suppliers. AMD, which projects $4.5 billion in AI chip sales for 2024, is also now supplying OpenAI through Microsoft's Azure. The Reuters report added that OpenAI has been mindful of maintaining positive relations with NVIDIA, avoiding aggressive recruiting from the chip giant while continuing to use its high-performance GPUs. Despite projected 2024 revenues of $3.7 billion, OpenAI anticipates a $5 billion loss due to substantial hardware, electricity, and cloud service expenses. By securing a broader supply chain, the company seeks to manage costs and optimise resources as it scales its AI services, including ChatGPT.

OpenAI reportedly is making its first AI chip with TSMC and Broadcom

OpenAI is undoubtedly a winner in the current artificial intelligence boom -- and now it's reportedly developing its own chips to power its growth. The ChatGPT maker is working with Taiwan Semiconductor Manufacturing Company (TSM+2.01%) and Broadcom (AVGO+4.69%) to make its own in-house AI chips as demand for infrastructure grows, Reuters reported, citing unnamed people familiar with the matter. It reportedly also is adding Advanced Micro Devices's (AMD+4.41%) AI chips to diversify supply from Nvidia (NVDA+1.03%). The company has reportedly considered ways to diversify its supply of chips and cut costs, including by raising capital for a network of chipmaking factories. But instead of the plan for foundries -- which it has dropped for now -- OpenAI is focusing on designing its own chips in-house, Reuters reported. OpenAI's work with Broadcom on its first in-house chips for AI inferencing has been going on for months, people told Reuters, adding that while demand is currently higher for AI training chips, analysts expect AI inferencing to eventually take the lead. Inferencing, which comes after training, is the process of a trained AI model making predictions from new data. Two people told Reuters that OpenAI is still deciding between either developing or acquiring other features for its AI chip, and could find more chipmaking partners. OpenAI has been able to establish manufacturing capacity with TSMC through its partnership with Broadcom, Reuters reported. The Taiwanese chipmaker reportedly could have OpenAI's first custom-designed chip by 2026, but that is not certain. Neither OpenAI, TSMC, nor Broadcom immediately responded to a request for comment. In September, OpenAI released a series of "reasoning" AI models that it said are "designed to spend more time thinking before they respond." OpenAI completed a $6.6 billion funding round earlier this month, giving it a valuation of $157 billion.

OpenAI reportedly working with Broadcom and TSMC to develop custom AI inference chip - SiliconANGLE

OpenAI reportedly working with Broadcom and TSMC to develop custom AI inference chip OpenAI is reportedly working with Broadcom Inc. and Taiwan Semiconductor Manufacturing Co. Ltd. to build a new artificial intelligence chip specifically designed to run AI models after they've been trained. The decision by OpenAI comes after the company has examined a range of options to diversify chip supply and reduce costs. According to Reuters, OpenAI is said to have considered building everything in-house and raising capital for an expensive plan to build its own foundries but has since dropped the plan due to the cost and time needed to do so, instead now pursuing plans to focus on in-house chip design efforts. Bloomberg reports that the decision to pursue in-house chip design is part of OpenAI not looking to replace GPUs such as those provided by Nvidia Corp., but to design a specialized chip that will undertake inference, the AI process of applying trained models to make predictions or decisions on new data in real-time applications. The same report notes that investors and analysts expect the need for chips to support inference will only grow as more tech companies use AI models to undertake more complex tasks. The decision by OpenAI to work with partners on custom chips is said by sources quoted by Bloomberg to be one made due to it being a quicker and more attainable path for now. However, the company may continue research on setting up its own network of foundries in the future. OpenAI's move into in-house chip design is part of a broader trend among major tech firms and AI companies to develop specialized hardware that can meet the unique demands of AI workloads more efficiently than general-purpose GPUs. Currently, OpenAI relies heavily on Nvidia's GPUs for training its models, a process that requires immense computational power to refine algorithms using vast datasets. However, inference requires different chip capabilities that are optimized for speed and energy efficiency rather than raw computational power. With chip costs continuing to rise relative to surging demand, such as Elon Musk's xAI Corp. seeking to double its Colossus data center to 200,000 Nvidia H100 graphics cards and an overall increased demand for AI-driven services, creating custom chips will allow OpenAI to tailor its infrastructure to meet both technical needs and budget constraints. By collaborating with Broadcom and TSMC, OpenAI can leverage established manufacturing expertise while moving faster than it could with a fully in-house production approach.

OpenAI Builds First Chip With Broadcom and TSMC, Scales Back Foundry Ambition

OpenAI has been working for months with Broadcom on its first AI chip OpenAI is working with Broadcom and TSMC to build its first in-house chip designed to support its artificial intelligence systems, while adding AMD chips alongside Nvidia chips to meet its surging infrastructure demands, sources told Reuters. OpenAI, the fast-growing company behind ChatGPT, has examined a range of options to diversify chip supply and reduce costs. OpenAI considered building everything in-house and raising capital for an expensive plan to build a network of factories known as "foundries" for chip manufacturing. The company has dropped the ambitious foundry plans for now due to the costs and time needed to build a network, and plans instead to focus on in-house chip design efforts, according to sources, who requested anonymity as they were not authorized to discuss private matters. The company's strategy, detailed here for the first time, highlights how the Silicon Valley startup is leveraging industry partnerships and a mix of internal and external approaches to secure chip supply and manage costs like larger rivals Amazon, Meta, Google and Microsoft. As one of the largest buyers of chips, OpenAI's decision to source from a diverse array of chipmakers while developing its customized chip could have broader tech sector implications. Broadcom stock jumped following the report, finishing Tuesday's trading up over 4.5 percent. AMD shares also extended their gains from the morning session, ending the day up 3.7 percent. OpenAI, AMD and TSMC declined to comment. Broadcom did not immediately respond to a request for comment. OpenAI, which helped commercialize generative AI that produces human-like responses to queries, relies on substantial computing power to train and run its systems. As one of the largest purchasers of Nvidia's graphics processing units (GPUs), OpenAI uses AI chips both to train models where the AI learns from data and for inference, applying AI to make predictions or decisions based on new information. Reuters previously reported on OpenAI's chip design endeavors. The Information reported on talks with Broadcom and others. OpenAI has been working for months with Broadcom to build its first AI chip focusing on inference, according to sources. Demand right now is greater for training chips, but analysts have predicted the need for inference chips could surpass them as more AI applications are deployed. Broadcom helps companies including Alphabet unit Google fine-tune chip designs for manufacturing and also supplies parts of the design that help move information on and off the chips quickly. This is important in AI systems where tens of thousands of chips are strung together to work in tandem. OpenAI is still determining whether to develop or acquire other elements for its chip design, and may engage additional partners, said two of the sources. The company has assembled a chip team of about 20 people, led by top engineers who have previously built Tensor Processing Units (TPUs) at Google, including Thomas Norrie and Richard Ho. Sources said that through Broadcom, OpenAI has secured manufacturing capacity with Taiwan Semiconductor Manufacturing Company to make its first custom-designed chip in 2026. They said the timeline could change. Currently, Nvidia's GPUs hold over 80% market share. But shortages and rising costs have led major customers like Microsoft, Meta, and now OpenAI, to explore in-house or external alternatives. OpenAI's planned use of AMD chips through Microsoft's Azure, first reported here, shows how AMD's new MI300X chips are trying to gain a slice of the market dominated by Nvidia. AMD has projected $4.5 billion in 2024 AI chip sales, following the chip's launch in the fourth quarter of 2023. Training AI models and operating services like ChatGPT are expensive. OpenAI has projected a $5 billion loss this year on $3.7 billion in revenue, according to sources. Compute costs, or expenses for hardware, electricity and cloud services needed to process large datasets and develop models, are the company's largest expense, prompting efforts to optimize utilization and diversify suppliers. OpenAI has been cautious about poaching talent from Nvidia because it wants to maintain a good rapport with the chip maker it remains committed to working with, especially for accessing its new generation of Blackwell chips, sources added.

OpenAI is creating its own in-house chip with Broadcom and TSMC as processing demands skyrocket

OpenAI is reportedly embarking on a new chapter in its artificial intelligence development journey -- it's building its own in-house chip. According to Reuters, which cited exclusive sources close to the company, OpenAI is collaborating with Broadcom and Taiwan Semiconductor Manufacturing Company (TSMC) to diversify its supply chain and control infrastructure costs as it seeks to keep up with the immense computational needs of AI. While OpenAI had previously considered building its own chip manufacturing foundries, the high costs and lengthy timeline led the company to prioritize in-house chip design. Currently, OpenAI has assembled a team of roughly 20 chip engineers, including experts with experience designing Google's Tensor Processing Units (TPUs). Together, they hope to develop OpenAI's first custom chip by 2026 -- though that timeline is flexible. The news on Tuesday sent Broadcom Inc. shares up 4% and TSMC's US-traded shares gained more than 1%. The move reflects OpenAI's ongoing strategy to secure consistent chip supply and manage escalating costs, a challenge that other tech giants like Amazon, Meta, Google, and Microsoft have also faced in their AI ventures. This effort includes OpenAI's recent decision to integrate AMD chips, along with Nvidia's GPUs, which continue to dominate the market but are in limited supply. In fact, tech giants have been literally begging Nvidia for more chips.

OpenAI reportedly builds custom AI chips as it embraces AMD -- company also abandons plans to build its own fabs

OpenAI is proceeding with the development of its first custom AI chip with Broadcom and expects to produce it for TSMC. However, reports Reuters, the company no longer intends to spearhead building a fab network. At the same time, the company continues to add more powerful chips from AMD and Nvidia to its fleet. In a bid to reduce reliance on Nvidia, OpenAI initially considered developing its chips both for training and inference and then facilitating the building of a dozen fabs (operated by prominent foundries like TSMC and Samsung Foundry), but high costs and long timelines made it impractical. Instead, OpenAI has prioritized designing custom AI chips for inference together with Broadcom and producing them at TSMC. For now, OpenAI will keep using GPUs from Nvidia and AMD for training. While high-demand AI GPUs like Nvidia's H100 and H200 are used for training of large language modes by pretty much everyone, which is why they are hard to get, demand for AI inference chips is projected to grow as more AI applications reach the market. OpenAI's upcoming custom-designed inference chip is slated for release by 2026. According to Reuters, this timeline could be adjusted based on project needs, but the focus is on inference tasks that enhance real-time AI responses. To support this new chip development, OpenAI has assembled a team of around 20 engineers led by experienced engineers like Thomas Norrie and Richard Ho, specialists who previously worked on Google's Tensor Processing Units (TPUs). The team is key to moving forward with the in-house design, which could allow for greater customization and efficiency. OpenAI now does the same thing as Amazon Web Services, Google, Meta, and Microsoft. These companies have chips for AI or general-purpose workloads, sometimes co-developed with Broadcom. In addition to its in-house custom silicon strategy, OpenAI diversifies its hardware suppliers to reduce dependency on Nvidia, which generally dominates the AI GPU and AI training hardware markets. OpenAI plans to deploy AMD's Instinct MI300X via Microsoft's Azure cloud platform, which will somewhat diversify its fleet. Despite ChatGPT's huge popularity, OpenAI projects a $5 billion loss this year against $3.7 billion in revenue due to high operating expenses that comprise cloud, electricity, and hardware costs. Diversifying hardware will probably help the company cut down its hardware costs, and custom chips are meant to reduce its power consumption, but that will only happen in 2026. While OpenAI is pursuing partnerships to broaden its hardware supply base, it is also mindful not to disrupt its relationship with Nvidia as the green company continues to develop the industry's highest-performing GPUs for AI. Therefore, OpenAI is poised to remain dependent on Nvidia if it wants to train the industry's best AI models. Nvidia's next-generation Blackwell GPUs for AI And HPC are poised to offer significant performance improvements over existing Hopper GPUs, enabling companies like OpenAI to train even more sophisticated AI models. However, Blackwell GPUs are more power-hungry than Hopper products. So, while their total cost of ownership, when performance is considered, may be lower than that of their predecessors, running them may be costlier, increasing OpenAI's costs.

OpenAI is Developing In-House AI Chip with Broadcom and TSMC: Report

The company is projected to face significant losses due to high compute costs in 2024. OpenAI is reportedly collaborating with Broadcom and TSMC to develop its first in-house chip designed to support its artificial intelligence (AI) systems. Additionally, it is incorporating AMD chips alongside Nvidia chips to meet growing infrastructure demands, Reuters reported, citing sources. Also Read: OpenAI Raises USD 6.6 Billion to Accelerate AI Research and Expansion OpenAI, the company behind ChatGPT, has reportedly explored various options to diversify chip supply and reduce costs. These options included building everything in-house and raising capital for an expensive plan to build a network of factories, or "foundries," for chip manufacturing, according to the report. However, the company has reportedly dropped the foundry plans due to the costs and time required to build such a network and will instead focus on in-house chip design efforts. As a major chip buyer, OpenAI's decision to source from multiple chipmakers while developing its own custom chip could have broader implications for the tech industry. Also Read: Microsoft, Dell, Google and Others Launch Initiatives to Propel AI Infrastructure and Innovation According to the report, OpenAI has been working with Broadcom for months to develop its first AI chip focused on inference. Broadcom assists companies like Alphabet's Google in fine-tuning chip designs for manufacturing and supplies components that facilitate rapid data transfer on and off the chips. This capability is crucial for AI systems, which require tens of thousands of interconnected chips to work seamlessly together, the report noted citing sources. OpenAI is still determining whether to develop or acquire other elements for its chip design and may bring in other partners, the report stated, citing two of the sources. OpenAI has assembled a chip team of about 20 engineers, led by former Google engineers who previously built Tensor Processing Units (TPUs), including Thomas Norrie and Richard Ho. Sources indicated that OpenAI, through Broadcom, has secured manufacturing capacity with Taiwan Semiconductor Manufacturing Company to produce its first custom-designed chip by 2026, though the timeline may be subject to change. Also Read: AI Can Help You Be More Productive at Work, Says Microsoft CEO and More The report also highlighted OpenAI's planned use of AMD chips through Microsoft's Azure, illustrating how AMD's new MI300X chips are attempting to capture a portion of the market dominated by Nvidia. AMD has projected USD 4.5 billion in AI chip sales for 2024, following the chip's launch in Q4 2023. Running AI models and services like ChatGPT is costly. According to the report, sources indicated that OpenAI projects a USD 5 billion loss this year, against USD 3.7 billion in revenue. Compute costs -- encompassing hardware, electricity, and cloud services for processing vast datasets and training models -- remain the company's largest expense, prompting initiatives to optimise resource use and diversify suppliers.

OpenAI reportedly planning to build its first AI chip in 2026

OpenAI is reportedly working with TSMC and Broadcom to build an in-house AI chip -- and beginning to use AMD chips alongside Nvidia's to train its AI. Reuters reports that OpenAI has -- at least for now -- abandoned plans to establish a network of factories for chip manufacturing. Instead, the company will focus on in-house chip design. OpenAI has for months been working with Broadcom to create an AI chip for running models, which could arrive as soon as 2026, reports Reuters. Meanwhile, OpenAI plans to use AMD chips through Microsoft's Azure cloud platform for model training. Previously, the company relied almost entirely on Nvidia GPUs for training, but chip shortages and delays -- and the high cost of training -- have spurred OpenAI to explore alternatives, according to Reuters.

OpenAI builds first chip with Broadcom and TSMC, scales back foundry ambition

The company has dropped the ambitious foundry plans for now due to the costs and time needed to build a network, and plans instead to focus on in-house chip design efforts, according to sources, who requested anonymity as they were not authorized to discuss private matters.OpenAI is working with Broadcom and TSMC to build its first in-house chip designed to support its artificial intelligence systems, while adding AMD chips alongside Nvidia chips to meet its surging infrastructure demands, sources told Reuters. OpenAI, the fast-growing company behind ChatGPT, has examined a range of options to diversify chip supply and reduce costs. OpenAI considered building everything in-house and raising capital for an expensive plan to build a network of factories known as "foundries" for chip manufacturing. The company has dropped the ambitious foundry plans for now due to the costs and time needed to build a network, and plans instead to focus on in-house chip design efforts, according to sources, who requested anonymity as they were not authorized to discuss private matters. The company's strategy, detailed here for the first time, highlights how the Silicon Valley startup is leveraging industry partnerships and a mix of internal and external approaches to secure chip supply and manage costs like larger rivals Amazon, Meta, Google and Microsoft. As one of the largest buyers of chips, OpenAI's decision to source from a diverse array of chipmakers while developing its customized chip could have broader tech sector implications. Broadcom stock jumped following the report, finishing Tuesday's trading up over 4.5%. AMD shares also extended their gains from the morning session, ending the day up 3.7%. OpenAI, AMD and TSMC declined to comment. Broadcom did not immediately respond to a request for comment. OpenAI, which helped commercialize generative AI that produces human-like responses to queries, relies on substantial computing power to train and run its systems. As one of the largest purchasers of Nvidia's graphics processing units (GPUs), OpenAI uses AI chips both to train models where the AI learns from data and for inference, applying AI to make predictions or decisions based on new information. Reuters previously reported on OpenAI's chip design endeavors. The Information reported on talks with Broadcom and others. OpenAI has been working for months with Broadcom to build its first AI chip focusing on inference, according to sources. Demand right now is greater for training chips, but analysts have predicted the need for inference chips could surpass them as more AI applications are deployed. Broadcom helps companies including Alphabet unit Google fine-tune chip designs for manufacturing and also supplies parts of the design that help move information on and off the chips quickly. This is important in AI systems where tens of thousands of chips are strung together to work in tandem. OpenAI is still determining whether to develop or acquire other elements for its chip design, and may engage additional partners, said two of the sources. The company has assembled a chip team of about 20 people, led by top engineers who have previously built Tensor Processing Units (TPUs) at Google, including Thomas Norrie and Richard Ho. Sources said that through Broadcom, OpenAI has secured manufacturing capacity with Taiwan Semiconductor Manufacturing Company to make its first custom-designed chip in 2026. They said the timeline could change. Currently, Nvidia's GPUs hold over 80% market share. But shortages and rising costs have led major customers like Microsoft, Meta, and now OpenAI, to explore in-house or external alternatives. OpenAI's planned use of AMD chips through Microsoft's Azure, first reported here, shows how AMD's new MI300X chips are trying to gain a slice of the market dominated by Nvidia. AMD has projected $4.5 billion in 2024 AI chip sales, following the chip's launch in the fourth quarter of 2023. Training AI models and operating services like ChatGPT are expensive. OpenAI has projected a $5 billion loss this year on $3.7 billion in revenue, according to sources. Compute costs, or expenses for hardware, electricity and cloud services needed to process large datasets and develop models, are the company's largest expense, prompting efforts to optimize utilization and diversify suppliers. OpenAI has been cautious about poaching talent from Nvidia because it wants to maintain a good rapport with the chip maker it remains committed to working with, especially for accessing its new generation of Blackwell chips, sources added. Nvidia declined to comment.

OpenAI bets on AMD, plans custom AI chips by 2026

OpenAI, the company behind ChatGPT, is expanding its efforts to secure reliable and cost-effective computing power for its AI models. By developing custom silicon, OpenAI aims to reduce dependence on external suppliers like NVIDIA, whose GPUs dominate the AI chip market. According to Reuters OpenAI has partnered with Broadcom and secured manufacturing capacity with Taiwan Semiconductor Manufacturing Company (TSMC), while incorporating AMD chips into its Microsoft Azure setup. OpenAI's journey towards developing its own AI chips started with assembling a team of about 20 people, including top engineers who previously worked on Google's Tensor Processing Units (TPUs). This in-house chip team, led by experienced engineers like Thomas Norrie and Richard Ho, is working closely with Broadcom to design and produce custom silicon that will focus on inference workloads. The chips are expected to be manufactured by TSMC, the world's largest semiconductor foundry, starting in 2026. The goal behind developing in-house silicon is twofold: to secure a stable supply of high-performance chips and to manage escalating costs associated with AI workloads. While demand for training chips is currently higher, industry experts anticipate that the need for inference chips will surpass training chips as more AI applications reach the deployment stage. Broadcom's expertise in helping fine-tune chip designs for mass production and providing components that optimize data movement makes it an ideal partner for this ambitious project. OpenAI had previously considered building its own chip foundries but ultimately decided to abandon those plans due to the immense costs and time required. Instead, OpenAI is focusing on designing custom chips while relying on TSMC for manufacturing. Alongside its partnership with Broadcom, OpenAI is also incorporating AMD's new MI300X chips into its Microsoft Azure setup. AMD introduced these chips last year as part of its data center expansion strategy, aiming to capture some of the market share currently held by NVIDIA. The inclusion of AMD chips will allow OpenAI to diversify its chip supply, reducing its dependence on a single supplier and helping to manage costs more effectively. AMD's MI300X chips are part of its push to compete with NVIDIA, which currently holds over 80% of the market share in AI hardware. The MI300X chips are designed to support AI workloads, particularly in inference and model training. By adding AMD chips into its infrastructure, OpenAI is hoping to alleviate some of the supply constraints it has faced with NVIDIA GPUs, which have been in high demand and subject to shortages. This strategic move is also a response to rising compute costs, which have become a major challenge for OpenAI. The company has been dealing with high expenses for hardware, electricity, and cloud services, leading to projected losses of $5 billion this year. Reducing its reliance on a single supplier like NVIDIA, which has been increasing its prices, could help OpenAI better manage these costs and continue developing its AI models without significant delays or interruptions. Despite the ambitious plan to develop custom chips, there are significant challenges ahead for OpenAI. Building an in-house silicon solution takes time and money, and the first custom-designed chips are not expected to be in production until 2026. This timeline puts OpenAI behind some of its larger competitors like Google, Microsoft, and Amazon, who have already made substantial progress in developing their own custom AI hardware. The partnership with Broadcom and TSMC represents an important step forward, but it also highlights the difficulties faced by companies trying to break into the chip market. Manufacturing high-performance AI chips requires substantial expertise, advanced production facilities, and considerable investment. TSMC, as the manufacturing partner, will play a key role in determining the success of this venture. The timeline for chip production could still change, depending on factors like design complexity and manufacturing capacity. Another challenge lies in talent acquisition. OpenAI is cautious about poaching talent from NVIDIA, as it wants to maintain a good relationship with the chipmaker, especially since it still relies heavily on NVIDIA for its current-generation AI models. NVIDIA's Blackwell chips are expected to be crucial for upcoming AI projects, and maintaining a positive relationship is essential for OpenAI's ongoing access to these cutting-edge GPUs. The main driver behind OpenAI's custom chip initiative is cost. Training and deploying large AI models like GPT-4 require massive computing power, which translates to high infrastructure expenses. OpenAI's annual compute costs are projected to be one of its largest expenses, with the company expecting a $5 billion loss this year despite generating $3.7 billion in revenue. By developing its own chips, OpenAI hopes to bring these costs under control, giving it a competitive edge in the crowded AI market. Custom silicon also offers performance benefits. By tailoring chips specifically for the needs of AI inference, OpenAI can optimize performance, improve efficiency, and reduce latency. This is particularly important for delivering high-quality, real-time responses in products like ChatGPT. While NVIDIA's GPUs are highly capable, custom-designed hardware can provide more targeted optimization, potentially leading to significant gains in performance and cost efficiency. The approach of blending internal and external chip solutions provides OpenAI with greater flexibility in how it scales its infrastructure. By working with Broadcom on custom designs while also incorporating AMD and NVIDIA GPUs, OpenAI is positioning itself to better navigate the challenges of high demand and supply chain limitations. This diversified approach will help the company adapt to changing market conditions and ensure that it has the computing resources necessary to continue pushing the boundaries of AI.

OpenAI reportedly talks custom silicon with Broadcom

OpenAI is reportedly in talks with Broadcom to build a custom inferencing chip. A Reuters report claims the AI upstart and the chip design firm have staged confidential discussions about custom silicon, with Taiwan Semiconductor Manufacturing Company involved as the likely foundry for the effort. Just why OpenAI wants its own inferencing chip is not known, but it's not hard to guess why such a move appeals: the startup has enormous cloud bills - some of them comped by partners like Microsoft - and might fancy running its own hardware instead. It certainly wouldn't be alone in finding on-prem operations are considerably cheaper than renting cloudy resources. Developing silicon tuned to its own services could be another motive. AI applications guzzle energy, and mutual optimization of hardware and software could mean OpenAI's services become more efficient. OpenAI has also reportedly tried to convince investors to build giant datacenters dedicated to running AI services. Perhaps those theoretical bit barns will be cheaper to build and/or run with custom silicon inside. Diversifying suppliers could be another motive. The world's foundries can only crank out so much stuff, and rely on supply chains that are sometimes tenuous. OpenAI would not be immune to those vagaries but could at least reduce its dependence on third-party suppliers of finished product. The Register can't imagine OpenAI wants to get into the mucky business of hardware sales - an industry that requires all sorts of bothersome investments in the real world and would therefore bloat its headcount. But as inferencing is a workload best run physically close to users - because latency sucks - a play that puts devices deep into networks can't be ruled out. That's how content delivery networks and the likes of Netflix operate. An architecture that places an OpenAI inferencing box on the network edge is not a fantastic notion. Custom inference chips are not novel. AWS has one called Inferentia. Google's Tensor processing units and Microsoft's Maia silicon can handle inferencing and training workloads. The suggestion that OpenAI is talking to Broadcom could be one reason the chip design firm's shares popped a little in late trading. Broadcom's most recent quarterly earnings predicted it would sell $12 billion of AI silicon this financial year alone - a billion bucks more than its previous forecast - but investors still appeared disappointed. Teaming with the hottest name in AI software would likely get Wall Street more excited. ®

OpenAI will start using AMD chips and could make its own AI hardware in 2026

OpenAI is reportedly working with Broadcom to develop new custom silicon designed to handle its large AI workloads for inference and secured manufacturing capacity with TSMC, according to sources speaking to Reuters. OpenAI has reportedly built a chip development team of about 20 people, including lead engineers who previously worked on Google's Tensor processors for AI. Still, on its current timeline, the custom-designed hardware may not start production until 2026. In the meantime, the sources also said OpenAI is incorporating AMD chips into its Microsoft Azure setup. AMD introduced its MI300 chips last year, which was a big part of the news this summer that its data center business has doubled in a single year as it chases market leader Nvidia. The Information had reported in July that OpenAI was in discussion with Broadcom and other semiconductor designers about developing its own AI chip, and earlier this year, Bloomberg reported that OpenAI was working to build its own network of foundries, but according to Reuters, those plans have been put on ice due to cost and time. The reported strategy puts OpenAI on a similar track to the other tech companies trying to manage costs and access to AI server hardware with custom chip designs. But Google, Microsoft, and Amazon are all already a few generations down the road in their efforts, and OpenAI may need significantly more funding to become a true competitor.

OpenAI to build its first AI chip with Broadcom and TSMC, scaling back its foundry ambitions

AI-Assisted TLDR: OpenAI is collaborating with Broadcom and TSMC to develop its first in-house chip for AI systems, shifting from plans to build its own chip fabrication plants due to cost and time constraints. The company is adopting a strategy of using industry partnerships and a mix of internal and external approaches to secure chip supply and manage costs, similar to larger tech companies.* Generated from the content by Anthony Garreffa below. OpenAI is working with Broadcom and TSMC on building its first in-house chip designed to support its extensive AI systems. The news is coming directly from Reuters, which is reporting from its sources who "requested anonymity as they were not authorized to discuss private matters" (but I guess, did so with one of the largest publications in the world). OpenAI did consider building everything in-house, with CEO Sam Altman having ambitious plans to have a global network of dedicated chip fabrication plants, but that is now not happening. Reuters reports that OpenAI "dropped the ambitious foundry plans for now due to the costs and time needed to build a network, and plans instead to focus on in-house chip design efforts". OpenAI's new strategy seems to be using industry partnerships, with a mix of internal and external approaches to secure chip supply, and manage costs like larger companies in Amazon, Meta, Google, and Microsoft. OpenAI is one of the biggest buyers of chips, so its decision to diversify its chipmakers, while still continuing to work on its in-house customized chips, could have "broader tech sector implications" adds Reuters.

Exclusive: OpenAI builds first chip with Broadcom and TSMC, scales back foundry ambition

Oct 29 (Reuters) - OpenAI is working with Broadcom (AVGO.O), opens new tab and TSMC (2330.TW), opens new tab to build its first in-house chip designed to support its artificial intelligence systems, while adding AMD (AMD.O), opens new tab chips alongside Nvidia (NVDA.O), opens new tab chips to meet its surging infrastructure demands, sources told Reuters. OpenAI, the fast-growing company behind ChatGPT, has examined a range of options to diversify chip supply and reduce costs. OpenAI considered building everything in-house and raising capital for an expensive plan to build a network of factories known as "foundries" for chip manufacturing. The company has dropped the ambitious foundry plans for now due to the costs and time needed to build a network, and plans instead to focus on in-house chip design efforts, according to sources, who requested anonymity as they were not authorized to discuss private matters. The company's strategy, detailed here for the first time, highlights how the Silicon Valley startup is leveraging industry partnerships and a mix of internal and external approaches to secure chip supply and manage costs like larger rivals Amazon, Meta, Google and Microsoft. As one of the largest buyers of chips, OpenAI's decision to source from a diverse array of chipmakers while developing its customized chip could have broader tech sector implications. OpenAI, AMD and TSMC declined to comment. Broadcom did not immediately respond to a request for comment. OpenAI, which helped commercialize generative AI that produces human-like responses to queries, relies on substantial computing power to train and run its systems. As one of the largest purchasers of Nvidia's graphics processing units (GPUs), OpenAI uses AI chips both to train models where the AI learns from data and for inference, applying AI to make predictions or decisions based on new information. Reuters previously reported on OpenAI's chip design endeavors. The Information reported on talks with Broadcom and others. OpenAI has been working for months with Broadcom to build its first AI chip focusing on inference, according to sources. Demand right now is greater for training chips, but analysts have predicted the need for inference chips could surpass them as more AI applications are deployed. Broadcom helps companies including Alphabet (GOOGL.O), opens new tab unit Google fine-tune chip designs for manufacturing and also supplies parts of the design that help move information on and off the chips quickly. This is important in AI systems where tens of thousands of chips are strung together to work in tandem. OpenAI is still determining whether to develop or acquire other elements for its chip design, and may engage additional partners, said two of the sources. The company has assembled a chip team of about 20 people, led by top engineers who have previously built Tensor Processing Units (TPUs) at Google, including Thomas Norrie and Richard Ho. Sources said that through Broadcom, OpenAI has secured manufacturing capacity with Taiwan Semiconductor Manufacturing Company (2330.TW), opens new tab to make its first custom-designed chip in 2026. They said the timeline could change. Currently, Nvidia's GPUs hold over 80% market share. But shortages and rising costs have led major customers like Microsoft, Meta, and now OpenAI, to explore in-house or external alternatives. OpenAI's planned use of AMD chips through Microsoft's Azure, first reported here, shows how AMD's new MI300X chips are trying to gain a slice of the market dominated by Nvidia. AMD has projected $4.5 billion in 2024 AI chip sales, following the chip's launch in the fourth quarter of 2023. Training AI models and operating services like ChatGPT are expensive. OpenAI has projected a $5 billion loss this year on $3.7 billion in revenue, according to sources. Compute costs, or expenses for hardware, electricity and cloud services needed to process large datasets and develop models, are the company's largest expense, prompting efforts to optimize utilization and diversify suppliers. OpenAI has been cautious about poaching talent from Nvidia because it wants to maintain a good rapport with the chip maker it remains committed to working with, especially for accessing its new generation of Blackwell chips, sources added. Nvidia declined to comment. Reporting by Krystal Hu in New York, Fanny Potkin in Singapore, Stephen Nellis in San Francisco, additional reporting by Anna Tong and Max Cherney in San Francisco; Editing by Kenneth Li and David Gregorio Our Standards: The Thomson Reuters Trust Principles., opens new tab Krystal Hu Thomson Reuters Krystal reports on venture capital and startups for Reuters. She covers Silicon Valley and beyond through the lens of money and characters, with a focus on growth-stage startups, tech investments and AI. She has previously covered M&A for Reuters, breaking stories on Trump's SPAC and Elon Musk's Twitter financing. Previously, she reported on Amazon for Yahoo Finance, and her investigation of the company's retail practice was cited by lawmakers in Congress. Krystal started a career in journalism by writing about tech and politics in China. She has a master's degree from New York University, and enjoys a scoop of Matcha ice cream as much as getting a scoop at work.

OpenAI edges closer to making its first AI chip in bid to power your favorite new apps

The move could help reduce the costs of running OpenAI-powered apps OpenAI is a step closer to developing its first AI chip, according to a new report - as the number of developers making apps on its platform soars alongside cloud computing costs. The ChatGPT maker was first reported to be in discussions with several chip designers, including Broadcom, back in July. Now Reuters is claiming that a new hardware strategy has seen OpenAI settle on Broadcom as its custom silicon partner, with the chip potentially landing in 2026. Before then, it seems OpenAI will be adding AMD chips to its Microsoft Azure system, alongside the existing ones from Nvidia. The AI giant's plans to make a 'foundry' - a network of chip factories - have been scaled back, according to Reuters. The reason for these reported moves is to help reduce the ballooning costs of AI-powered applications. OpenAI's new chip apparently won't be used to train generative AI models (which is the domain of Nvidia chips), but will instead run the AI software and respond to user requests. During its DevDay London event today (which followed the San Francisco version on October 1), OpenAI announced some improved tools that it's using to woo developers. The biggest one, Real-time API, is effectively an Advanced Voice Mode for app developers, and this API now has five new voices that have improved range and expressiveness. Right now, three million developers from around the world are using OpenAI's API (application programming interface), but the problem is that many of its features are still too expensive to run at scale. OpenAI says it's reduced the price of API tokens (in other words, how much it costs developers to use its models) by 99% since the launch of GPT-3 in June 2020, but there's still a long way to go - and this custom AI chip could be an important step towards making AI-powered apps cost-effective and truly mainstream. The sky-high costs of cloud AI processing are still a handbrake on apps building OpenAI's tools into their offerings, but some startups have already taken the plunge. The popular online video editor Veed plugs into several OpenAI models to offer features like automated transcripts and the ability to pick out the best soundbites from long-form videos. An AI-powered notepad called Granola also leverages GPT-4 and GPT-4o to transcribe meetings and send you follow-up tasks, without needing a meeting bot to join your call. Away from consumer apps, a startup called Tortus is using GPT-4o and OpenAI's voice models to help doctors. Its tools can listen to doctor-patient chats and automate a lot of the admin like updating health records, while apparently also improving diagnosis accuracy. Leaving aside the potential privacy and hallucination concerns of AI models, developers are clearly keen to tap into the power of OpenAI's tools - and there's no doubt that its low-latency, conversational voice mode has massive potential for customer service. Still, while you can expect to be talking to one of OpenAI's voice models when calling a store or customer service line soon, those AI running costs could slow down the rate of adoption - which is why OpenAI is seemingly keen to develop its own AI chip sooner rather than later.

OpenAI, Broadcom Working to Develop AI Chip Focused on Inference

OpenAI is working with Broadcom Inc. to develop a new artificial intelligence chip specifically focused on running AI models after they've been trained, according to two people familiar with the matter. The AI startup and chipmaker are also consulting with Taiwan Semiconductor Manufacturing Co., the world's largest chip contract manufacturer, said the people, who asked not to be identified because the discussions are private. OpenAI has been planning a custom chip and working on such uses for the technology for around a year, the people said, but the discussions are still at an early stage.

OpenAI's Push for Custom AI Chip -- Strategy To Reduce Soaring Cloud Costs?

In 2024, OpenAI will spend $4 billion running its AI models on Microsoft's cloud. OpenAI is reportedly exploring the prospect of developing its own inference chip, joining a host of chip makers and other Big Tech players doubling down on the technology. Amid surging demand for AI processing capacity, the move could help ease the burden on the firm's skyrocketing cloud costs, which are projected to reach as much as $7 billion in 2024. OpenAI and Broadcom Exploring AI Chip Possibilities On Tuesday, Oct. 29, Reuters reported that OpenAI is working with Broadcom to design a new AI chip. The proposed solution wouldn't replace Nvidia GPUs, which OpenAI and its peers depend on for AI training, but would be used for inference -- the process of running pre-trained AI models like the ones that generate ChatGPT outputs. Although discussions are still in the early stages, the report said the two firms have already consulted with the Taiwan Semiconductor Manufacturing Company (TSMC) about a potential foundry partnership. (Broadcom is a "fabless" manufacturer that outsources production to foundries like TSMC.) Meeting Infrastructure Demands With the demand for AI inference rising, OpenAI isn't the only company looking to build up its capacity with new semiconductor solutions. The Big Three cloud providers -- Amazon Web Services (AWS), Google, and Microsoft -- have also developed custom inference chips, as has Meta. For the hyperscalers, the move is a logical response to an evolving cloud market that will help them adapt to customers' changing requirements in the age of AI. Meanwhile, AI developers like Meta and OpenAI could benefit from reduced cloud costs by building custom data center solutions and realigning their relationships with infrastructure providers. OpenAI's $7 Billion Cloud Bill In 2024, OpenAI is reportedly on track to run up a $4 billion cloud bill from inference workloads alone, with another $3 billion in AI training costs on top of that. The massive cloud expenses, which are nearly double the firm's estimated revenues for the year, are being channeled straight into Microsoft's pocket. As part of their expansive relationship , Microsoft is OpenAI's exclusive cloud provider, granting the AI company discounted rates on its Azure usage.