ByteDance Ramps Up Efforts to Develop In-House AI Chips

China's ByteDance boosts efforts for self-designed AI chips

TikTok owner ByteDance (BDNCE) is advancing efforts to make its own AI chips as it intends to get an edge over its competitors in China's AI chatbot market, The Information reported. ByteDance is planning for mass manufacturing of two semiconductors it has designed by 2026, in collaboration with Taiwan Semiconductor Manufacturing (TSM), the report added citing people familiar with the matter. The move could reduce the Chinese company's reliance on costly Nvidia (NVDA) chips for developing and operating AI models, the report added. China's cloud services market has seen a price war with players such as Baidu (BIDU) Alibaba (BABA), ByteDance (BDNCE), Tencent (OTCPK:TCEHY) (OTCPK:TCTZF) and iFlytek cutting prices of their large language models, or LLMs, which power AI chatbots. Reducing chip costs is a need for ByteDance. Last year the company released its first AI chatbot Doubao which provides generative AI content of text- and image. This year, ByteDance announced a group of low-cost LLMs, some of which costed up to 99% less than comparable offerings from OpenAI for generating a token -- a word or part of a word generated by the models. Microsoft (MSFT)-backed ChatGPT creator OpenAI does not provide its AI models in China. ByteDance has been investing to develop generative AI models. This year, the company placed orders for over 200,000 Nvidia H20 chips, the most advanced chip Nvidia made for the Chinese market which are allowed to be sold under the U.S. export rules, the report noted. This order costed over $2B and ByteDance is still waiting to take full delivery, as per the report. Now, ByteDance intends to order several hundred thousand of its own training and inference chips from TSM. The cost of these internally designed chips would be billions of dollars less than purchasing chips from Nvidia, as per estimates from The Information. However, the chips are still in the design phase, and ByteDance's plans could change. The chip designs for its 2026 semiconductors are based on the advanced 5-nanometer technology, which is only one generation behind TSM's most cutting-edge tech. The 5 nm process, which Chinese chip makers have found difficult to replicate, is the same generation of technology Nvidia is using for its upcoming flagship AI chip system Blackwell, the report noted. The H20 can achieve about 25% the performance of the Nvidia H100, which was Nvidia's most advanced chip when it was released in 2022. The U.S. had restricted Nvidia from selling the A100 and more powerful successors, including the H100, in China, as part of Washington's efforts aimed at hindering China's access to advanced semiconductor technology used for AI. The H20 remains in high demand as Chinese companies navigate U.S. export control curbs. As the computing power of a single H20 is low compared to the more common Nvidia chips used by U.S. cloud providers, the Chinese firms need to compensate by buying more of them to make a single cluster with the same level of computing power. This adds to their costs, according to the report. The U.S. curbs apply to individual chips based on their performance, but clusters do not have such restrictions. One of the people familiar with ByteDance's chips said there were other ways Chinese chip engineers could use to increase the performance of chips, such as boosting the connection between each chip in a cluster and improving the chip's performance when accessing high-bandwidth computer memory. The U.S. restrictions do not restrict either of these methods. ByteDance desires clusters powered by its own chips to get four times the computing performance of a single H100 chip for about the same cost. The H100 chips can cost over $30,000 each. The chip efforts by ByteDance are in the footsteps of companies, including Alphabet's (GOOG) (GOOGL) unit Google, Amazon (AMZN) and Microsoft, who in the past few years have been trying to reduce their dependence on Nvidia chips for developing and running AI models. Some of these companies also offer their own chips as an alternative to Nvidia for customers using their cloud services, the report noted. Chinese companies, such as Tencent, Alibaba and Baidu have been developing their own AI chips for years. Baidu is developing a new AI chip called Kunlun 3, for its generative AI products, the report added, citing a person familiar with the project. The Baidu chip is at the final stage of the design phase and nearing the start of production at TSM, the report noted. In July, it was reported that Nvidia was working on a new chip for China which has a special server design that would not violate U.S. export restrictions. Meanwhile, Chinese tech giant Huawei Technologies is reportedly nearing to unveil a new chip to use for AI that would compete with Nvidia's chips in the Chinese market.

TikTok's owner wants to design its own AI chips

As China advances its artificial intelligence industry -- much to the frustration of the U.S. -- competition is brewing between its homegrown AI developers. Now, one is reportedly working on its own chips with a major chipmaker. ByteDance has designed two chips with Taiwan Semiconductor Manufacturing Company (TSM) that it plans to mass produce by 2026, The Information reported, citing unnamed people familiar with the matter. By producing its own chips, the company could become less dependent on Nvidia's (NVDA) pricey graphics processing units, or GPUs, which are subject to U.S. export controls, people told The Information. Neither ByteDance nor TSMC immediately responded to Quartz's request for comment. Last August, ByteDance released its AI-powered chatbot, Doubao, which has proven to be a tough rival to Baidu's (BIDU) Ernie Bot -- considered China's version of OpenAI's ChatGPT, which is not allowed in the country. In China, Doubao passed Ernie Bot in downloads last year, and had more active monthly users on iOS, Bloomberg reported. ByteDance launched a series of Doubao large language models (LLMs) for enterprises in May, which cost less than models from rivals who have had to cut prices by up to 97% amid rising competition, according to The Information. The TikTok owner has reportedly placed orders for over 200,000 of Nvidia's H20 chips -- one of three specifically designed to comply with U.S. export controls. The company is still waiting for the over $2 billion order, The Information reported. However, ByteDance now plans to order chips for training and inferencing from TSMC, which would save it billions of dollars, The Information estimated, adding that the chips are still being designed and plans are not finalized. Meanwhile, Baidu has reportedly designed its own AI chip, the Kunlun 3, which will soon start manufacturing at TSMC, a person told The Information. Earlier this month the U.S. released its latest set of export controls on critical technologies as it steps up its efforts to curb China's technological advances. Chinese companies' ability to work with TSMC depends on compliance with the controls, which are updated annually.