ByteDance's AI Chip Development: Conflicting Reports and Strategic Moves

ByteDance Denies Rumors of Developing "In-House" AI Chips, Claims That It is Focusing On Cost Optimization

ByteDance, the Chinese tech giant, has reportedly refuted claims of developing its own AI semiconductors, saying that it has no plans to replace NVIDIA. For those unaware, ByteDance was reported to have contacted the Taiwan semiconductor giant TSMC in an attempt to develop two in-house AI chips based on the foundry's 5nm process. This move by ByteDance was rumored to have been planned to reduce the firm's reliance on NVIDIA and their AI chips, given that the firm has accumulated massive amounts of Team Green's AI accelerators, worth around $2 billion, in this year alone. However, a new report by SCMP claims that ByteDance has denied developing its own AI chips and that the company has no plans to replace NVIDIA, at least in the shorter term. The report states that ByteDance's initiatives in the semiconductor industry won't stop, and that they "are in the early stage, focusing on cost optimisation of recommendations, advertising and other businesses." So, the firm's statement doesn't explicitly mention that they are not developing in-house AI chips. Hence, it does show that the project is under wraps for now. It makes sense for a company like ByteDance to switch towards AI solutions that are viable for them, not just in terms of the capabilities they bring onboard but also in how readily they are available. Every other mainstream tech giant, such as Google, Amazon, and Microsoft, is looking towards building their own AI chip portfolio simply because the waiting times given by NVIDIA are too high amid strong market demand; hence, in order to stay competitive, having an in-house AI chip setup makes sense. In terms of ByteDance, well, the firm was rumored to have been developing two different AI chips targeting the inferencing and training applications, which were said to be based on TSMC's 5nm technology; hence if we compare the processes alone, ByteDance might achieve an in-house solution which comes near to NVIDIA's Hopper generation, that is only if we look at raw power. The firm's software stack needs to be capable enough to support the AI chip, so moving toward an in-house solution might not seem as easy as it sounds.

TikTok owner ByteDance taps TSMC to make its own AI GPUs to stop relying on Nvidia -- the company has reportedly spent over $2 billion on Nvidia AI GPUs

According to news outlet The Information, ByteDance, the parent company of TikTok, is developing two AI GPUs that will enter mass production by 2026. TSMC will make both products. Assuming that the information from an unofficial source is accurate, ByteDance will reduce reliance on Nvidia for AI hardware while staying within the U.S. export regulations. ByteDance's lineup of AI GPUs, which are in the design phase and will enter mass production in a year, if not later, includes one for AI training and another for AI inference. Broadcom, which has already built AI chips for Google, is expected to design the AI chips. The GPUs are said to be produced on one of TSMC's N4/N5 process technologies, so a similar node to TSMC's 4NP is used to build Nvidia's Blackwell series GPUs for AI and HPC. ByteDance's GPUs are expected to enter mass production by 2026, so expect their deployment in 2026. ByteDance has reportedly spent over $2 billion on more than 200,000 Nvidia H20 GPUs (i.e., about $10,000 per unit, which is a bit less than $12,000 - $13,000) for its AI efforts this year alone, and many of these GPUs have yet to be delivered to the company. This heavy investment highlights the importance of AI to ByteDance's general strategy. According to the report, the shortage of Nvidia GPUs and their high price are among the reasons why ByteDance decided to build its own AI hardware. Nvidia designed its DGX H20 and some other GPUs specifically for the Chinese market in response to U.S. export controls imposed last year. As a result, the HGX H20 is a massively cut-down GPU (compared to the H100) that still sells for a whopping $10,000 if the information about the price is correct. For example, while Nvidia's HGX H20 only offers 296 INT8/FP8 TOPS/TFLOPS and 148 BF16/FP16 TFLOPS performance for AI computations, the fully-fledged H100 delivers 3,958 INT8/FP8 TOPS/TFLOPS as well as 1,979 BF16/FP16 TFLOPS performance for AI. Yet, with 96 GB of HBM3 memory onboard, up to 4.0 TB/s of memory bandwidth, and 8-way GPU capability, Nvidia's HGX H20 is still in high demand by Chinese companies in real-world applications. Nvidia's processor reportedly beats Huawei's competitors. While ByteDance will unlikely be able to make its GPUs significantly faster than Nvidia's HGX H20 due to U.S. export control rules (as TSMC will be unable to ship such GPUs to ByteDance), they will be significantly cheaper for the company. There is a massive catch about ByteDance's initiative to develop its GPUs for AI. The company now relies on Nvidia's CUDA and supporting software stack for AI training and inference. Once it goes with its AI GPUs, it must develop its software platform and ensure its software stack is fully compatible with its hardware. Although many Chinese companies have developed AI GPUs to reduce reliance on Nvidia, those chips are used for select workloads and continue to rely on Nvidia's GPUs for others. Nvidia expects to ship over one million HGX H20 units to its Chinese customers this year, nearly double Huawei's projected sales of 550,000 Ascend 910B AI GPUs for 2024. Nvidia's H20 GPUs could generate over $12 billion in revenue, surpassing the company's total China earnings from the previous year, including sales of other hardware types, such as GPUs for gamers.

TikTok parent company ByteDance to have 2 custom AI chips made on TSMC 5nm process in 2026

TikTok parent company ByteDance is developing not one but two new AI GPUs that are reportedly being made on TSMC 5nm process node, and will enter mass production in 2026. The information is coming from The Information, where according to their sources ByteDance will reduce its reliance on NVIDIA for its AI hardware, all the while staying in the lines of US export regulations. ByteDance's new AI GPUs are in the design phase, with one of them being for AI training and the other for AI inference. ByteDance's new AI GPUs are said to be made on TSMC 4N/5N process nodes, which is similar to the 4NP process node that TSMC uses to make NVIDIA's new Blackwell AI GPUs. The TikTok parent company reportedly spent over $2 billion buying over 200,000+ NVIDIA H20 AI GPUs (which cost around $10,000 per H20 AI GPU) this year alone, with many of the AI GPUs not yet delivered by NVIDIA. The shortage of NVIDIA AI GPUs and their higher price tags are some of the reasons ByteDance has decided to make its own AI hardware, even with NVIDIA tweaking the DGX H20 and other AI GPUs specifically for the Chinese market, after US export controls continued getting tightened. NVIDIA's DGX H20 AI GPU features only 296 INT8/FP8 TOPS/TFLOPS and 148 BF16/FP16 TFLOPS performance for AI computations, meanwhile the full H100 AI GPU features 3,958 INT8/FP8 TOPS/TFLOPS as well as 1,979 BF16/FP16 TFLOPS performance for AI... over 10x the performance. But, the DGX H20 has 96GB of HBM3 memory with up to 4.0TB/sec of memory bandwidth and 8-way GPU support, so it's a popular AI GPU offer in China... but ByteDance wants more performance, and to not have to deal with NVIDIA anymore it seems (even after spending billions on NVIDIA AI hardware). However, ByteDance might be developing its own AI hardware, but right now... right now it depends on NVIDIA CUDA and supporting software stack for its AI training and inference. ByteDance will have to build its own software platform and working software stack is fully compatible with its brand-new hardware.