NVIDIA AI GPUs Cheaper to Rent in China Than in the US

Nvidia's AI GPUs are cheaper to rent in China than in the U.S. -- $6 per hour for eight Nvidia A100 GPUs

Despite U.S. export restrictions, Nvidia's GPUs are easily accessible in China via the cloud. That access is cheaper in Tianxia than in the U.S., according to a report by the Financial Times. Four small-scale Chinese providers offer 8-way Nvidia A100 servers at around 40% lower per hour than U.S. cloud providers. FT believes that the availability of these GPUs points to widespread smuggling and a robust resale market in China. Chinese cloud providers charge significantly lower rates for renting Nvidia AI chip-powered servers. Small companies in China charge about $6 per hour for servers using eight Nvidia A100 GPUs, while similar services in the U.S. cost about $10 per hour. This pricing disparity reflects the abundant supply of these GPUs in China despite U.S. efforts to limit their distribution in Tianxia. While small vendors in China can offer competitive prices using these smuggled Nvidia GPUs, larger companies like Alibaba and ByteDance charge higher rates. These major players, focused on compliance, avoid using illegally obtained GPUs. Their prices for Nvidia-powered servers range from double to quadruple what smaller firms charge, although they offer discounts that bring their rates in line with Amazon Web Services (AWS), which charges $15 to $32 per hour, Financial Times reports. Nvidia's AI GPUs, particularly the A100 and H100 models, have been restricted from direct export to China since October 2022, which is why Nvidia invented A800 and H800 GPUs with fewer external connections to prevent building high-end supercomputers based on these GPUs. Then, the U.S. restricted sales of A800 and H800 to Chinese entities in October 2023. Still, there are plenty of such GPUs in the country, as the report from Financial Times shows. Nvidia's A100 is sold openly on Chinese platforms like Taobao and Xiaohongshu, often at slight markups from their prices outside of China. In Shenzhen's Huaqiangbei electronics market, H100 GPUs for AI and HPC are sold for prices ranging from $23,000 to $30,000. This is slightly higher than Nvidia's official pricing of $20,000 to $23,000 for these GPUs. Nvidia's A100 and H100 GPUs are often brought into China from countries such as Japan, Malaysia, and Indonesia, with Hong Kong acting as a key transit hub. Companies are finding various ways to bypass U.S. export regulations. For example, Chinese firms may set up new entities in countries like Japan or Malaysia to acquire Nvidia GPUs and ship them back to China. Smugglers have even etched the serial numbers from GPUs to avoid detection by authorities. For obvious reasons, this black market access greatly undermines U.S. efforts to control China's use of cutting-edge technology. Nvidia says it works with its direct partners to ensure compliance with U.S. export laws. However, the company acknowledges that it cannot track second-hand sales of its products.

NVIDIA's AI Chip Renting Services In China Are Much Cheaper Compared To The US, For As Little As $6/Hr

NVIDIA's AI GPU "renting services" have seen a drastic rise in China, as small-scale CSPs provide AI chips that are much cheaper to rent compared to the US. AI accelerator "decentralization" has recently picked up a massive pace in the markets, especially in regions influenced by US sanctions, such as China. With the AI hype picking up in Chinese markets, the region has found new loopholes in US policies, and one way to get access to cutting-edge AI GPUs is by renting them out. We recently reported on how Chinese AI engineers have been accessing NVIDIA's chips by working with specific global brokers and using cryptocurrency as a means for financial payments to maintain anonymity. Now, The Financial Times reports that Chinese CSPs have stepped into the business, offering their hardware stack for renting out, and interestingly, the prices are phenomenally lower compared to what firms are offering in the US. It is disclosed that small CSPs in China provide companies with an AI server that consists of eight NVIDIA's A100 AI GPUs, and it only costs $6 per hour to rent them out, while in the US, the same setup would cost around $10, marking around a 50% difference. Interestingly, despite the US sanctions, NVIDIA's H100 and A100 AI GPUs are readily available in China, which is why the rental costs are much lower than in other regions. According to a Chinese startup founder, it is estimated that more than 100,000 of NVIDIA's H100 AI accelerators are present in the country, and they are being sold openly on Chinese marketplaces, notably Xiaohongshu and Alibaba's Taobao, and given the compact size of the chips, they are being smuggled all over the country. NVIDIA's H100 is being sold in China for around $23,000-$30,000, a drastic drop from its peak price when it was being sold in the black markets for around astronomical prices after the Biden administration halted the sales of the chip. Apart from this, multiple Chinese firms are setting up subsidiaries in other countries and then acquiring NVIDIA's AI chips in an attempt to work around the US policies, so it's safe to say that the sanctions haven't had much of an impact on the markets.

China has 'renting services' for NVIDIA AI GPUs, cheaper than the US at as low as $6 per hour

Chinese cloud service providers have "renting services" where they rent out their hardware stack, with prices that are radically cheaper than those available in the United States... as low as $6 per hour. The Financial Times reports that Chinese CSPs (cloud service providers) are renting out their AI GPU hardware, with small Chinese CSPs providing companies with an AI server that packs 8 x NVIDIA A100 AI GPUs that costs around $6 per hour to rent them out. If you're in the US, that would cost you around 50% more at $10 per hour. US sanctions to the side, NVIDIA's newer H100 and older A100 AI GPUs are easily available in China, which is why the rental costs are so much lower than other regions. It's estimated that over 100,000 x NVIDIA H100 AI GPUs are in China right now, and they're openly being sold on Chinese marketplaces, smuggled all across the country according to one Chinese startup founder. NVIDIA's H100 is sold in China for around $23,000 to $30,000, which is much cheaper than its peak when it was nigh impossible to buy higher-end AI chips in China because of US sanctions by the Biden administration.

Nvidia's AI chips are cheaper to rent in China than US

Supply of processors helps Chinese startups advance AI technology despite US restrictions. The cost of renting cloud services using Nvidia's leading artificial intelligence chips is lower in China than in the US, a sign that the advanced processors are easily reaching the Chinese market despite Washington's export restrictions. Four small-scale Chinese cloud providers charge local tech groups roughly $6 an hour to use a server with eight Nvidia A100 processors in a base configuration, companies and customers told the Financial Times. Small cloud vendors in the US charge about $10 an hour for the same setup. The low prices, according to people in the AI and cloud industry, are an indication of plentiful supply of Nvidia chips in China and the circumvention of US measures designed to prevent access to cutting-edge technologies. The A100 and H100, which is also readily available, are among Nvidia's most powerful AI accelerators and are used to train the large language models that power AI applications. The Silicon Valley company has been banned from shipping the A100 to China since autumn 2022 and has never been allowed to sell the H100 in the country. Chip resellers and tech startups said the products were relatively easy to procure. Inventories of the A100 and H100 are openly advertised for sale on Chinese social media and ecommerce sites such as Xiaohongshu and Alibaba's Taobao, as well as in electronics markets, at slight mark-ups to pricing abroad. China's larger cloud operators such as Alibaba and ByteDance, known for their reliability and security, charge double to quadruple the price of smaller local vendors for similar Nvidia A100 servers, according to pricing from the two operators and customers. After discounts, both Chinese tech giants offer packages for prices comparable to Amazon Web Services, which charges $15 to $32 an hour. Alibaba and ByteDance did not respond to requests for comment. "The big players have to think about compliance, so they are at a disadvantage. They don't want to use smuggled chips," said a Chinese startup founder. "Smaller vendors are less concerned." He estimated there were more than 100,000 Nvidia H100 processors in the country based on their widespread availability in the market. The Nvidia chips are each roughly the size of a book, making them relatively easy for smugglers to ferry across borders, undermining Washington's efforts to limit China's AI progress. "We bought our H100s from a company that smuggled them in from Japan," said a startup founder in the automation field who paid about 500,000 yuan ($70,000) for two cards this year. "They etched off the serial numbers." Nvidia said it sold its processors "primarily to well-known partners ... who work with us to ensure that all sales comply with US export control rules". "Our pre-owned products are available through many second-hand channels," the company added. "Although we cannot track products after they are sold, if we determine that any customer is violating US export controls, we will take appropriate action." The head of a small Chinese cloud vendor said low domestic costs helped offset the higher prices that providers paid for smuggled Nvidia processors. "Engineers are cheap, power is cheap, and competition is fierce," he said. In Shenzhen's Huaqiangbei electronics market, salespeople speaking to the FT quoted the equivalent of $23,000-$30,000 for Nvidia's H100 plug-in cards. Online sellers quote the equivalent of $31,000-$33,000. Nvidia charges customers $20,000-$23,000 for H100 chips after recently cutting prices, according to Dylan Patel of SemiAnalysis. One data center vendor in China said servers made by Silicon Valley's Supermicro and fitted with eight H100 chips hit a peak selling price of 3.2 million yuan after the Biden administration tightened export restrictions in October. He said prices had since fallen to 2.5 million yuan as supply constraints eased. Several people involved in the trade said merchants in Malaysia, Japan, and Indonesia often shipped Supermicro servers or Nvidia processors to Hong Kong before bringing them across the border to Shenzhen. The black market trade depends on difficult-to-counter workarounds to Washington's export regulations, experts said. For example, while subsidiaries of Chinese companies are banned from buying advanced AI chips outside the country, their executives could establish new companies in countries such as Japan or Malaysia to make the purchases. "It's hard to completely enforce export controls beyond the US border," said an American sanctions expert. "That's why the regulations create obligations for the shipper to look into end users and [the] commerce [department] adds companies believed to be flouting the rules to the [banned] entity list." Additional reporting by Michael Acton in San Francisco. © 2024 The Financial Times Ltd. All rights reserved. Please do not copy and paste FT articles and redistribute by email or post to the web.