AI enthusiast adds Nvidia Tesla V100 to gaming PC for $266 to run local LLM at 32 tokens per second

AI enthusiast adds Nvidia Tesla V100 as loud as a lawnmower to gaming PC for $266 -- 32GB of VRAM rig can run 27 billion parameter model at 32 tokens per second

Now they have a total 32GB VRAM system on a budget, for local LLM inference. A computing enthusiast has repurposed a very noisy and largely obsolete enterprise GPU (with lots of VRAM) for local LLM inference purposes. They are now enjoying a system that has doubled its total VRAM quota to 32GB for just a $266 (£200) outlay. That's a good result, especially in the midst of a RAMpocalypse. Oscar Molnar explains that a cheap Tesla V100 SXM2 with 16GB HBM2 was sourced, as was an SXM2-to-PCIe adapter, and a PWM mod for the loud-as-a-lawnmower cooler, to complete this VRAM expansion for the hefty local LLMs project. Indeed, these GPUs do look cheap right now, as I can see them listed on eBay US for under $140 each, if you don't mind buying from China. As mentioned above, you can't just get one of these Tesla V100 SXM2 cards with abundant VRAM and plug it into your PC. Molnar says they spent about $66 on an SXM2-to-PCIe adapter, also on eBay. You might think that was enough. However, the PC and local LLMs enthusiast baulked at the noise of "the fan from hell," which came as standard with the Tesla V100 SXM2. That shrieking cooler was measured outputting 82dB of noise. Molnar described it as "somewhere between a garbage disposal and a lawnmower." This may be the most complicated tweak yet, but basically the existing fan wires just needed rerouting and plugging into the motherboard PWM fan header. You could also simply purchase a "2.54mm male to PH2.0 female jumper cable" for the task. Apparently, the fan only needs to run at 10% to keep the Tesla V100 under 50C at full load. 27 billion parameter LLM runs at 32 tokens per second With the hardware all now fitted and finessed, Molnar had a 32GB VRAM system at their disposal - that's a PC with RTX 4080: 16GB VRAM, Ada architecture and Tesla V100: 16GB VRAM, Volta architecture. They note you can get Tesla V100s with 32GB of VRAM, but they are double the price. Getting the system to make use of this 32GB of total VRAM for LLMs wasn't tricky, says the DIYer. They used NixOS with a legacy Nvidia driver that overlapped support for both Volta and Ada architectures. Testing a local LLM, they got a 27 billion parameter model running at 32 tokens per second, which they say is "fast enough for interactive use" and faster than most cloud API alternatives. Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.

A Modder's RTX 4080 Was Enough To Play AAA Games, But Not For Running LLMs, So He Integrated NVIDIA's Tesla V100 At A Throwaway Price To Run 27B AI Models

Running higher-quality AI models means your existing GPU will have to be equipped with a ton of VRAM to make the experience enjoyable. For one RTX 4080 owner, playing the most visually taxing and graphically demanding games might be a walk in the park, but for running LLMs, it's a Herculean task. Wanting to accomplish both feats in a single gaming PC, a modder successfully ran NVIDIA's Tesla V100 in his system, but encountered a few challenges along the way. Adding the Tesla V100 grants the modder 32GB of usable VRAM, sufficient for running significantly improved AI models like Qwen 3.6 at 32 tokens/second Before you ask, it's impossible to attach the Tesla V100 to a desktop motherboard like a "plug and play" job, requiring Tymscar to purchase an SXM2-to-PCIe adapter. Sourcing the GPUs with 16GB of HBM2 memory and the accessory cost him £200, which translates into roughly $266. On eBay, you can grab these for only $100 apiece. With the Tesla V100's 5,120 CUDA cores and 4,096-bit bus width that delivers 900GB/s of bandwidth, the GPU still has some computing juice remaining. Successfully sourcing an SXM2-to-PCIe adapter wasn't the most difficult of challenges, but it isn't simple either, especially when you find out later that the Tesla V100 doesn't have a PCIe slot, display inputs, or PCIe power connectors. As you can see in the image below, attaching the GPU to the vapor chamber-style cooler might be perfectly fine for those who don't mind the excessive noise, but at 82dB, it'll make anyone uncomfortable. With a little tweaking here and there to reduce the fan noise by using a 9V battery and a PWM jumper, the V100's modded cooler was now operating at 10 percent of the original maximum RPM. With this problem out of the way, the modder had successfully found a way to get 32GB of usable VRAM into his system. Now, no game will ever require a whopping 32GB of video memory, unless you decide to run a newer title at 16K resolution, so the best use case would be to fire up Qwen3.6 27B. The modder was running Qwen3.6-27B-MTP quantized at Q5_K_M, which comes in at 19GB, and with a context size of 128K tokens, there was sufficient VRAM to run the LLM at 32 tokens per second. Prompt processing was between 133 and 160 tokens per second, making it decent performance if you happen to stumble across previous-generation AI GPUs for home-computing purposes. Best of all, for less than $300, you can have your very own small to medium-sized AI models running at home, free of cost and without any internet connection. News Source: Tymscar Follow Wccftech on Google to get more of our news coverage in your feeds.