2 Sources
[1]
Homelabber's second-gen Optane build uses two RTX 3090s to run 2.8-trillion-parameter Kimi AI model at 3.11 tokens per second -- moving nearly half the model into a RAM drive boosted performance by 41%
Faster now, but the builder says the Optane sticks still need fans of their own. A homelabber detailed improvements to the second-gen Optane AI build in a follow-up post on Reddit, showing the addition of two RTX 3090s to run the 2.8-trillion-parameter Moonshot Kimi K3 model at 3.11 tokens per
[2]
Two NVIDIA GeForce RTX 3090 GPUs and Intel Optane run 2.8-trillion-parameter Kimi K3 at 3.11 tokens/sec
A homelabber reports 41% faster text generation with Moonshot's Kimi K3 after moving 414GB of model data from Intel Optane into a RAM drive. A PC enthusiast has managed to run Moonshot AI's massive 2.8-trillion-parameter Kimi K3 language model locally at 3.11 tokens per second, using two NVIDIA
Share
Copy Link
A Reddit hobbyist is running Moonshot AI's Kimi K3, a model with 2.8 trillion parameters, on his own hardware at 3.11 tokens per second
1
2
. That is a frontier-class model, the kind normally confined to data centers, running on an Intel Xeon server paired with Intel Optane persistent memory and two Nvidia GeForce 3090 graphics cards1
.
Source: Tom's Hardware
The trick is the model's design. Per Tom's Hardware, the Kimi K3 that the builder runs is a mixture-of-experts design: it holds 896 expert networks but routes each token through only 16 of them, plus 2 shared ones
1
2
. That leaves about 104 billion of its 2.8 trillion parameters active per token1
2
. Moonshot released the model in July 2026, and demand for it overwhelmed the launch2
.The hardware split does the rest. The two GeForce 3090s provide 48 gigabytes of video memory, just 5.5% or so of the model's 865 gigabytes
1
2
. They handle the parts touched on every token, though: attention, dense layers, shared experts and the running context cache. The routed experts sit on the CPU side, stored in Optane plus ordinary memory1
2
. Quantization shrinks the file too: the builder runs a compressed version averaging under 3 bits per parameter, down from Moonshot's own 4-bit release at roughly 1.56 terabytes1
.The latest speedup came from storage, not silicon. Moving 414 gigabytes of the model file, 45 of its 94 shards, into a drive held in system memory lifted generation speed 41% and prompt evaluation 26%
1
2
. With both cards in place and the memory drive absent, generation ran at 2.20 tokens per second1
.The data vanishes on reboot, but to the system it looks like a disk
1
. Ordinary memory in the machine sits between the two: slower than video memory, but faster than Optane, so shards held in it no longer have to be read off the slowest tier1
.Related Stories
Getting the cards in meant cutting the case metal and relocating the power supply. Cooling remains the big unsolved problem. Two 12-watt blower fans still blow at the Optane memory modules, and the builder warns that sustained heat could damage the ordinary memory sitting next to them
1
2
.The result matched what he expected. He had predicted, per the original build thread, "somewhere between 2 and 4 tokens per second", and the finished build landed inside that range
1
. It is still slow for practical use, though. That is still below average human reading speed, about 4 to 5 tokens per second, and the results are self-reported rather than independently verified2
.The project is not finished. Next on the list is customized quantization, tuning the compression per layer rather than uniformly, which might ease the pressure on both bandwidth and capacity
1
. The builder says the setup "might be faster yet", and notes output quality has to be weighed alongside speed1
.Summarized by
Navi
26 Jul 2026•Technology

24 Oct 2026•Technology

27 Jul 2026•Technology
