A Reddit hobbyist is running Moonshot AI's Kimi K3, a model with 2.8 trillion parameters, on his own hardware at 3.11 tokens per second

1

2

. That is a frontier-class model, the kind normally confined to data centers, running on an Intel Xeon server paired with Intel Optane persistent memory and two Nvidia GeForce 3090 graphics cards

1

.

Source: Tom's Hardware

Source: Tom's Hardware

Why the model fits at all

The trick is the model's design. Per Tom's Hardware, the Kimi K3 that the builder runs is a mixture-of-experts design: it holds 896 expert networks but routes each token through only 16 of them, plus 2 shared ones

1

2

. That leaves about 104 billion of its 2.8 trillion parameters active per token

1

2

. Moonshot released the model in July 2026, and demand for it overwhelmed the launch

2

.

The hardware split does the rest. The two GeForce 3090s provide 48 gigabytes of video memory, just 5.5% or so of the model's 865 gigabytes

1

2

. They handle the parts touched on every token, though: attention, dense layers, shared experts and the running context cache. The routed experts sit on the CPU side, stored in Optane plus ordinary memory

1

2

. Quantization shrinks the file too: the builder runs a compressed version averaging under 3 bits per parameter, down from Moonshot's own 4-bit release at roughly 1.56 terabytes

1

.

The memory drive did the heavy lifting

The latest speedup came from storage, not silicon. Moving 414 gigabytes of the model file, 45 of its 94 shards, into a drive held in system memory lifted generation speed 41% and prompt evaluation 26%

1

2

. With both cards in place and the memory drive absent, generation ran at 2.20 tokens per second

1

.

The data vanishes on reboot, but to the system it looks like a disk

1

. Ordinary memory in the machine sits between the two: slower than video memory, but faster than Optane, so shards held in it no longer have to be read off the slowest tier

1

.

What the speed cost

Getting the cards in meant cutting the case metal and relocating the power supply. Cooling remains the big unsolved problem. Two 12-watt blower fans still blow at the Optane memory modules, and the builder warns that sustained heat could damage the ordinary memory sitting next to them

1

2

.

The result matched what he expected. He had predicted, per the original build thread, "somewhere between 2 and 4 tokens per second", and the finished build landed inside that range

1

. It is still slow for practical use, though. That is still below average human reading speed, about 4 to 5 tokens per second, and the results are self-reported rather than independently verified

2

.

The project is not finished. Next on the list is customized quantization, tuning the compression per layer rather than uniformly, which might ease the pressure on both bandwidth and capacity

1

. The builder says the setup "might be faster yet", and notes output quality has to be weighed alongside speed

1

.

© 2026 TheOutpost.AI All rights reserved