Sipeed K3 RISC-V SBC runs 30B AI LLMs locally at 15 tokens per second for $299

Sipeed K3 Brings Local Inference to Open Architecture

Sipeed has launched its K3 series single board computers, marking a significant step forward for RISC-V SBC platforms in AI applications. Built in partnership with SpacemiT, a fabless Chinese semiconductor designer, these compact systems can run 30B-parameter LLMs locally at speeds ranging from 10 to 15 tokens per second 2

2

. The K3 series starts at $299 for the 8GB model and scales up to $629 for the 32GB flagship configuration, positioning itself as an open-source alternative to proprietary AI hardware for enthusiasts and researchers exploring local inference capabilities .

SpacemiT Fusion Architecture Powers AI Matrix Units

At the heart of the Sipeed K3 lies SpacemiT's Key Stone K3 SoC, featuring what the company calls "Fusion Architecture" with dedicated matrix multiplication blocks. The chip integrates 8 X100 cores for general-purpose computing, each equipped with 4MB of L2 cache and performing comparably to ARM's Cortex-A76 core . Alongside these sit 8 A100 AI matrix units with Tightly Coupled Memory, supporting up to 1024-bit RVV 1.0 vector processing . The entire system operates at 2.4 GHz and delivers up to 130,000 DMIPS for general-purpose computing 2

2

.

60 TOPS NPU Handles Quantized LLMs

The 60 TOPS NPU built into the K3 supports multiple data types including BF16, FP16, FP8, INT8, and INT4, providing flexibility for running quantized LLMs . Unlike traditional NPU designs that operate separately, both X100 cores and A100 AI matrix units connect to the memory controller via a coherent interconnect bus, enabling zero-copy operations where CPU and AI cores share the same memory space . The dual 32-bit controllers support LPDDR4x-4200 and LPDDR5-6400 memory, delivering up to 51GB/s of bandwidth . Sipeed demonstrates the platform running Qwen-3.5 35B at 15 tokens per second, with the board scoring an 84% intelligence rating of a 235B model 2

2

.

Pico-ITX and CoM260 Form Factors Target Edge Applications

Sipeed offers the K3 in two distinct form factors designed for Edge applications and networking use cases. The K3 CoM260 Kit measures 69.6mm x 45mm and features a 260-pin SO-DIMM slot, making it pin-compatible with NVIDIA's Jetson Orin Nano carrier boards 2

2

. The Pico-ITX version resembles a Raspberry Pi at 100mm x 86mm, featuring 2 USB Type-C ports with Power Delivery and Alt-DP support, plus 1 10GbE port and 1 1GbE port . The SoC operates at a TDP of 15-25W, making it suitable for compact deployments . Both platforms officially support Ubuntu 26.04 and ROS, providing a full LTS environment .

Practical Path for AI LLMs on Open Instruction Set

The 32GB version can accommodate a quantized version of Qwen 3.6 A3B 35B requiring approximately 22GB, though space constraints mean smaller models like Gemma 4 26B A4B at around 15GB may prove more practical . Sipeed offers three memory configurations: 8GB, 16GB, and 32GB, with pricing ranging from $299-$309 for the entry-level 8GB models to $629-$639 for the top-tier 32GB configurations, with a $10 difference between the Kit and ITX board versions 2

2

. As one of the first RVA23-compliant platforms with full Ubuntu 26.04 LTS support, the K3 represents a milestone for researchers and enthusiasts seeking to explore local inference on an open instruction-set architecture . While it won't challenge NVIDIA's dominance in high-end GPUs, the platform offers a practical entry point into the RISC-V landscape for AI experimentation and development.