Hand-cranked AI device proves edge inference can work without massive data centers

Hand-cranked AI box lets you get a workout while you wait for answers

Datacenters got you down? Worried that even the most innocuous questions will spin up AI models running in water-guzzling, energy-sucking, planet-destroying hyperscalers? You need CrankGPT. No, we're not talking about surrendering to AI psychosis: we're talking about a literal hand-cranked machine loaded with a voice agent that can respond to questions and even translate speech into other languages, provided someone keeps the power flowing. There's an onboard custom-built capacitor board to store some juice, mind you, but it only provides around 20 seconds of crank-free runtime before you've gotta keep crankin' to keep it alive. That, and it takes a bit of time to get it running - according to the documentation website, it's a 30-second process "from the moment you start cranking to the moment you're having a conversation with CrankGPT." According to the AI expert duo behind the device, computer scientist Katrin Tomanek and former Google Advanced Technology and Projects Group technical project lead Alex Kauffmann, CrankGPT still delivers impressive results despite the need to perform some hard physical labor for your tokens (though we'd argue some exercise for your AI might not be a bad thing). "Asking Claude to add two numbers for you is like swatting a fly with a wrecking ball," Kauffmann told The Register in an email. This tongue-in-cheek demonstration, Kauffmann said, may be a bit of light fun, but it's an exercise in demonstrating what his and Tomanek's AI company, Squeez, is all about: small, private specialized AI models that, in a pinch, might not even need very much energy or a connection to the web to operate. "Squeez produces customized, efficient, and private models that can run on small, inexpensive hardware to solve specific problems," Kauffmann explained, citing tasks like voice recognition for someone with a strong accent or speech impediment, or specially-trained, local AIs that are subject matter experts in topics like gardening or auto repair, but won't touch subjects outside their wheelhouse. Contrary to the flashy dot-com for CrankGPT the pair have set up, Kauffmann told me, Squeez has no plans to pursue spin cycle class-powered AI stacks for dev teams, though he said if anyone wants to foot the bill, he'd be happy to give it a shot. "Off-the-shelf bike generators are shockingly expensive and they're fussy to build," Kauffmann said. Still, "a good biker can maintain a steady 120W output, so a class of twenty could power a Blackwell." Speaking of wheelhouses, what's inside that box? If there's a tiny computer in a 3D-printed box with a crank attached, there's a good possibility it's going to be a Raspberry Pi, and that's the case here. CrankGPT's brain is built on a stock RPi 5 with 8 GB of RAM and a cooling fan HAT, and audio input and output are handled by a dedicated I/O HAT designed for voice assistants running RPis. Power comes from the aforementioned crank, which is actually an off-the-shelf 20W switchable voltage hand crank unit built for emergency USB device charging, and is stored in the custom capacitor unit the duo built. "The neatest part of the whole thing is that you can actually feel the inference," Kauffmann told us. "The amount of resistance the crank presents varies depending on the amount of work the board is doing, so when it's really working (generating words for instance), the crank becomes much harder to turn than when it's idling waiting for you to say something." As for software, the device is running the most stripped-down, bare bones instance of DietPi the pair could compile, which is able to boot into a functional userspace in about three seconds. The voice agent is the truly original piece of work done for the project, as detailed in the documentation page, and was built entirely from scratch. "We wanted to understand the system end to end and have as few dependencies as possible," the documentation page notes. It's available on GitHub for those interested in trying it out. Speech recognition is handled by the Moonshine automatic speech recognition engine, chosen for its speed, while text-to-speech synthesis is handled by Piper, chosen again for its low-resource edge inference capabilities. As for the models running on the thinking itself, there are a few that are behind CrankGPT, with Liquid LFM2 1.2B providing a general-purpose voice agent, and Gemma 3 1B being used for translation. CrankGPT can switch between translation and various prompts (e.g., general question answering and games like two truths and a lie) via a knob on the side of the enclosure. "It's entirely configurable," Kauffmann told us. "We added a couple of physical inputs (the knob, a button, a switch) to make experimentation easier." Kauffmann added that he and Tomanek were surprised by how well the translation function worked. "We did no fine tuning, it's just a two-line prompt and it works really well for high-coverage languages," he explained. While the demonstration focuses on audio prompts and responses, Kauffmann explained that the device supports all sorts of different models, with the only real limitation being inference time and the amount of hand cranking one wants to do to get their response. "We've generated images (small), made poetry (bad), and written code using the same setup," the CrankGPT makers wrote in their documentation, all with "a hand crank, a little computer, and a small stack of speech and language models running locally." If you're interested in building your own CrankGPT model, keep an eye on the documentation page we linked earlier in this story, as Kauffmann told us he and Tomanek are planning to release all the plans and schematics in the coming days, while the aforementioned custom voice agent is already available for tinkering. "It's a pretty straightforward setup, the only tricky part is that SBCs like the Raspberry Pi will sometimes draw enough current to trigger a little generator's overcurrent protection," Kauffmann told us. If you have a spare $300 lying around (that's what Kauffmann estimates the RAM pricing surge has driven the build cost up to, from the $150 he spent when building CrankGPT last year), then you, too, may soon be able to build your own completely off-grid, standalone AI box so you can keep chatting with your favorite micro LLM if and when its bigger cousins knock the grid offline. ®

There's one AI machine that doesn't need a nuclear power station to run, and it points to a potential way forward in the memory crisis

AI is ruining everything, right? The economy, air quality, and computing as a whole have all felt the impact of the exponential growth in data centers for machine learning. However, there's one little hardware AI project that proves bigger isn't always better, and even shows a possible way out of the current memory crisis. It's called CrankGPT by Squeez Labs (via Hackaday) and the gist of it is simple: Have a little microcomputer run a tiny local model for AI voice assistance, then stick it in a box and power the whole thing with a hand crank. No massive power station, no endless racks of GPUs, no DRAM-destroying demands. The computer in question is powered by a standard 8 GB Raspberry Pi and pretty much nothing else. It handles the voice recognition node, the local LLM (large language model), and the text-to-speech stuff. CrankGPT's creators built their own edge voice agent to process the complete algorithm (i.e. voice input > LLM stage > text-to-voice output). There's a brief demo of CrankGPT in action at the bottom of the webpage for the project, and it seems to work pretty well. Of course, there are strict limitations as to what it can do, as the Raspberry Pi 5 isn't exactly designed to be an inference powerhouse. It also takes roughly 30 seconds of cranking for the system to boot and be ready for any input, too. What interests me most about CrankGPT is the fact that, as a proof of concept, it shows that edge AI has a clear future ahead of it. Being entirely offline and with a local LLM, it's unmatched for privacy, but I reckon there's something more significant here. The hardware required for this is extremely light: just a little processor, 8 GB of LPDDR4X, and a small SD card to host the OS and required data. Video credit: Squeez Labs AI training is always going to be done via hulking data centers, but ChatGPT shows that you don't need the same for small-scale inference. If a hand-powered box can do it, then so can a basic laptop, phone, or even a watch. This hardware already exists on a vast scale across the world; all that's needed are the right AI models and agents to make it all work as intended. Should inference truly head off in that direction, it could significantly lessen the rampant demand for the kind of DRAM and NAND flash used in massive AI machines, and thus help bring an end to the current memory crisis. With hundreds of billions of dollars invested in AI training and inference, though, there's not much impetus for the industry to scale things right back and target the hardware that we already have. But wholesale change rarely happens overnight; all that's needed is for someone to show the way forward, and that's what CrankGPT has done.