Tiny Corp enables Nvidia GPUs on Macs with Apple-approved driver for AI workloads

Someone finally got an RTX 5090 running on a Mac - no hacks required

Serving tech enthusiasts for over 25 years. TechSpot means tech analysis and advice you can trust. In context: Back in 2018, Apple yanked Nvidia support from macOS entirely, and that was pretty much it for CUDA on the platform. Developers who wanted GPU compute from Team Green on their Macs were out of luck for years. But that's now changing. Tiny Corp, the same company that built the tinybox AI accelerator, has written its own Nvidia GPU driver completely from scratch. It's called TinyGPU, and it's an open-source macOS kernel extension. Better yet, Apple has signed off on it. That means you don't need workarounds like setting up a virtual machine or messing with System Integrity Protection to run it. All you need to do is plug in an external GPU over Thunderbolt or USB4, approve the extension, and it works. One YouTuber has already tested the driver using his RTX 5090 with 32GB of VRAM. Alex Ziskind plugged the GPU into a Mac Mini M4 Pro, and it ran just fine. That's impressive considering this is Nvidia Blackwell silicon hooked up to Apple Silicon via a single cable. But actual performance is a more complicated story. In an inference test using Llama 3.1 8B - a popular open-source AI model - the RTX 5090 managed roughly 7.48 tokens per second. Tokens per second is basically how fast an AI model can spit out its response, and that number isn't going to blow anyone away. Where things did get interesting, though, was in chat-style tasks. Time to first token, which is how long you wait before the response even starts appearing, was three to four times quicker than what you'd get from using Metal natively. That made the experience feel noticeably snappy, according to Ziskind. Now for the catch. Running llama.cpp, which is one of the most popular open-source tools for local AI inference, on Metal is still about ten times faster overall. That's a big gap, though it's not really a Thunderbolt bandwidth problem. According to Ziskind, the real culprit is kernel efficiency: the 5090's memory can do 1.8TB/s, but the driver is only reaching 33GB/s. Then again, this isn't meant to compete with llama.cpp right now. The point is that the driver, compiler pipeline, and memory management are all in place. With that out of the way, Tiny Corp can start working on optimizations.

Apple rubberstamps an open source driver to allow Nvidia GPUs to run on Macs, though gaming isn't on the table just yet

It wasn't that long ago that you could happily use an Nvidia graphics card in an Apple Mac, either for gaming or to use Team Green's CUDA ecosystem. But once Apple switched to its own rendering API, that was the end of it all. Until now, that is, because thanks to a new open-source driver, you can go all AppleCUDA once again. Coding and tech YouTuber Alex Ziskind (via Videocardz) has posted a video where they try out this new driver on a Mac mini, connecting a GeForce RTX 50-series graphics card to it via an eGPU dock, over a USB4 cable. The secret (well, not secret now) sauce is an app called TinyGPU, developed by tiny corp, which also created the special driver that it uses to make everything work. One important thing to note is that Apple has fully approved all of this, as well as AMD and Nvidia, so you don't need to be doing any kind of homebrew shenanigans or the like: just plug all your hardware together (making sure the graphics card is correctly powered), install the app and driver, setup your compiler, and you should be good to go. Now, before you get excited about being able to do some serious gaming on a Mac with that spare RTX 5090 you happen to have lying around, tiny corp's work is focused on AI only. And even then, Ziskind shows that there is still a lot of performance being left behind, especially with the 5090 that they used. In general, while all three tested RTX 50-series graphics cards crunched through considerably more tokens per second than the M4 Pro powering the Mac mini, the software stack wasn't making full use of the Blackwell GPU's capabilities. But this isn't to take anything away from what tiny corp has done, and if you have an AMD RDNA 3, Nvidia Ampere, or more recent GPU, then you'll be able to get stuck in yourself and see what kind of AI experiments you'll now be able to do on your Mac. Since all of tiny corp's GPU runtimes are open-source and available on GitHub, I wouldn't be surprised if someone figures out a way to get everything working with games, too, though this is likely to be one heck of a challenge.