Apple Minions turns idle Macs into virtual GPU cloud, slashing AI queue times from 10+ hours to minutes

Apple Minions Transforms Idle Macs Into Virtual GPU Cloud

Apple researchers have unveiled details about Minions, an internal distributed system that converts idle Macs into a virtual GPU cloud to handle AI workloads more efficiently. Described in a recent arXiv paper published on Apple's Machine Learning Research site, the system addresses a critical compute bottleneck that previously left some inference jobs waiting in GPU queues for over 10 hours1

1

. With Minions, tasks can begin execution within minutes by tapping into Apple Silicon GPUs across macOS devices on the network2

2

.

The system operates through a pull-based execution model with no central scheduler, allowing Mac devices to autonomously grab work from a shared queue. Teams can initiate requests as simply as sending a Slack message asking colleagues to run the Minion Executor and target specific projects or tasks1

1

. This informal participation model has proven effective at redistributing GPU resources that would otherwise sit unused across Apple's internal network.

Architecture Enables Fault-Tolerant Inference Distribution

Minions relies on four primary components working in concert: the Minion Server, PostgreSQL Metadata Store, Minion Executor, and Minion Watchdog2

2

. The Minion Server splits submitted tasks into subtasks stored in a PostgreSQL database, providing the coordination layer that receives user requests and makes them available for execution. On each participating Mac, the Minion Executor pulls the next subtask over REST APIs, reads inputs from Amazon S3, grabs the required model if needed, runs the subtask on its own GPU, then returns results to S31

1

.

The Minion Watchdog monitors this process for failures by tracking heartbeats that Macs send every five minutes. When a Mac goes to sleep, drops off the network, powers off, or when a user stops the executor to regain control of the machine, unfinished subtasks automatically return to the queue for reassignment1

1

. Database statistics feed into Grafana-based dashboards for real-time visibility into system performance2

2

.

System Handles Two Types of Machine Learning Inference Workloads

The system relies specifically on Apple Silicon for AI inference, handling two distinct types of jobs: embeddings with open-source Hugging Face transformer models, and text generation from large language models through Ollama, which is installed on each participating Mac1

1

. Each Mac runs a whole model on its own piece of work using unified memory, rather than splitting any single model across multiple devices. This approach mirrors what enthusiasts practice in home labs and resembles the volunteer computing model pioneered by SETI@home and Folding@home, which distributed small chunks of work to idle machines1

1

.

Mac owners can determine which jobs their machines prioritize through dynamic task binding, such as preferring one team's or project's jobs, or rely on the default first-in, first-out scheme1

1

. This flexibility allows teams to balance their compute needs while maintaining control over their hardware resources.

Data Confidentiality Concerns Persist Despite Internal Deployment

While Minions delivers significant benefits, data confidentiality remains a concern even for internal work. Teams worry about their data running on devices outside their group1

1

. Apple aims to mitigate these concerns with team- and organization-scoped deployments that use their own databases and separate pools of Macs1

1

. This segmentation approach allows different groups to maintain isolation while still benefiting from the distributed infrastructure.

Beyond reduced GPU queue times, the paper cites that requests with Minions do not require new GPU requests, infrastructure cost is zeroed, and scaling effort transitions from manual to organic1

1

. However, these zero infrastructure costs do not account for electricity consumption or hardware wear from increased utilization.

Future Plans Include Distributed Model Training and Global Federation

The researchers indicate that future work will extend Minions to distributed model training, expanding beyond its current inference-only capabilities1

1

. More ambitiously, they suggest that if idle Macs across organizations or potentially across the global community can be securely federated, they could form a shared GPU pool1

1

. For now, the Macs act as overflow AI capacity only inside Apple.

This development comes as Apple grapples with AI compute bottlenecks across its operations. In its quarterly filing in July, Apple acknowledged that it "depends on access to sufficient computing resources" in an ecosystem with "constrained supply, extended lead times, and increasing costs"1

1

. The risk that it may be unable to secure sufficient capacity, even with its own data centers and third-party cloud providers, makes the Minions approach increasingly practical. Apple's Private Cloud Compute, the server side of Apple Intelligence for more complex requests, now also runs on Nvidia GPUs in Google Cloud, first announced at WWDC in June1

1

. High Mac demand for local AI led then-CEO Tim Cook to warn in April that Mac mini and Mac Studio supply might take months to catch up1

1

.