Apple's AI Advancements: Leveraging Google's Custom Chips for iPhone Intelligence

Apple Intelligence trained on Google's custom chips, rather than on hardware from Nvidia

Apple announced iOS 18 at its WWDC developers conference on June 10. One of the biggest software updates we've ever seen, iOS 18 brings some incredible new features and more customization options to the iPhone than ever before. But the biggest addition is Apple Intelligence - Apple's set of AI features. All these features are powered by Apple's own AI models. What makes these models so special is that they work entirely on your device, rather than sending things to servers. Now, we know a little bit more about how Apple trained these AI models. Per an official Apple research paper, the tech giant trained its models using Google's custom chips rather than hardware from Nvidia. It turns out, Apple ditched Nvidia and instead opted for Google's TPUv4 and TPUv5 chips to churn through the mountains of data needed for their Apple Intelligence Foundation Language Models (AFMs). These AFMs are the brains behind the flashy Apple Intelligence features that are starting to roll out to developers. Apple's main LLM (large language model), had the muscle of 8,192 TPUv4 chips working in unison. Picture it as eight slices of 1,024 chips each. The training was very intense, involving a triple-stage process with trillions of tokens - 6.3 trillion to start, followed by a mere 1 trillion, and a final stretch with 100 billion tokens for context-lengthening. The data buffet for these AFMs was pretty lavish too, with contributions from the Applebot web crawler (following robots.txt, mind you), various licensed datasets, and a sprinkle of public code, math, and other datasets for good measure. The AFM-on-device model, the slimmer sibling designed for offline features, underwent some serious knowledge distillation. This model, a tidy 3 billion parameters, was distilled from the 6.4 billion parameter server model and trained using a single slice of 2,048 TPUv5p chips. In terms of performance, Apple claims its AFM-server and AFM-on-device models are top-notch, excelling in benchmarks like Instruction Following, Tool Use, Writing, and more.

The iPhone is about to get smarter, in part thanks to Google chips - 9to5Mac

Apple Intelligence just arrived in beta this week, and now the company has published an in-depth overview of how some of its AI features were created. A key tidbit? Two of Apple's foundation models were created using Google-made chips. Apple tends to shy away from sharing many details on its inner practices of product development. However, with AI and ML features, the company has long published its research for all to see. The latest publication is titled 'Apple Intelligence Foundation Language Models,' and it's one of the first papers since WWDC's introduction of Apple Intelligence. The document is written by researchers and for researchers, so it's not the easiest to parse. However, a standout tidbit involves the chips used to train two of the Apple Intelligence language models. In this report we will detail how two of these models -- AFM-on-device (AFM stands for Apple Foundation Model), a 3 billion parameter language model, and AFM-server, a larger server-based language model -- have been built and adapted to perform specialized tasks efficiently, accurately, and responsibly. These two foundation models are part of a larger family of generative models created by Apple to support users and developers; this includes a coding model (based on an AFM language model) to build intelligence into Xcode, as well as a diffusion model to help users express themselves visually, for example, in the Messages app. The two models mentioned, AFM-on-device and AFM-server, were not trained using Apple's in-house Apple Silicon chips. Instead, Apple turned to Google Tensor chips -- like what's found in Pixel phones -- to train its models. And according to the paper, it took a whole lot of Tensor chips to do the work. It is interesting that Apple went with Google's Tensor chips rather than the Nvidia chips that other companies tend to rely on. The paper doesn't get into an explanation of that, but perhaps future publications will. What should we make of this news? Without further details from Apple, it's hard to know what to think. Knowing Apple's preference for doing as much work in-house as possible, it's very possible the company has already moved on from Google Tensor for model training and is using an advanced version of Apple Silicon. In any case, what this does mean is that the iPhone getting a lot smarter with Apple Intelligence is, in part, thanks to Google.