Apple Unveils New AI Technique to Enhance User Experience and Compete with ChatGPT

Apple just revealed an AI technique to better compete against ChatGPT

Apple's new strategy reduces some of a newer LLM's mistakes by up to 40% When an AI lab updates its underlying large language model it can often result in unexpected behavior including a complete change to the way it responds to queries. Researchers at Apple have developed new ways to improve a user's experience when an AI model they were used to working with gets upgraded. In a paper, Apple's researchers said that users develop their own system to interact with an LLM, including prompt styles and techniques. Switching to a newer model can be a draining task that dampens their experience using the AI model. An update could result in forcing users to change the way they write prompts and while early adopters of models from ChatGPT might accept this, a mainstream audience using iOS will likely find this unacceptable. To solve this issue, the team looked into creating metrics to compare regression and inconsistencies between different model versions and also developed a training strategy to minimize those inconsistencies from happening in the first place. While it isn't clear whether this will be part of a future iOS Apple Intelligence, it's clear Apple is preparing itself for what happens when it does update its underlying models, ensuring Siri responds in the same way, to the same queries in future. Using their new method the researchers said they managed to reduce negative flips, which is when an old model gives a correct answer while a newer model gives an incorrect one, by up to 40%. The paper's authors also argued in favor of ensuring that mistakes a new model makes are consistent with those you might see an older model make. "We argue that there is value in being consistent when both models are incorrect," they said, adding that, "A user may have developed coping strategies on how to interact with a model when it is incorrect." Inconsistencies would therefore lead to user dissatisfaction. They called the method used to overcome these obstacles MUSCLE (an acronym for Model Update Strategy for Compatible LLM Evolution) which does not require the base model's training to be changed and relies on training adapters, which are basically plugins for LLMs. They referred to these as compatibility adapters. To test if their system worked, the research team updated LLMs like Llama and Phi and sometimes found negative flips of up to 60% in different tasks. Tests they ran included asking the updated models math questions to see if they still got the answer to a particular problem correct. Using their proposed MUSCLE system, the researchers say they managed to mitigate quite a number of those negative flips. Sometimes by up to 40%. Given the fast pace with which chatbots like ChatGPT and Google's Gemini are being updated, Apple's research has the potential to make newer versions of these tools more dependable. It would be a pity if users had to make tradeoffs between switching to newer models but suffering from a worse user experience.

Apple tackles one of the most frustrating aspects with AI | Digital Trends

As companies like Google, Anthropic, and OpenAI update and upgrade their AI models, the way that those LLMs interact with users is sure to change as well. However, getting used to the new system can become a hassle for users who then have to adjust how they pose their queries in order to get the results they've come to expect. An Apple research team has developed a new method to streamline that upgrade transition while reducing inconsistencies between the two versions by as much as 40%. As part of their study, "MUSCLE: A Model Update Strategy for Compatible LLM Evolution," published July 15, the researchers argue that when upgrading their models, developers tend to focus more on upping the overall performance, rather than making sure that the transition between models is seamless for the user. That includes making sure that negative flips, wherein the new model predicts the incorrect output for a test sample that was correctly predicted by the older model, are kept to a minimum. Recommended Videos This is because, the study authors argue, each user has their own quirks, quibbles, and personalized ways of interacting with chatbots. Having to continually adjust and adapt the manner in which they interact with a model can become an exhausting affair -- one that is antithetical to Apple's desired user experience. The research team even argues that incorrect predictions by the AI should remain between versions, "There is value in being consistent when both models are incorrect," they wrote. "A user may have developed coping strategies on how to interact with a model when it is incorrect." Apple presents MUSCLE A Model Update Strategy for Compatible LLM Evolution Large Language Models (LLMs) are frequently updated due to data or architecture changes to improve their performance. When updating models, developers often focus on increasing overall performance... pic.twitter.com/ATm2zM4Poc — AK (@_akhaliq) July 15, 2024 To address this, the researchers first developed metrics by which to measure the degree of regression between models and then developed a strategy to minimize their occurrence. The result is MUSCLE, a strategy that doesn't require developers to retrain the entire base model and instead relies on the use of training adapters. Adapters small AI modules that can integrate at different points along the overall LLM. Developers can then fine-tune these specific modules instead of the entire model. This enables the model as a whole to perform distinct tasks at a fraction of the training cost and with only a small increase in the number of parameters. They're essentially plug-ins for large language models that allow us to fine-tune specific sections of the overall AI instead of the whole thing. The research team upgraded LLMs including Meta's Llama and Microsoft's Phi as part of their study, using specific math queries as samples, and found that negative flips occurred as much as 60% of the time. By incorporating the MUSCLE strategy, the team wasn't able to fully eliminate negative flips, but they did manage to reduce their occurrence by as much as 40% compared to the control.