Google DeepMind's AI Models Achieve Silver Medal Performance in International Mathematical Olympiad

Meet AlphaProof and AlphaGeometry2: The new AI mathematician overtaking humans?

The system solved about 83% of math Olympiad problems from the past 25 years As artificial intelligence (AI) models continue to develop, Google announces that two two AI systems from Google DeepMind together solved four of the six problems in this year's International Mathematical Olympiad. According to Google the AI systems competed with two silver medalists high school students, eventually beating them. During the competition AlphaProof and AlphaGeometry2 were able to solve a range of math problems step-by-step. This is expected to be a "grand challenge" in machine learning and has been beyond the reach of other state-of-the-art AI systems. The 'DeepMind' behind AlphaProof and AlphaGeometry2 Google explained that AlphaProof teaches itself by 'trial-and-error' method without human intervention. This method is known as reinforcement learning. The method is powered by DeepMind's Go-mastering AlphaGo, Starcraft-crushing AlphaStar and other AI systems developed by Google. According to sources, the team first fine-tuned Google's Gemini model to translate 1 million mathematics problem statements from English into a programming language called Lean. The difficult problems were then given to AlphaProof and AlphaGeometry2 so that they could generate potential solutions. Lastly, they were then checked against possible proof steps, initially made by experts. According to Google, the system solved about 83% of math Olympiad problems from the past 25 years compared to its predecessor that could only solve 53%. Moreover, both the AI systems AlphaProof and AlphaGeometry 2 scored 28 out of 42 possible points. "These are extremely hard mathematical problems and no AI system has ever achieved a high success rate in these types of problems," Pushmeet Kohli, vice president of research focused on AI for science, DeepMind, highlighted in a press briefing. AI solving math: Boon or Bane The success of AlphaProof and AlphaGeometry 2 in winning a silver medal at the International Mathematical Olympiad might mark a significant milestone in the development of AI. By automating repetitive and laborious tasks, artificial intelligence can help mathematicians focus on more theoretical and creative aspects of their work, thereby resulting in discoveries. However, it looks like this can also raise questions about the future of human intelligence. The growing concern in the global economy today is that AI might completely take over human jobs. Early reports suggest that to enhance productivity, enterprises are adopting AI-based technologies. Such initiatives are now replacing human jobs and making us more dependent on machineIt is expected that AI converting into perfectionists for 'logical reasoning' can add to these concerns. Another concern can be that the advancements of AI systems might result in increased 'cost structures' of companies. AI models mostly come at high cost structures. This might lead companies to initiate 'cost-cutting' through reducing employee count. Industry reacts "AlphaProof and AlphaGeometry2 highlights AI's growing capability to tackle complex mathematical problems. By successfully solving Olympiad-level problems, these systems demonstrate a remarkable combination of creativity and precise logical reasoning. This accomplishment underscores AI's potential to enhance human abilities across various scientific and engineering fields," Devroop Dhar, co-founder and managing director, Primus Partners, explained. "These models serve as a reminder of AI's enormous potential in advancing mathematics research and problem-solving skills. However, the idea that AI could surpass human mathematicians raises questions about the future of mathematics and how technology will affect our ability to comprehend complex problems," Heather Dawe, chief data scientist and head of Responsible AI, UK UST, said. Furthermore, "We need to understand that Mathematicians provide not just answers but also vital questions and abstract thinking that AI cannot yet achieve. While machines are proficient in executing calculations and solving practical problems, the creation of new mathematical theories and understanding of the deeper aspects of mathematics still rely on human ingenuity and creativity," Ganesh Gopalan, co-founder and CEO, Gnani.ai., concluded.

Google AI earns silver medal equivalent at International Mathematical Olympiad

AlphaProof and AlphaGeometry 2 solve problems, with caveats on time and human assistance On Thursday, Google DeepMind announced that AI systems called AlphaProof and AlphaGeometry 2 reportedly solved four out of six problems from this year's International Mathematical Olympiad (IMO), achieving a score equivalent to a silver medal. The tech giant claims this marks the first time an AI has reached this level of performance in the prestigious math competition -- but as usual in AI, the claims aren't as clear-cut as they seem. Further Reading Google says AlphaProof uses reinforcement learning to prove mathematical statements in the formal language called Lean. The system trains itself by generating and verifying millions of proofs, progressively tackling more difficult problems. Meanwhile, AlphaGeometry 2 is described as an upgraded version of Google's previous geometry-solving AI modeI, now powered by a Gemini-based language model trained on significantly more data. According to Google, prominent mathematicians Sir Timothy Gowers and Dr. Joseph Myers scored the AI model's solutions using official IMO rules. The company reports its combined system earned 28 out of 42 possible points, just shy of the 29-point gold medal threshold. This included a perfect score on the competition's hardest problem, which Google claims only five human contestants solved this year. A math contest unlike any other The IMO, held annually since 1959, pits elite pre-college mathematicians against exceptionally difficult problems in algebra, combinatorics, geometry, and number theory. Performance on IMO problems has become a recognized benchmark for assessing an AI system's mathematical reasoning capabilities. Further Reading Google states that AlphaProof solved two algebra problems and one number theory problem, while AlphaGeometry 2 tackled the geometry question. The AI model reportedly failed to solve the two combinatorics problems. The company claims its systems solved one problem within minutes, while others took up to three days. Google says it first translated the IMO problems into formal mathematical language for its AI model to process. This step differs from the official competition, where human contestants work directly with the problem statements during two 4.5-hour sessions. Google reports that before this year's competition, AlphaGeometry 2 could solve 83 percent of historical IMO geometry problems from the past 25 years, up from its predecessor's 53 percent success rate. The company claims the new system solved this year's geometry problem in 19 seconds after receiving the formalized version. Limitations Despite Google's claims, Sir Timothy Gowers offered a more nuanced perspective on the Google DeepMind models in a thread posted on X. While acknowledging the achievement as "well beyond what automatic theorem provers could do before," Gowers pointed out several key qualifications. "The main qualification is that the program needed a lot longer than the human competitors -- for some of the problems over 60 hours -- and of course much faster processing speed than the poor old human brain," Gowers wrote. "If the human competitors had been allowed that sort of time per problem they would undoubtedly have scored higher." Further Reading Gowers also noted that humans manually translated the problems into the formal language Lean before the AI model began its work. He emphasized that while the AI performed the core mathematical reasoning, this "autoformalization" step was done by humans. Regarding the broader implications for mathematical research, Gowers expressed uncertainty. "Are we close to the point where mathematicians are redundant? It's hard to say. I would guess that we're still a breakthrough or two short of that," he wrote. He suggested that the system's long processing times indicate it hasn't "solved mathematics" but acknowledged that "there is clearly something interesting going on when it operates." Even with these limitations, Gowers speculated that such AI systems could become valuable research tools. "So we might be close to having a program that would enable mathematicians to get answers to a wide range of questions, provided those questions weren't too difficult -- the kind of thing one can do in a couple of hours. That would be massively useful as a research tool, even if it wasn't itself capable of solving open problems."

Google AI narrowly misses Gold in International Mathematics Competition: Report

In a stunning display of mathematical prowess, Google's AI systems, AlphaProof and AlphaGeometry 2, have achieved silver medal-level performance at the prestigious International Mathematical Olympiad (via India Today). AlphaProof, a groundbreaking AI system introduced by Google, excels in formal mathematical reasoning, reported the publication. Utilizing a blend of language models and the AlphaZero reinforcement learning algorithm -- renowned for mastering chess and Go -- AlphaProof trains itself to tackle complex math problems using Lean, a formal language for mathematics. Demonstrating its capabilities, AlphaProof successfully solved two challenging algebra problems and one number theory problem during the IMO, including the competition's most difficult problem, a feat achieved by only five human contestants. Reportedly, the second AI system, AlphaGeometry 2, is a notable advancement over Google's earlier geometry-solving AI. Using a neuro-symbolic hybrid method, it integrates an advanced language model with a robust symbolic engine. This enhancement enabled AlphaGeometry 2 to solve intricate geometry problems more efficiently. During the IMO, AlphaGeometry 2 impressively solved Problem 4 in just 19 seconds, which involved complex geometric constructions and a deep understanding of angles, ratios, and distances. Trained on a vast dataset encompassing 25 years of historical IMO geometry problems, AlphaGeometry 2 boasts an impressive 83 per cent success rate in solving these challenges. Google's AI systems achieved a score of 28 out of 42 points at the IMO, falling just one point short of a gold medal. Renowned mathematicians, such as Fields Medal recipient Prof Sir Timothy Gowers and Dr. Joseph Myers, Chair of the IMO 2024 Problem Selection Committee, reviewed the AI's solutions. They concluded that the AI could produce impressive and non-obvious solutions, highlighting a significant milestone in AI's ability to perform complex mathematical reasoning. This achievement underscores Google's progress in advancing AI technology, with the potential to revolutionize various fields by assisting mathematicians in exploring new hypotheses, solving longstanding problems, and automating time-consuming elements of mathematical proofs. In the future, Google intends to share additional technical information about AlphaProof and to further investigate various AI methodologies to improve mathematical reasoning, adds the publication. Their goal is to create AI systems that collaborate with human mathematicians, thereby advancing the frontiers of science and technology. 3.6 Crore Indians visited in a single day choosing us as India's undisputed platform for General Election Results. Explore the latest updates here!

Google's AI solved four out of six problems in one of the world's hardest maths competitions, equivalent to a silver medal standard 'in a certain sense'

The International Mathematical Olympiad is not just a terrifying sequence of words for someone as maths-blind as myself, but also a notoriously challenging world championship mathematics competition for high school students from over 100 different countries. Each year students compete to show off their mathematical prowess in a chosen host country, each aiming to solve problems that would make the rest of us cower in fear. Google DeepMind has announced that two of its AI systems, AlphaProof and AlphaGeometry 2, took on this year's contest questions as a combined system. The AI had its solutions scored by previous gold-medalist winners Professor Sir Timothy Gowers and Dr Joseph Myers, the latter of which is Chair of the IMO 2024 Problem Selection Committee itself. Not only did the AI chalk up a combined score of 28 out of 42, one point off the 29 required for a gold medal, but also achieved a perfect score on the competition's hardest problem (via Ars Technica). Just as well really, as two combinatorics problems remained unsolved. Still, stick to what you're good at, ey? There's a slight fly in the ointment, however. In a Twitter thread, Prof Sir Timothy Gowers points out that while the AI did indeed score higher than most, it needed a lot longer than human competitors to do so. Human candidates submit their answers in two four-and-a-half-hour sessions -- and while one problem was solved by the AI within minutes, it took up to three days to solve the others. "If the human competitors had been allowed that sort of time per problem they would undoubtedly have scored higher," wrote Sir Gowers. "Nevertheless, (i) this is well beyond what automatic theorem provers could do before, and (ii) these times are likely to come down as efficiency gains are made." Not only that, but it's not like the AI sat down in front of a test paper and began chewing on its pencil. The problems were manually translated into Lean, a proof assistant and programming language, so the autoformalization of the questions was carried out by old-fashioned humans. Still, as the good Professor points out, what the AI has achieved here is a lot more involved and nuanced than simply brute forcing the problems: "We might be close to having a program that would enable mathematicians to get answers to a wide range of questions, provided those questions weren't *too* difficult -- the kind of thing one can do in a couple of hours." "Are we close to the point where mathematicians are redundant? It's hard to say. I would guess that we're still a breakthrough or two short of that."

Google DeepMind: New AI Models Can Earn Silver Medal in Math Olympiad

Google DeepMind introduced two new artificial intelligence (AI) models and said they correctly answered four out of six questions in a math competition that has become a benchmark measuring the capabilities of AI systems. The new AI models are a reinforcement-learning based system for formal math reasoning called AlphaProof and a new version of the company's geometry-solving system called AlphaGeometry 2, Google DeepMind said in a Thursday (July 25) press release. "Together, these systems solved four out of six problems from this year's International Mathematical Olympiad (IMO), achieving the same level as a silver medalist in the competition for the first time," the company said in the release. The IMO, which is a competition for elite pre-college mathematicians, has become a benchmark for an AI system's advanced mathematical reasoning capabilities, according to the release. After the problems for this year's competition were manually translated into formal mathematical language for the systems to understand, AlphaProof solved two algebra problems and one number theory problem, while AlphaGeometry 2 proved the geometry problem, the release said. The two combinatorics problems included in the competition remained unsolved, per the release. Earning a perfect score on each of the four problems they solved, the systems achieved a final score of 28 points -- equivalent to the top end of the silver-medal category and one point below the gold-medal threshold of 29, which was achieved by 58 of 609 contestants at the official competition, according to the release. "We're excited for a future in which mathematicians work with AI tools to explore hypotheses, try bold new approaches to solving long-standing problems and quickly complete time-consuming elements of proofs -- and where AI systems like Gemini become more capable at math and broader reasoning," Google DeepMind said in the release. Bloomberg reported Thursday that solving math problems has become a key proof point in the AI industry, where it's difficult to compare different models. Large language models tend to have greater linguistic intelligence than mathematical intelligence, per the report. PYMNTS reported in November that an AI model capable of doing math reliably is an enticing concept because math represents a foundation of learning for other, more abstract tasks.

Google latest models prove AI doesn't have to suck at math

Sure, it took three days to do what teenaged brainiacs do in nine hours - but who's counting? Researchers at Google DeepMind claim they've developed a pair of AI models capable of taking home a silver medal in the International Mathematical Olympiad (IMO) - although not within the allotted time limit. Dubbed AlphaProof and AlphaGeometry 2, these models are designed to help solve one of the bigger hurdles facing popular AI systems today: thanks to limitations in artificial reasoning and training data, they kind of suck at math. To overcome this, DeepMind developed AlphaProof - which combines a language model with its AlphaZero reinforcement learning algorithm. You may recall this is the same reinforcement model that DeepMind used to master chess, shogi and Go a few years back. AlphaProof trains itself to prove mathematical statements using a functional programming language called Lean. "Formal languages offer the critical advantage that proofs involving mathematical reasoning can be formally verified," the DeepMind team wrote in a recent blog. This means that AlphaProof can not only provide an answer, but prove that it's correct. This differs from existing natural language processing, which will confidently hallucinate a plausible sounding answer - but doesn't actually know whether it's correct. That's not to say NLP or LLMs can't be useful. DeepMind fine-tuned a Gemini LLM to translate natural language problem statements into ones the AlphaZero algorithm can interpret and use. "When presented with a problem, AlphaProof generates candidates and then proves or disproves them by searching other possible proof steps in Lean. Every time a solution is verified it's used to reinforce the model. The DeepMind team did this for millions of problems leading up to the IMO competition - in fact, because it's a reinforcement model, training continued throughout the competition. AlphaGeometry 2, on the other hand, functions in much the same way as its predecessor, detailed back in January. It combines a neural language model with a "rule-bound deduction engine" and the two work together to find proofs for problems. As DeepMind explains it, the language model is used to identify patterns and suggest useful constructs. The symbolic engine, meanwhile, uses formal logic to arrive at solutions. The downside to this approach is the second step is comparatively slow. For its second-gen AlphaGeometry model, DeepMind explained the model's language processing is based on Gemini and trained using an "order of magnitude" more synthetic data than its predecessor. The Symbolic Engine has also been sped up considerably and is said to be "two orders of magnitude" faster. To put these to the test, the DeepMind team tasked AlphaProof and AlphaGeometry 2 with solving the six advanced mathematics problems faced by competitors in this year's IMO. The competition - which dates back to 1959 and sees pre-college mathematicians tackle some of the hardest problems in algebra, combinatorics, geometry, and number theory - has become something of a proving ground for machine learning devs in recent years. According to DeepMind, the two models were able to complete four of the six problems - AlphaProof solved two algebra problems and one number theory problem, and AlphaGeometry 2 tackled this year's geometry problem. Unfortunately, the latter was no match for the two combinatorics questions. Tallied up, DeepMind's models still did fairly well, with a score of 28 out of 42 - the equivalent of a silver medal and one point off from gold. However, there seems to be plenty of room for improvement. At least for this competition, DeepMind conceded it was still necessary to translate the problems given to competitors manually into a formal mathematical language the models could understand. The models also failed to solve the majority of the problems within the allotted time period - which spans two 4.5 hour sessions, spread across two days. While the researchers were able to solve one of the problems within a few minutes, the others took upwards of three days to solve. The DeepMind researchers are not done yet, of course. They report they are already experimenting with a natural language reasoning system, built on Gemini, that wouldn't require problems to be translated into a formal language and could be combined with other AI systems. That should speed things up a bit. ®

Google AI systems make headway with math in progress toward reasoning

Alphabet's Google unveiled a pair of artificial intelligence systems on Thursday that demonstrated advances in solving complex mathematical problems, a key frontier of generative AI development. The current class of AI models, which work by statistically predicting the next word, have struggled with abstract math, which requires greater reasoning capabilities resembling human intelligence. DeepMind, the company's AI unit, published results showing that its new AI models in development, called AlphaProof and AlphaGeometry 2, solved four out of six questions at the 2024 International Math Olympiad, a prominent competition for high school students. Google said in a blog post that one question was solved within minutes, but others took up to three days, longer than the competition's time limit. Still, the results represent the best marks in the competition by an AI system to date. (For top technology news of the day, subscribe to our tech newsletter Today's Cache) Folds and faults: On AlphaFold 3 The company said it created AlphaProof, a system focused on reasoning, by combining a version of Gemini, the language model behind its chatbot of the same name, with AlphaZero, another AI system which previously bested humans in board games such as chess and Go. AlphaProof solved three of the competition's problems, including the most difficult question, which was solved by just five out of more than 600 human contestants. An additional math problem was solved by AlphaGeometry 2. Reuters reported earlier in July that Microsoft-backed OpenAI was developing reasoning technology under the code name "Strawberry." The project, formerly known as Q*, was considered so much of a breakthrough that several staff researchers wrote a letter to OpenAI's board of directors in November warning that it could threaten humanity, as Reuters first reported. Read Comments

AI Will Win IOI (Not IMO) Gold in 2025

AI's journey towards excellence in mathematics is marked by both impressive breakthroughs and persistent challenges. Adam D'Angelo, the CEO of Quora, recently posed a question on the timeline for AI to achieve a gold medal in the International Olympiad in Informatics (IOI). He inquired about predictions, if any, regarding when AI might reach this significant milestone. In response, one-third of the participants estimated that AI would achieve this level of success by the year 2025. The estimate isn't way-off to be honest. Recently, DeepMind's AlphaProof and AlphaGeometry 2 AI models worked together to tackle questions from the IMO. The DeepMind team scored 28 out of 42 - enough for a silver medal but one point short of gold. And it's not just about maths. DeepMind has a history of beating humans in other sports. Systems like AlphaGo went on to master the games of chess, Go and Shogi. It even put some of the world champions to the ground. "Even if I become the number one, there is an entity that cannot be defeated," said Lee Se-dol, the South Korean Go champion, who was defeated by AlphaGo by 4-1. In the medical field, DeepMind has developed a AlphaFold 3 that can accurately predict the structure that proteins will fold into in a matter of days, solving a 50-year-old "grand challenge" that could pave the way for better understanding of diseases and drug discovery. Before this year's competition, AlphaGeometry 2 could solve 83% of all historical IMO geometry problems from the past 25 years, compared to the 53% rate achieved by its predecessor. For IMO 2024, AlphaGeometry 2 solved Problem 4 within 19 seconds after receiving its formalisation. The breakthrough is being celebrated in the developers' community. OpenAI research scientist Mo Bavarian, who had won a silver medal in IMO, said that he could have imagined that a computer system would achieve a similar feat within his lifetime. "And yet here we are," he added. Scott Wu, the builder of Cognitive Labs, reflected on this achievement with amazement. "Olympiads were my whole life as a kid. Never thought they'd get solved by AI just ten years later," he said. Google's senior product manager, Logan Kilpatrick, emphasised the significance of this accomplishment in the broader context of AI development. "Models that can solve really hard maths and physics problems are on the critical path to AGI, and today we took another step towards that," he said. In recent times, AI models have made tremendous progress in the field of mathematics. Models like AlphaGeometry have demonstrated remarkable problem-solving abilities, rivalling human experts. As per Nature, the AlphaGeometry software, created by AI experts at Google DeepMind, accurately responded to 25 of 30 queries posed to it. "Astonishing and amazing," is how IMO president Gregor Dolinar described these outcomes. AlphaGometry is built by a group at Google DeepMind and New York University led by Trieu H Trinh. It uses a combination of symbolic AI, which Luong characterises as accurate, and a neural network that is more akin to LLMs, which handles the rapid, imaginative aspect of problem-solving to provide answers to maths problems of this level. Additionally, AlphaGeometry solved a problem from the 2004 IMO that had eluded specialists in a more general way. Then there is NuminaMath 7B TIR, a joint collaboration between Numina and Hugging Face, which managed to solve 29 out of 50 problems in the AI Maths Olympiad. NuminaMath is a mix of open-source libraries, notably TRL, PyTorch, vLLM, and DeepSpeed. Last week saw another new model for maths reasoning MathΣtral. It is tailored to tackle complex, multi-step logical reasoning challenges in STEM fields. For instance, MathΣtral 7B achieves significant accuracy enhancements, scoring 68.37% on maths through majority voting and 74.59% with a strong reward model among 64 candidates. OpenAI is also working on a new AI technology under the code name 'Strawberry'. This project aims to significantly enhance the reasoning capabilities of its AI models. With enhanced problem-solving abilities, AI could solve complex mathematical problems, help in engineering calculations, and even participate in theoretical research. As per reports, Strawberry scored 90% on the maths test for neural networks. The success of AlphaGeometry and NuminaMath in solving IMO geometry problems suggests that AI may soon be able to compete with the best human minds in mathematics. In the medical field, researchers from Google and DeepMind have developed Med-Gemini, a new family of highly capable multimodal AI models specialised for medicine. The models outperformed human experts on tasks such as medical text summarisation and referral letter generation. On the MedQA benchmark, which assesses medical question-answering abilities, Med-Gemini achieved an accuracy of 91.1%, surpassing the previous best by 4.6%. In multimodal tasks, the models outperformed GPT-4 by an average of 44.5%. Most of the AI systems that exist today are being built by past Olympiad champions. In fact, Prafulla Dhariwal, who played a significant role in the making of GPT-4o, represented India in IMO. Scott Wu, the brilliance behind Devin - touted as the most capable autonomous coding agent - is known for solving complex mathematical problems at the back of his hand. His brother Neal Wu, who is also building Cognition Labs is also a maths legend himself. Demis Hasabis, the co-founder of DeepMind, has won the World Games Championships at the Mind Sports Olympiad a record five times. According to venture capitalist and Meta board director Peter Theil, it will take at least 3-5 years for AI systems to possess the capability to solve all problems presented in the prestigious International Mathematical Olympiad. Barring a few AI models, a majority of them have failed in IMO tests. Answering questions correctly requires mathematical creativity that AI systems have long struggled with as put out by Microsoft engineer Shital Shah in a post on X. GPT-4, for instance, which has shown remarkable reasoning ability in other domains, scored 0 per cent on IMO geometry questions, while specialised AIs struggle to answer as well as average contestants. When tested, GPT-4, GPT-4o, and Claude 3.5 Sonnet all failed to solve the first IMO question correctly. While pointing out incorrect cases helped Claude 3.5 Sonnet briefly, it ultimately continued on the wrong path. Also, while AlphaProof and AlphaGeometry 2 were able to score perfect marks in four of the six questions, in the other two they were unable to even begin working towards an answer. Moreover, DeepMind, unlike human competitors, was given no time limit. While students get nine hours to tackle the problems, the DeepMind systems took three days working round the clock to solve one question, despite blitzing another in seconds. Eureka Labs founder Andrej Karpathy has also proved that models exhibit puzzling inconsistencies, such as struggling with seemingly simple tasks. AI's journey towards excellence in mathematics is marked by both impressive breakthroughs and persistent challenges. As AI systems continue to evolve and improve, they bring us closer to the possibility of achieving gold medals in prestigious competitions like the IOI.