Mathematics Unveils the Inner Workings of AI: PNNL Researchers Bridge the Gap

How math reveals the bleeding edge of AI

Mathematicians love the certainty of proofs. This is how they verify that their intuition matches observable truth. This logical love of proofs is uniquely suited to opening up the black box of artificial intelligence (AI) to scrutiny with the goal of establishing trust. Mathematics and AI may seem to be natural partners, but AI is only just beginning to receive substantial attention from the mathematics community. At Pacific Northwest National Laboratory (PNNL), mathematicians are exploring the limits of what AI can achieve while assuring that we can believe what it says to us. The relationship between mathematics and AI Henry Kvinge, a mathematician at PNNL who is deeply involved in AI research, breaks down the relationship into three buckets that each play a vital role in harnessing the potential of AI systems. Math to understand AI: Math offers powerful tools for deciphering the inner workings of complex AI models, he says. For instance, researchers strive to decode what happens within neural networks -- the black boxes of AI -- to ensure that the decisions they make do not rely on misguided reasoning or faulty logic. Mathematics helps dissect these complex systems, identifying underlying failures and optimizing performance. Recently, Kvinge wrote an article that summarizes his thoughts on how math concepts like curvature, shape, and symmetry provide toolkits for analyzing the accuracy of a given AI model for an assigned task. Math to inspire AI development: Mathematics provides the language to describe the high-level design principles of AI systems. By encoding mathematical principles, AI models can be constrained, while still being allowed to learn a solution from the data organically. This approach can lead to reduced training time, requiring less computational resources, Kvinge said. For example, developers often talk about AI training in the context of a loss landscape with mountains and valleys corresponding to weak and strong models, respectively. During training the goal is to get the model to settle in a relatively shallow, broad valley corresponding to a robust model. In this context, math concepts like symmetry and curvature can be used to develop training routines that nudge the model toward more favorable solutions, he added. AI for mathematics: Just as scientists have begun to explore how AI can contribute to faster scientific discoveries, mathematicians are realizing that AI can be trained to tackle complicated and unsolved problems in math, Kvinge said. Recently, Kvinge and his colleagues showed that an AI algorithm could provide the insights necessary to rediscover a challenging result in algebraic combinatorics. The researchers set up their problem so that they were able to crack open the black box and peer inside to see how the algorithm landed on its results. This type of study is still relatively rare since reverse engineering the solution found by a deep learning-based system is laborious and requires special expertise. As Kvinge notes, many traditional mathematicians remain skeptical about the efficacy of AI in advancing pure mathematics. However, this result showed the successful application of a narrow AI model to a specific mathematical problem. Results like this one should gradually warm the broader community to the potential of AI-driven research, he added. The research outcome was part of a larger 4-year investment by PNNL to explore the role of Mathematics for Artificial Reasoning in Science (MARS). The success of this initial work led to new questions and challenges that Kvinge and his team will tackle under PNNL's new investment called Generative AI: Foundations for the Future and through his collaborative work exploring topology, algebra, and geometry to address fundamental data science challenges. Along that front, Kvinge encouraged his math colleagues to explore the array of AI-friendly unsolved math problems that the MARS team collaborated on to produce for the math community. The curated set of ML Benchmarks in Algebraic Combinatorics problems, along with published solutions, is available on GitHub. Collaboration between mathematicians and data engineers Kvinge's experience has been that in AI development, many of the most important advances are being driven by engineering. Mathematicians, traditionally inclined toward working from first principles, can best collaborate by being open to applying math to problems and observations made on real-world systems by the machine learning community, he said. This dynamic keeps the contributions of math grounded in practical problems. By immersing themselves in data and analyzing emergent behaviors within AI models, mathematicians can propose more relevant solutions. Mathematics offers a framework for making design decisions, while leaving the execution to the learning algorithm. "There's this famous article in machine learning called 'The Bitter Lesson,' which basically says that as much as possible, you should try to remove human choices from the learning process and instead rely on data and compute," he said. "Through its ability to frame concepts at a very abstract level, mathematics can help us impose minimal constraints on a system, leaving the rest for the learning algorithm." The problem is that, he added, "These models will sometimes pick up on some spurious correlations that are related to how you collected or preprocessed the data. It's like a student finding a way to ace the test without actually learning the material. For the time being, it's important to have a human in the loop to catch this type of thing." But breaking into neural networks to do that kind of detective work requires math. And that's going to keep Kvinge and his colleagues busy for the foreseeable future.

How Math Reveals the Bleeding Edge of AI

Newswise -- Mathematicians love the certainty of proofs. This is how they verify that their intuition matches observable truth. This logical love of proofs is uniquely suited to opening up the black box of artificial intelligence (AI) to scrutiny with the goal of establishing trust. Mathematics and AI may seem to be natural partners, but AI is only just beginning to receive substantial attention from the mathematics community. At Pacific Northwest National Laboratory (PNNL), mathematicians are exploring the limits of what AI can achieve while assuring we can believe what it says to us. Henry Kvinge, a mathematician at PNNL who is deeply involved in AI research, breaks down the relationship into three buckets that each play a vital role in harnessing the potential of AI systems. Math to understand AI. Math offers powerful tools for deciphering the inner workings of complex AI models, he says. For instance, researchers strive to decode what happens within neural networks -- the black boxes of AI -- to ensure that the decisions they make do not rely on misguided reasoning or faulty logic. Mathematics helps dissect these complex systems, identifying underlying failures and optimizing performance. Recently, Kvinge wrote an article that summarizes his thoughts on how math concepts like curvature, shape, and symmetry provide toolkits for analyzing the accuracy of a given AI model for an assigned task. Math to inspire AI development. Mathematics provides the language to describe the high-level design principles of AI systems. By encoding mathematical principles, AI models can be constrained, while still being allowed to learn a solution from the data organically. This approach can lead to reduced training time, requiring less computational resources, Kvinge said. For example, developers often talk about AI training in the context of a loss landscape with mountains and valleys corresponding to weak and strong models, respectively. During training the goal is to get the model to settle in a relatively shallow, broad valley corresponding to a robust model. In this context, math concepts like symmetry and curvature can be used to develop training routines that nudge the model toward more favorable solutions, he added. AI for mathematics. Just as scientists have begun to explore how AI can contribute to faster scientific discoveries, mathematicians are realizing that AI can be trained to tackle complicated and unsolved problems in math, Kvinge said. Recently, Kvinge and his colleagues showed that an AI algorithm could provide the insights necessary to rediscover a challenging result in algebraic combinatorics. The researchers set up their problem so that they were able to crack open the black box and peer inside to see how the algorithm landed on its results. This type of study is still relatively rare since reverse engineering the solution found by a deep learning-based system is laborious and requires special expertise. As Kvinge notes, many traditional mathematicians remain skeptical about the efficacy of AI in advancing pure mathematics. However, this result showed the successful application of a narrow AI model to a specific mathematical problem. Results like this one should gradually warm the broader community to the potential of AI-driven research, he added. The research outcome was part of a larger 4-year investment by PNNL to explore the role of Mathematics for Artificial Reasoning in Science (MARS). The success of this initial work led to new questions and challenges that Kvinge and his team will tackle under PNNL's new investment called Generative AI: Foundations for the Future and through his collaborative work exploring topology, algebra, and geometry to address fundamental data science challenges. Along that front, Kvinge encouraged his math colleagues to explore the array of AI-friendly unsolved math problems that the MARS team collaborated on to produce for the math community. The curated set of ML Benchmarks in Algebraic Combinatorics problems, along with published solutions, is available on GitHub. Kvinge's experience has been that in AI development, many of the most important advances are being driven by engineering. Mathematicians, traditionally inclined toward working from first principles, can best collaborate by being open to applying math to problems and observations made on real-world systems by the machine learning community, he said. This dynamic keeps the contributions of math grounded in practical problems. By immersing themselves in data and analyzing emergent behaviors within AI models, mathematicians can propose more relevant solutions. Mathematics offers a framework for making design decisions, while leaving the execution to the learning algorithm. "There's this famous article in machine learning called 'The Bitter Lesson,' which basically says that as much as possible, you should try to remove human choices from the learning process and instead rely on data and compute," he said. "Through its ability to frame concepts at a very abstract level, mathematics can help us impose minimal constraints on a system, leaving the rest for the learning algorithm." The problem is that, he added, "These models will sometimes pick up on some spurious correlations that are related to how you collected or preprocessed the data. It's like a student finding a way to ace the test without actually learning the material. For the time being, it's important to have a human in the loop to catch this type of thing." But breaking into neural networks to do that kind of detective work requires math. And that's going to keep Kvinge and his colleagues busy for the foreseeable future. Pacific Northwest National Laboratory draws on its distinguishing strengths in chemistry, Earth sciences, biology and data science to advance scientific knowledge and address challenges in energy resiliency and national security. Founded in 1965, PNNL is operated by Battelle and supported by the Office of Science of the U.S. Department of Energy. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit the DOE Office of Science website. For more information on PNNL, visit PNNL's News Center. Follow us on Twitter, Facebook, LinkedIn and Instagram.