AI's Double-Edged Sword: Revolutionizing Scientific Research While Raising Ethical Concerns

AI could usher in a golden age of research, but only if it isn't restricted to a few major private companies

But the evolving role of AI in scientific discovery also raises questions and concerns. Will a lack of access to increasingly capable AI tools restrict the ability of many institutions to carry out research at the cutting edge? The physics and chemistry Nobels were actually awarded for radically different advances. The physics prize, which went to John Hopfield and Geoffrey Hinton, recognized their development of algorithms and ideas that advanced a subset of AI called machine learning. This is where algorithms get better at what they do by analyzing large amounts of data (a process called training), then applying these lessons to other unseen data. The chemistry prize was awarded to the Google DeepMind team for an impressive scientific breakthrough by an AI system called AlphaFold. This tool is trained to predict the structures of proteins and how they fold -- a scientific challenge that had remained unsolved for half a century. As such, the Nobel prize would have been granted to any team that solved this, regardless of the methods used. It was not a prize for a development in AI; it was a prize for an important discovery carried out by an AI system. Nonetheless, we are moving in a novel direction. AI in science is transitioning from being solely the object of investigation, to becoming the mechanism of investigation. Reaching human performance The transformation of AI's role in academic research began well before 2024, and even before the advent of ChatGPT and the accompanying marketing hype around AI. It began when these systems first achieved human-level performance in crucial tasks related to scientific research. In 2015, Microsoft's ResNet surpassed human performance on ImageNet, a test that evaluates the ability of AI systems to carry out image classification and other graphics-related tasks. In 2019, Facebook's RoBERTa (an evolution of Google's BERT) exceeded human ability on the GLUE test, mastering tasks like text classification and summarization. These milestones -- achieved by large private research labs -- enabled researchers to leverage AI for a wide range of different tasks, such as using satellite images to analyze levels of poverty and using medical images to detect cancer. Automating tasks traditionally done by humans reduces costs and expands the scope of research -- in part by enabling the execution of inherently subjective tasks to become more objective. AI in science today goes beyond data collection and processing -- it plays a growing role in understanding the data. In chemistry and physics, for example, AI is extensively used for forecasting complex systems, such as weather patterns or protein structures. In social and medical sciences, however, understanding often hinges on causality, not just prediction. For example, to assess the impact of a policy, researchers need to estimate how things would have unfolded without it -- a counterfactual path that can never be directly observed. Medical science tackles this through randomized trials. These are studies in which the participants are divided by chance into separate groups to compare the effects of different treatments. And this is an approach increasingly adopted in social sciences too, as evidenced by the 2019 economics Nobel awarded to Abhijit Banerjee, Esther Duflo and Michael Kremer for their work on poverty reduction. However, in macroeconomics, such experiments are impractical -- no country would adopt random trade strategies for the sake of research. Enter AI, which has transformed the study of large economic systems. Computer-based tools can produce models to explain how aspects of the economy work that are far more nuanced than those humans can put together. Susan Athey and colleagues' work on the impact of computer science and advanced statistics on economic research was a popular favorite for the 2024 Nobel prize in economics, although it didn't win. The key role for humans While AI excels at collecting and analyzing data, humans still hold the key role: understanding how this data connects to reality. For example, a large language model (the technology behind AI chatbots like ChatGPT) can write a sentence such as "that saxophone can't fit in the brown bag because it's too big." And it can identify whether "it" refers to the saxophone or the bag -- an impressive feat compared with what was possible just a decade ago. But the AI doesn't relate this to any understanding of 3D objects. It operates like a brain in a vat, confined to its feedback loop of solving text-based tasks without engaging with the physical world. Unlike AI, humans are shaped by diverse needs: navigating a 3D world, socializing, avoiding conflict, fighting when necessary, and building safe, equitable societies. AI systems, by contrast, are single-task specialists. Large language models are trained solely to generate coherent text, with no connection to broader reality or practical goals. The leap to true understanding will come only when a single AI system can pursue multiple, general goals simultaneously, integrating tasks and linking words to real-world solutions. Perhaps then, we'll see the first Nobel prize graciously accepted by an AI system. Predicting exactly when or how this shift will unfold is impossible, but its implications are too significant to ignore. The rise of AI-driven research could usher in a golden age of scientific breakthroughs, or a deeply divided future where many labs (in particular public labs, especially in the global south) lack the advanced AI tools to carry out cutting-edge research. Names like Google, Microsoft, Facebook, OpenAI and Tesla are now at the forefront of basic research -- a major departure from the days when public and academic institutions led the charge. This new reality raises pressing questions. Can we fully trust AI developed by private companies to shape scientific research? It also raises questions about how we address the risks of concentrated power, threats to open science (making research freely accessible), and the uneven distribution of scientific rewards between countries and communities. If we are to celebrate the first AI to win a Nobel prize for its own discovery, we must ensure the conditions are in place not to see it as the triumph of some humans over others, but as a victory for humanity as a whole.

AI could usher in a golden age of research - but only if these cutting-edge tools aren't restricted to a few major private companies

United Nations University provides funding as a member of The Conversation UK. 2024 has been called the year of AI in science. It saw the Nobel prizes in both physics and chemistry awarded to groups of AI researchers. But the evolving role of AI in scientific discovery also raises questions and concerns. Will a lack of access to increasingly capable AI tools restrict the ability of many institutions to carry out research at the cutting edge? The physics and chemistry Nobels were actually awarded for radically different advances. The physics prize, which went to John Hopfield and Geoffrey Hinton, recognised their development of algorithms and ideas that advanced a subset of AI called machine learning. This is where algorithms get better at what they do by analysing large amounts of data (a process called training), then applying these lessons to other unseen data. The chemistry prize was awarded to the Google DeepMind team for an impressive scientific breakthrough by an AI system called AlphaFold. This tool is trained to predict the structures of proteins and how they fold - a scientific challenge that had remained unsolved for half a century. As such, the Nobel prize would have been granted to any team that solved this, regardless of the methods used. It was not a prize for a development in AI; it was a prize for an important discovery carried out by an AI system. Nonetheless, we are moving in a novel direction. AI in science is transitioning from being solely the object of investigation, to becoming the mechanism of investigation. Reaching human performance The transformation of AI's role in academic research began well before 2024, and even before the advent of ChatGPT and the accompanying marketing hype around AI. It began when these systems first achieved human-level performance in crucial tasks related to scientific research. In 2015, Microsoft's ResNet surpassed human performance on ImageNet, a test that evaluates the ability of AI systems to carry out image classification and other graphics-related tasks. In 2019, Facebook's RoBERTa (an evolution of Google's BERT) exceeded human ability on the GLUE test, mastering tasks like text classification and summarisation. These milestones - achieved by large private research labs - enabled researchers to leverage AI for a wide range of different tasks, such as using satellite images to analyse levels of poverty and using medical images to detect cancer. Automating tasks traditionally done by humans reduces costs and expands the scope of research - in part by enabling the execution of inherently subjective tasks to become more objective. AI in science today goes beyond data collection and processing - it plays a growing role in understanding the data. In chemistry and physics, for example, AI is extensively used for forecasting complex systems, such as weather patterns or protein structures. In social and medical sciences, however, understanding often hinges on causality, not just prediction. For example, to assess the impact of a policy, researchers need to estimate how things would have unfolded without it - a counterfactual path that can never be directly observed. Medical science tackles this through randomised trials. These are studies in which the participants are divided by chance into separate groups to compare the effects of different treatments. And this is an approach increasingly adopted in social sciences too, as evidenced by the 2019 economics Nobel awarded to Abhijit Banerjee, Esther Duflo and Michael Kremer for their work on poverty reduction. However, in macroeconomics, such experiments are impractical - no country would adopt random trade strategies for the sake of research. Enter AI, which has transformed the study of large economic systems. Computer-based tools can produce models to explain how aspects of the economy work that are far more nuanced than those humans can put together. Susan Athey and colleagues' work on the impact of computer science and advanced statistics on economic research was a popular favourite for the 2024 Nobel prize in economics, although it didn't win. The key role for humans While AI excels at collecting and analysing data, humans still hold the key role: understanding how this data connects to reality. For example, a large language model (the technology behind AI chatbots like ChatGPT) can write a sentence such as "that saxophone can't fit in the brown bag because it's too big". And it can identify whether "it" refers to the saxophone or the bag - an impressive feat compared with what was possible just a decade ago. But the AI doesn't relate this to any understanding of 3D objects. It operates like a brain in a vat, confined to its feedback loop of solving text-based tasks without engaging with the physical world. Unlike AI, humans are shaped by diverse needs: navigating a 3D world, socialising, avoiding conflict, fighting when necessary, and building safe, equitable societies. AI systems, by contrast, are single-task specialists. Large language models are trained solely to generate coherent text, with no connection to broader reality or practical goals. The leap to true understanding will come only when a single AI system can pursue multiple, general goals simultaneously, integrating tasks and linking words to real-world solutions. Perhaps then, we'll see the first Nobel prize graciously accepted by an AI system. Predicting exactly when or how this shift will unfold is impossible, but its implications are too significant to ignore. The rise of AI-driven research could usher in a golden age of scientific breakthroughs, or a deeply divided future where many labs (in particular public labs, especially in the global south) lack the advanced AI tools to carry out cutting-edge research. Names like Google, Microsoft, Facebook, OpenAI and Tesla are now at the forefront of basic research - a major departure from the days when public and academic institutions led the charge. This new reality raises pressing questions. Can we fully trust AI developed by private companies to shape scientific research? It also raises questions about how we address the risks of concentrated power, threats to open science (making research freely accessible), and the uneven distribution of scientific rewards between countries and communities. If we are to celebrate the first AI to win a Nobel prize for its own discovery, we must ensure the conditions are in place not to see it as the triumph of some humans over others, but as a victory for humanity as a whole.

AI could crack unsolvable problems -- and humans won't be able to understand the results

AI promises to accelerate scientific discovery, but if scientists aren't careful public trust may be left behind. Artificial intelligence (AI) has taken centre stage in basic science. The five winners of the 2024 Nobel Prizes in Chemistry and Physics shared a common thread: AI. Indeed, many scientists -- including the Nobel committees -- are celebrating AI as a force for transforming science. As one of the laureates put it, AI's potential for accelerating scientific discovery makes it "one of the most transformative technologies in human history". But what will this transformation really mean for science? AI promises to help scientists do more, faster, with less money. But it brings a host of new concerns, too -- and if scientists rush ahead with AI adoption they risk transforming science into something that escapes public understanding and trust, and fails to meet the needs of society. The illusions of understanding Experts have already identified at least three illusions that can ensnare researchers using AI. The first is the "illusion of explanatory depth". Just because an AI model excels at predicting a phenomenon -- like AlphaFold, which won the Nobel Prize in Chemistry for its predictions of protein structures -- that doesn't mean it can accurately explain it. Research in neuroscience has already shown that AI models designed for optimised prediction can lead to misleading conclusions about the underlying neurobiological mechanisms. Second is the "illusion of exploratory breadth". Scientists might think they are investigating all testable hypotheses in their exploratory research, when in fact they are only looking at a limited set of hypotheses that can be tested using AI. Sign up for the Live Science daily newsletter now Get the world's most fascinating discoveries delivered straight to your inbox. Contact me with news and offers from other Future brandsReceive email from us on behalf of our trusted partners or sponsorsBy submitting your information you agree to the Terms & Conditions and Privacy Policy and are aged 16 or over. Finally, the "illusion of objectivity". Scientists may believe AI models are free from bias, or that they can account for all possible human biases. In reality, however, all AI models inevitably reflect the biases present in their training data and the intentions of their developers. Related: 'It might pave the way for novel forms of artistic expression': Generative AI isn't a threat to artists -- it's an opportunity to redefine art itself Cheaper and faster science One of the main reasons for AI's increasing appeal in science is its potential to produce more results, faster, and at a much lower cost. An extreme example of this push is the "AI Scientist" machine recently developed by Sakana AI Labs. The company's vision is to develop a "fully AI-driven system for automated scientific discovery", where each idea can be turned into a full research paper for just US$15 -- though critics said the system produced "endless scientific slop". Do we really want a future where research papers can be produced with just a few clicks, simply to "accelerate" the production of science? This risks inundating the scientific ecosystem with papers with no meaning and value, further straining an already overburdened peer-review system. We might find ourselves in a world where science, as we once knew it, is buried under the noise of AI-generated content. A lack of context The rise of AI in science comes at a time when public trust in science and scientists is still fairly high , but we can't take it for granted. Trust is complex and fragile. As we learned during the COVID pandemic, calls to "trust the science" can fall short because scientific evidence and computational models are often contested, incomplete, or open to various interpretations. However, the world faces any number of problems, such as climate change, biodiversity loss, and social inequality, that require public policies crafted with expert judgement. This judgement must also be sensitive to specific situations, gathering input from various disciplines and lived experiences that must be interpreted through the lens of local culture and values. As an International Science Council report published last year argued, science must recognise nuance and context to rebuild public trust. Letting AI shape the future of science may undermine hard-won progress in this area. If we allow AI to take the lead in scientific inquiry, we risk creating a monoculture of knowledge that prioritizes the kinds of questions, methods, perspectives and experts best suited for AI. This can move us away from the transdisciplinary approach essential for responsible AI, as well as the nuanced public reasoning and dialogue needed to tackle our social and environmental challenges. A new social contract for science As the 21st century began, some argued scientists had a renewed social contract in which scientists focus their talents on the most pressing issues of our time in exchange for public funding. The goal is to help society move toward a more sustainable biosphere -- one that is ecologically sound, economically viable and socially just. The rise of AI presents scientists with an opportunity not just to fulfil their responsibilities but to revitalize the contract itself. However, scientific communities will need to address some important questions about the use of AI first. For example, is using AI in science a kind of "outsourcing" that could compromise the integrity of publicly funded work? How should this be handled? What about the growing environmental footprint of AI? And how can researchers remain aligned with society's expectations while integrating AI into the research pipeline? The idea of transforming science with AI without first establishing this social contract risks putting the cart before the horse. RELATED STORIES -- Future AI models could be turbocharged by brand new system of logic that researchers call 'inferentialism' -- Mathematicians devised novel problems to challenge advanced AIs' reasoning skills -- and they failed almost every test -- Meet Evo, an AI model that can predict the effects of gene mutations with 'unparalleled accuracy' Letting AI shape our research priorities without input from diverse voices and disciplines can lead to a mismatch with what society actually needs and result in poorly allocated resources. Science should benefit society as a whole. Scientists need to engage in real conversations about the future of AI within their community of practice and with research stakeholders. These discussions should address the dimensions of this renewed social contract, reflecting shared goals and values. It's time to actively explore the various futures that AI for science enables or blocks -- and establish the necessary standards and guidelines to harness its potential responsibly. This edited article is republished from The Conversation under a Creative Commons license. Read the original article.