Explainable AI: A Game-Changer for Geosciences and Natural Hazard Management

Experts underscore the value of explainable AI in geosciences

by Timon Meyer, Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI In a new paper published in Nature Geoscience, experts from Fraunhofer Heinrich-Hertz-Institut (HHI) advocate for the use of explainable artificial intelligence (XAI) methods in geoscience. The researchers aim to facilitate the broader adoption of AI in geoscience (e.g., in weather forecasting) by revealing the decision processes of AI models and fostering trust in their results. Fraunhofer HHI, a world-leader in XAI research, coordinates a UN-backed global initiative that is laying the groundwork for international standards in the use of AI for disaster management. AI offers unparalleled opportunities for analyzing data and solving complex and nonlinear problems in geoscience. However, as the complexity of an AI model increases, its interpretability may decrease. In safety-critical situations, such as disasters, the lack of understanding of how a model works -- and the resulting lack of trust in its results -- can hinder its implementation. XAI methods address this challenge by providing insights into AI systems, identifying data- or model-related issues. For instance, XAI can detect "false" correlations in training data -- correlations irrelevant to the AI system's specific task that may distort results. "Trust is crucial to the adoption of AI. XAI acts as a magnifying lens, enabling researchers, policymakers, and security specialists to analyze data through the 'eyes' of the model so that dominant prediction strategies -- and any undesired behaviors -- can be understood," explains Prof. Wojciech Samek, Head of Artificial Intelligence at Fraunhofer HHI. The paper's authors analyzed 2.3 million arXiv abstracts of geoscience-related articles published between 2007 and 2022. They found that only 6.1% of papers referenced XAI. Considering its immense potential, the authors sought to identify challenges preventing geoscientists from adopting XAI methods. Focusing on natural hazards, the authors examined use cases curated by the International Telecommunication Union/World Meteorological Organization/UN Environment Focus Group on AI for Natural Disaster Management. After surveying researchers involved in these use cases, the authors identified key motivations and hurdles. Motivations included building trust in AI applications, gaining insights from data, and improving AI systems' efficiency. Most participants also used XAI to analyze their models' underlying processes. Conversely, those not using XAI cited the effort, time, and resources required as barriers. "XAI has a clear added value for the geosciences -- improving underlying datasets and AI models, identifying physical relationships that are captured by data, and building trust among end users -- I hope that once geoscientists understand this value, it will become part of their AI pipeline," says Dr. Monique Kuglitsch, Innovation Manager at Fraunhofer HHI and Chair of the Global Initiative on Resilience to Natural Hazards Through AI Solutions. To support XAI adoption in geoscience, the paper provides four actionable recommendations: In addition to Fraunhofer HHI experts Monique Kuglitsch, Ximeng Cheng, Jackie Ma, and Wojciech Samek, the paper was authored by Jesper Dramsch, Miguel-Ángel Fernández-Torres, Andrea Toreti, Rustem Arif Albayrak, Lorenzo Nava, Saman Ghaffarian, Rudy Venguswamy, Anirudh Koul, Raghavan Muthuregunathan, and Arthur Hrast Essenfelder.

Explainability can foster trust in artificial intelligence in geoscience - Nature Geoscience

Artificial intelligence (AI) offers unparalleled opportunities for analysing multidimensional data and solving complex and nonlinear problems in geoscience1,2,3. However, as the complexity and potentially the predictive skill of an AI model increases, its interpretability -- the ability to understand the model and its predictions from a physical perspective -- may decrease3,4. In critical situations, such as scenarios caused by natural hazards, the resulting lack of understanding of how a model works and consequent lack of trust in its results can become a barrier to its implementation5. Here we argue that explainable AI (XAI) methods, which enhance the human-comprehensible understanding and interpretation of opaque 'black-box' AI models, can build trust in AI model results and encourage greater adoption of AI methods in geoscience6. Trust is crucial to the adoption of AI. Thus some researchers advocate for inherently interpretable AI models; in other words, models that provide their own explanations. Others, however, prefer to retain the predictive capabilities of deep neural networks -- models able to capture highly complex and nonlinear patterns in data but with limited interpretability -- and to circumvent black-box issues through XAI methods, which provide "an explanation to the user that justifies its recommendation, decision, or action". These methods can provide insight into an AI system, identifying issues related to data or the model. For example, XAI can detect spurious correlations in training data and otherwise imperceptible perturbations to remote sensing images. In this sense, XAI can be regarded as a magnifying lens, enabling the human expert to analyse data through the 'eyes' of the model so that the dominant prediction strategies -- and any undesired behaviours -- can be understood. Another benefit of XAI is that it can highlight linkages between input variables and model predictions, which may motivate further research and support an enhanced understanding of features as well as spatiotemporal processes. For example, researchers have used XAI on an inventory of landslide data to understand why AI models classify slopes as susceptible (or not) to failure and to gain insight into failure mechanisms. XAI has also been applied to time series of a meteorological drought index to determine the importance of climatic variables such as precipitation for meteorological drought prediction. In the latter example, the results aligned with physical model interpretations, emphasizing the need to include specific climatic variables as predictors in the model. Figure 1 demonstrates the possible benefits of XAI across different dimensions, using natural hazards as an example domain.