IBM and ESA Launch TerraMind: A Groundbreaking AI Model for Earth Observation

IBM and ESA launch AI model with 'intuitive' understanding of Earth

IBM and the European Space Agency (ESA) today launched TerraMind, a new open-source AI model with an "intuitive" understanding of Earth. According to the research team, the system is the best-performing AI model for Earth observation. In an ESA-led evaluation, TerraMind beat 12 leading AI models on the PANGAEA benchmark -- a community standard for Earth observation. The model excelled at various real-world tasks, including land cover classification, change detection, and multi-sensor analysis. On average, it outperformed other models by 8% or more. "To me, what sets TerraMind apart is its ability to go beyond simply processing earth observations with computer vision algorithms," said Juan Bernabé-Moreno, director of IBM Research UK and Ireland. "It instead has an intuitive understanding of geospatial data and our planet." TerraMind is a generative AI model that can understand different types of data -- such as images, text, and time-based sequences (like climate patterns) -- and spot connections between these different kinds of information. That's particularly useful when dealing with an immensely complex system like Earth. The model was trained on 9 million samples drawn from nine different data types, including satellite images, climate records, terrain features, and vegetation maps. The broad dataset covered every region and biome on Earth. It was designed to minimise bias and ensure the model can be used reliably across the globe, the researchers said. TerraMind is built on Prithvi, an open-source family of foundational climate models launched by IBM and NASA in 2023. The Prithvi models require relatively less computational power than traditional climate modelling software, making them potentially more environmentally friendly. A standout feature of TerraMind is its "Thinking-in-Modalities" (TiM) tuning. Similar to chain-of-thought reasoning in language models, TiM lets TerraMind self-generate extra data to improve its performance. "TiM tuning boosts data efficiency by self-generating the additional training data relevant to the problem being addressed -- for example, by telling the model to 'think' about land cover when mapping water bodies," said Johannes Jakubik, an IBM research scientist based in Zurich. TerraMind was built in collaboration with Polish spacetech firm KP Labs, the Jülich Supercomputing Centre in Germany, and the German Space Agency (DLR). The model is now available open-source on Hugging Face. Fine-tuned versions will be released in the coming months. ESA, NASA, and IBM are by no means the only organisations experimenting with AI models for climate forecasting. Another example emerged from Google DeepMind, which recently unveiled an AI weather forecaster that makes faster and more accurate predictions than the best system available today.

IBM's open-source TerraMind AI uses 9 data modalities to transform Earth observation - SiliconANGLE

IBM's open-source TerraMind AI uses 9 data modalities to transform Earth observation IBM Corp. is partnering with the European Space Agency to create an artificial intelligence system that will keep tabs on climate change and other global issues, such as water scarcity, in real time using space-based data. IBM says the new system, called TerraMind, is the most advanced "Earth observation" model ever conceived, and it's being made available to researchers through the Hugging Face platform today. The open-source model was trained on TerraMesh, which is the largest set of freely available geospatial data, created by researchers as part of the TerraMind project. IBM says TerraMind is a multimodal model based on a novel "symmetric transformer-based encoder-decoder architecture." It can handle pixel-based, token-based and sequence-based inputs and learn correlations across modalities. Although it was trained on more than 500 billion tokens of data, it's a relatively small and lightweight model that uses about 10 times fewer computing resources than existing models for each modality it covers. As a result, it's possible to scale TerraMind at a much lower cost, IBM says. TerraMind's multimodal capabilities are essential for Earth observation tasks, IBM says, because there are so many factors that need to be considered when weighing up the impact of climate change. For instance, when it comes to predicting the risk of water scarcity, it's necessary to take into account factors such as climate, temperature, rainfall, land use, vegetation and agricultural activities. Previously, researchers would have to use multiple models to study each of these factors separately. TerraMind brings all of this information together in a single model to create a more informed picture of what's happening in a given location, allowing it to predict water scarcity risks with unprecedented accuracy "What sets TerraMind apart is its ability to go beyond simply processing Earth observations with computer vision algorithms," said IBM Research UK and Ireland Director Juan Bernabé-Moreno. To build the TerraMesh dataset that underpins TerraMind, IBM's researchers compiled data on everything from biomes to land use, land cover types and regions, to ensure that the model can be used to make predictions in any part of the planet, with minimal bias. TerraMesh contains more than nine million globally distributed and spatiotemporally aligned data samples across nine core modalities. These include observations from the ESA's satellites, information on surface characteristics, such as vegetation and land use, and geometry used to describe locations and features, such as latitude, longitude, altitude and simple text descriptions of different regions. IBM says tests demonstrate that TerraMind is by far and away the best-performing foundational model for Earth observation ever made. The ESA led these evaluations, pitting TerraMind against 12 other models in its class on the PANGEA benchmark, which is used to measure AI performance on real-world tasks such as land cover classification, change detection, environmental monitoring and multi-temporal analysis. According to the ESA, TerraMind outperformed competing models by 8% or more on every task within the PANGEA benchmark tests. ESA Earth Observation Programmes Director Simonetta Cheli said TerraMind's advantage stems from the insights provided by so many different modalities. "The ability to intuitively bring in contextual information and generate unseen scenarios is a critical step in unlocking the value of ESA data," she said. "Compared to competitive models, it can uncover a deeper understanding of the Earth for researchers and businesses alike." Going further, TerraMind also employs a new technique called "Thinking-in-Modalties" tuning, which is similar to the "Chain-of-Thought" method used in reasoning-focused large language models such as OpenAI's GPT-4o and DeepSeek Ltd.'s R1. TiM, as it's known, involves self-generating additional synthetic training data from other modalities to enhance TerraMind's performance beyond what regular fine-tuning could achieve. IBM Research scientist Johannes Jakubik, who helped develop the TiM technique, said it works by generating synthetic data related to the problem the model is trying to address. So it might be programmed to generate additional detailed maps when asked to "think" about land cover when mapping bodies of water. "This breakthrough can unlock unprecedented accuracy when specializing TerraMind for particular use cases," Jakubik said. IBM said TerraMind can be applied to solve long-term issues such as natural disaster management, environment monitoring, high-precision agriculture, urban and regional planning, critical infrastructure monitoring, forestry and biodiversity monitoring, as well as short-term problems like wildfire and flood monitoring. TerraMind is yet another example of how AI is impacting almost every aspect of technology, said Holger Mueller of Constellation Research Inc. "AI models that can make sense of our planet will be extremely useful when it comes to predicting, simulating and explaining natural and man-made events," the analyst said. "IBM is advancing this with its new Thinking-in-Modalities approach, with nine core modalities based on more than nine million data points. It's by far and away the most complete AI model of our planet, and it's great that it's being made open source." IBM has already published TerraMind on Hugging Face and the IBM Geospatial Studio, and it is planning to release fine-tuned version for disaster response and other high-impact use cases in the next few weeks. "We are unlocking the full potential of space-based data to protect our planet," said ESA Earth Observation Data Scientist Nicolas Longepe. "This project is a perfect example where the scientific community, big tech companies, and experts have collaborated to leverage this technology for the benefit of Earth sciences. The magic happens when earth observation data experts, machine learning experts, data scientists, and HPC engineers come together."