Microsoft's Aurora AI: Revolutionizing Weather Forecasting and Environmental Predictions

Microsoft says its Aurora AI can accurately predict air quality, typhoons, and more | TechCrunch

One of Microsoft's latest AI models can accurately predict air quality, hurricanes, typhoons, and other weather-related phenomena, the company claims. In a paper published in the journal Nature and an accompanying blog post this week, Microsoft detailed Aurora, which the tech giant says can forecast atmospheric events with greater precision and speed than traditional meteorological approaches. Aurora, which has been trained on more than a million hours of data from satellites, radar and weather stations, simulations, and forecasts, can be fine-tuned with additional data to make predictions for particular weather events. AI weather models are nothing new. Google DeepMind has released a handful over the past several years, including WeatherNext, which the lab claims beats some of the world's best forecasting systems. Microsoft is positioning Aurora as one of the field's top performers -- and a potential boon for labs studying weather science. In experiments, Aurora predicted Typhoon Doksuri's landfall in the Philippines four days in advance of the actual event, beating some expert predictions, Microsoft says. The model also bested the National Hurricane Center in forecasting five-day tropical cyclone tracks for the 2022-2023 season, and successfully predicted the 2022 Iraq sandstorm. While Aurora required substantial computing infrastructure to train, Microsoft says the model is highly efficient to run. It generates forecasts in seconds compared to the hours traditional systems take using supercomputer hardware. Microsoft, which has made the source code and model weights publicly available, says that it's incorporating Aurora's AI modeling into its MSN Weather app via a specialized version of the model that produces hourly forecasts, including for clouds.

30-day forecast? Weather prediction might be able to look beyond 2 weeks

It's a truism almost as old as modern weather prediction: Any forecast beyond 2 weeks will fall apart because of the way tiny perturbations compound in the atmosphere. The 2-week limit, grounded in chaos theory and notions of the "butterfly effect" from the 1960s, has been handed down from generation to generation, says Peter Dueben, head of earth system modeling at the European Centre for Medium-Range Weather Forecasts, the world's leading forecaster. "It's basically a God-given rule." But even the gods can be wrong. Using an artificial intelligence (AI) weather model developed by Google, atmospheric scientists have found that forecasts of 1 month or more into the future might be possible. "We haven't found a limit to how far you can go out," says Trent Vonich, a doctoral student at the University of Washington (UW) who led the work, released late last month as a preprint on arXiv. "We ran out of memory first." The result has caused a stir ever since Vonich and Gregory Hakim, his adviser, spoke this year at the annual meeting of the American Meteorological Society, says Amy McGovern, a computer scientist and meteorologist at the University of Oklahoma. Using powerful computer models, researchers have already pushed meaningful forecasts out to about 10 days, coming ever closer to the 2-week limit. Showing this limit can in principle be broken "means that AI will be able to do this someday, which is really exciting," she says. The paper has caveats. For one thing, it does not make actual forecasts beyond 2 weeks, points out Tobias Selz, an atmospheric scientist at the Ludwig Maximilian University of Munich. So far, he says, the UW researchers have only tested their AI forecasts with reconstructed snapshots of past weather. Moreover, as Selz and colleagues demonstrated in a 2023 study in Geophysical Research Letters, the AI models ignore the small-scale atmospheric processes -- effects as small as a butterfly flapping -- that are thought to snowball and drive the predictability limit. "I'm really reluctant to use these models to make statements about atmospheric predictability." The notion of an intrinsic forecasting limit goes back to Edward Lorenz, the famed mathematician and meteorologist at the Massachusetts Institute of Technology (MIT). In a 1963 paper, he pointed out that even a small difference in rendering the initial state of the atmosphere or a similarly chaotic system would ultimately cause forecasts to diverge wildly. Then, in a 1969 paper, he suggested that, even if these initial conditions were known almost perfectly, the system would still have a predictability limit driven by the rapid error growth at small scales. Lorenz did not, however, actually specify a 2-week limit. According to a recent historical study led by Bo-wen Shen, a mathematician at San Diego State University, Lorenz put forward a variety of possible limits but never settled on one. The 2-week figure came instead from MIT's Jule Charney and other pioneers who were gauging the capabilities of the world's first numerical weather models at about the same time. Shen also notes that Lorenz's 1969 modeling exercise relied on equations that were highly sensitive to their input data, which has caused Shen to wonder whether the butterfly effect is an artifact. Either way, there's no reason to think the 2-week limit is a rule, he says. "It's not a physically based law. It's an empirical assumption." In their new work, Vonich and Hakim relied on Google's GraphCast, an AI model trained on 40 years of "reanalysis data" -- high-resolution snapshots of the planet's weather based on observations and short-term model forecasts. The duo wanted to see how well GraphCast would work if they could somehow radically boost the accuracy of the initial conditions, the starting snapshot. They did this by comparing the final state of the atmosphere from reanalysis data with GraphCast's forecasts. Shortcomings in a forecast could then be used to adjust the initial conditions of the reanalysis data the model used to start its forecast, potentially bringing them closer to the atmosphere's true state. Operational weather models may also be tuned backward in this way, as subsequent observations are amassed. But the calculations needed to look back more than 12 hours in time quickly grow overwhelming. The structure of GraphCast, by contrast, makes such analyses easy to run thousands of times over and further back in time, allowing the model to home in on a near-perfect starting snapshot for the atmosphere, Hakim says. "Basically they were handing this to us on a silver platter." With the trained initial conditions, GraphCast's accuracy for its 10-day forecast improved by 86% on average -- "absolutely massive" in weather terms, Vonich says. Even more surprising, the model showed skill at predicting weather more than 33 days in the future. It was hard for Hakim to believe at first given what he had learned. "It's almost like a disconnect from reality," he says. "Yet here are the results. You can repeat this calculation." The duo also looked at how the model was altering initial conditions, fearing it was doing something unrealistic. They found the model was making small adjustments to parameters such as temperature across large scales. It also seemed to be strengthening certain wind patterns that traditional weather models have been known to dampen. It just goes to show there are ways for AI, if it has enough data, to overcome the approximations and errors that get baked into traditional models, says Animashree Anandkumar, a computer scientist at the California Institute of Technology. "Once you throw everything away, you have a chance to rethink things." Selz, however, says there is no evidence that the adjusted initial conditions are actually closer to the true reality seen in the atmosphere, he says. The adjustments could simply be creating a starting point that's ideal for GraphCast forecasts, in a sort of self-fulfilling prophesy. If that perfect version is perturbed at all, Selz suspects, the lengthened forecast window could close again. "And that's exactly what the butterfly effect tells you." Regardless, the work is raising a lot of questions about the received wisdom, says Dueben, who has always been a little skeptical of how well the butterfly effect applies to weather. "It's probably too narrow a view to say it's only small scales moving upward and destroying prediction limits," he says. That view is echoed by James Doyle, a research meteorologist at the Naval Research Laboratory. Lorenz wasn't wrong that the small errors can proliferate, he says. "But maybe it's not as critical." For now, a monthlong forecast is still aspirational, as it would require a far more refined view of the atmosphere than is currently possible with satellites and weather balloons. But if the new forecast horizon continues to beckon, Doyle says, now is not the time to draw back from weather research. "It tells us there is more to be gained by pushing the models out further."

A new AI-based weather tool surpasses current forecasts

Aurora can be tweaked to predict air pollution, ocean waves, tropical cyclones and more Weather forecasting is getting cheaper and more accurate. An AI model named Aurora used machine learning to outperform current weather prediction systems, researchers report May 21 in Nature. Aurora could accurately predict tropical cyclone paths, air pollution and ocean waves, as well as global weather at the scale of towns or cities -- offering up forecasts in a matter of seconds. The fact that Aurora can make such high-resolution predictions using machine learning impressed Peter Dueben, who heads the Earth system modeling group at the European Centre for Medium-Range Weather Forecasts in Bonn, Germany. "I think they have been the first to push that limit," he says. As climate change worsens, extreme weather strikes more often. "In a changing climate, the stakes for accurate Earth systems prediction could not be higher," says study coauthor Paris Perdikaris, an engineer at the University of Pennsylvania in Philadelphia. And in recent months, the U.S. government has cut funding and fired staff at the National Weather Service, making it more difficult for this agency to get important warnings out in time. Aurora is one in a series of machine learning models that have been steadily improving weather prediction since 2022, Dueben says. His group has used machine learning models similar to Aurora to provide forecasts for two years. "We're running them every single day," he says. Microsoft's MSN Weather app already incorporates Aurora's data into its forecasts. Standard forecasting systems don't use machine learning. They model Earth's weather by solving complex math and physics equations to simulate how conditions will likely change over time. But simulating a system as chaotic as the weather is an extremely difficult challenge. In July 2023, for example, official forecasts a few days in advance of Typhoon Doksuri got its path wrong. When the storm hit the Philippines, there was little warning. Dozens of people died in flooding, landslides and accidents. In a test scenario, Aurora correctly predicted Typhoon Doksuri's track from data collected four days in advance. The team looked at the tracks that seven major forecasting centers had forecasted for cyclones that took place in 2022 and 2023. For every single storm, the AI model's predictions were 20 to 25 percent more accurate. Outperforming the official forecasts for cyclones at up to five days in advance "has never been done before," says study coauthor Megan Stanley, a machine intelligence researcher based at Microsoft Research in Cambridge, England. "As we all know from many cases of typhoons and hurricanes, having even a day's advance notice is enough to save a lot of lives," she says. Unlike standard forecasts, machine learning models don't simulate physics and solve complex math formulas to make predictions. Instead, they analyze large datasets on how weather has changed over time. Aurora took in more than a million hours' worth of information about Earth's atmosphere. It learned how weather patterns tend to evolve. But that was just the start. Aurora is a foundation model. In AI, a foundation model is sort of like a high school graduate. A new grad knows a lot of useful stuff already, but with some additional training, they could perform all sorts of different jobs. Similarly, a foundation model can go through a process called fine-tuning to learn to perform different kinds of specialized tasks. During Aurora's fine-tuning, the team fed the model new kinds of data about different Earth systems, including cyclone tracks, air pollution and ocean waves. The number-crunching for a physics-based weather forecasting model may take several hours on a supercomputer. And developing a new physics-based model takes "decades," Dueben says. Developing Aurora took eight weeks. Because models like Aurora can often be run on a typical desktop and don't require a supercomputer, they could make powerful weather forecasting more accessible to people and places that can't afford to run their own physics-based simulations. And because Aurora is a foundation model that can be fine-tuned, it could potentially help with any kind of Earth forecasting. Stanley and her colleagues imagine fine-tuning the system to predict changes in sea ice, floods, wildfires and more.

AI is good at weather forecasting. Can it predict freak weather events?

Increasingly powerful AI models can make short-term weather forecasts with surprising accuracy. But neural networks only predict based on patterns from the past -- what happens when the weather does something that's unprecedented in recorded history? A new study led by scientists from the University of Chicago, in collaboration with New York University and the University of California Santa Cruz, is testing the limits of AI-powered weather prediction. In research published May 21 in Proceedings of the National Academy of Sciences, they found that neural networks cannot forecast weather events beyond the scope of existing training data -- which might leave out events like 200-year floods, unprecedented heat waves or massive hurricanes. This limitation is particularly important as researchers incorporate neural networks into operational weather forecasting, early warning systems, and long-term risk assesments, the authors said. But they also said there are ways to address the problem by integrating more math and physics into the AI tools. "AI weather models are one of the biggest achievements in AI in science. What we found is that they are remarkable, but not magical," said Pedram Hassanzadeh, an associate professor of geophysical sciences at UChicago and a corresponding author on the study. "We've only had these models for a few years, so there's a lot of room for innovation." Gray swan events Weather forecasting AIs work in a similar way to other neural networks that many people now interact with, such as ChatGPT. Essentially, the model is "trained" by feeding it a bunch of text or images into a model and asking it to look for patterns. Then, when a user presents the model with a question, it looks back at what it's previously seen and uses that to predict an answer. In the case of weather forecasts, scientists train neural networks by feeding them decades' worth of weather data. Then a user can input data about the current weather conditions and ask the model to predict the weather for the next several days. The AI models are very good at this. Generally, they can achieve the same accuracy as a top-of-the-line, supercomputer-based weather model that uses 10,000 to 100,000 times more time and energy, Hassanzadeh said. "These models do really, really well for day-to-day weather," he said. "But what if next week there's a freak weather event?" The concern is that the neural network is only working off the weather data we currently have, which goes back about 40 years. But that's not the full range of possible weather. "The floods caused by Hurricane Harvey in 2017 were considered a once-in-a-2,000-year event, for example," Hassanzadeh said. "They can happen." Scientists sometimes refer to these events as "gray swan" events. They're not quite all the way to a black swan event -- something like the asteroid that killed the dinosaurs -- but they are locally devastating. The team decided to test the limits of the AI models using hurricanes as an example. They trained a neural network using decades of weather data, but removed all the hurricanes stronger than a Category 2. Then they fed it an atmospheric condition that leads to a Category 5 hurricane in a few days. Could the model extrapolate to predict the strength of the hurricane? The answer was no. "It always underestimated the event. The model knows something is coming, but it always predicts it'll only be a Category 2 hurricane," said Yongqiang Sun, research scientist at UChicago and the other corresponding author on the study. This kind of error, known as a false negative, is a big deal in weather forecasting. If a forecast tells you a storm will be a Category 5 hurricane and it only turns out to be a Category 2, that means people evacuated who may not have needed to, which is not ideal. But if a forecast underestimates a hurricane that turns out to be a Category 5, the consequences would be far worse. Hurricane warnings and why physics matters The big difference between neural networks and traditional weather models is that traditional models "understand" physics. Scientists design them to incorporate our understanding of the math and physics that govern atmospheric dynamics, jet streams and other phenomena. The neural networks aren't doing any of that. Like ChatGPT, which is essentially a predictive text machine, they simply look at weather patterns and suggest what comes next, based on what has happened in the past. No major service is currently using only AI models for forecasting. But as their use expands, this tendency will need to be factored in, Hassanzadeh said. Researchers, from meteorologists to economists, are beginning to use AI for long-term risk assessments. For example, they might ask an AI to generate many examples of weather patterns, so that we can see the most extreme events that might happen in each region in the future. But if an AI cannot predict anything stronger than what it's seen before, its usefulness would be limited for this critical task. However, they found the model could predict stronger hurricanes if there was any precedent, even elsewhere in the world, in its training data. For example, if the researchers deleted all the evidence of Atlantic hurricanes but left in Pacific hurricanes, the model could extrapolate to predict Atlantic hurricanes. "This was a surprising and encouraging finding: it means that the models can forecast an event that was unpresented in one region but occurred once in a while in another region," Hassanzadeh said. Merging approaches The solution, the researchers suggested, is to begin incorporating mathematical tools and the principles of atmospheric physics into AI-based models. "The hope is that if AI models can really learn atmospheric dynamics, they will be able to figure out how to forecast gray swans," Hassanzadeh said. How to do this is a hot area of research. One promising approach the team is pursuing is called active learning -- where AI helps guide traditional physics-based weather models to create more examples of extreme events, which can then be used to improve the AI's training. "Longer simulated or observed datasets aren't going to work. We need to think about smarter ways to generate data," said Jonathan Weare, professor at the Courant Institute of Mathematical Sciences at New York University and study co-author. "In this case, that means answering the question 'where should I place my training data to achieve better performance on extremes?' Fortunately, we think AI weather models themselves, when paired with the right mathematical tools, can help answer this question." University of Chicago Prof. Dorian Abbot and computational scientist Mohsen Zand were also co-authors on the study, as well as Ashesh Chattopadhyay of the University of California Santa Cruz. The study used resources maintained by the University of Chicago Research Computing Center.

Breakthrough AI model could transform how we prepare for natural disasters

As climate-related disasters grow more intense and frequent, an international team of researchers has introduced Aurora -- a groundbreaking AI model designed to deliver faster, more accurate, and more affordable forecasts for air quality, ocean waves, and extreme weather events. This model, called Aurora, has been trained on over a million hours of data. According to the researchers, it could revolutionize the way we prepare for natural disasters and respond to climate change. From deadly floods in Europe to intensifying tropical cyclones around the world, the climate crisis has made timely and precise forecasting more essential than ever. Yet traditional forecasting methods rely on highly complex numerical models developed over decades, requiring powerful supercomputers and large teams of experts. According to its developers, Aurora offers a powerful and efficient alternative using artificial intelligence. Machine learning at the core 'Aurora uses state-of-the-art machine learning techniques to deliver superior forecasts for key environmental systems -- air quality, weather, ocean waves, and tropical cyclones,' explains Max Welling, machine learning expert at the University of Amsterdam and one of the researchers behind the model. Unlike conventional methods, Aurora requires far less computational power, making high-quality forecasting more accessible and scalable -- especially in regions that lack expensive infrastructure. Trained on a million hours of earth data Aurora is built on a 1.3 billion parameter foundation model, trained on more than one million hours of Earth system data. It has been fine-tuned to excel in a range of forecasting tasks: Forecasting that's fast, accurate, and inclusive As climate volatility increases, rapid and reliable forecasts are crucial for disaster preparedness, emergency response, and climate adaptation. The researchers believe Aurora can help by making advanced forecasting more accessible. 'Development cycles that once took years can now be completed in just weeks by small engineering teams,' notes AI researcher Ana Lucic, also of the University of Amsterdam. 'This could be especially valuable for countries in the Global South, smaller weather services, and research groups focused on localised climate risks.' 'Importantly, this acceleration builds on decades of foundational research and the vast datasets made available through traditional forecasting methods,' Welling adds. Aurora is available freely online for anyone to use. If someone wants to fine-tune it for a specific task, they will need to provide data for that task. 'But the "initial" training is done, we don't need these vast datasets anymore, all the information from them is baked into Aurora already', Lucic explains. A future-proof forecasting tool Although current research focuses on the four applications mentioned above, the researchers say Aurora is flexible and can be used for a wide range of future scenarios. These could include forecasting flood risks, wildfire spread, seasonal weather trends, agricultural yields, and renewable energy output. 'Its ability to process diverse data types makes it a powerful and future-ready tool', states Welling. As the world faces more extreme weather -- from heatwaves to hurricanes -- innovative models like Aurora could shift the global approach from reactive crisis response to proactive climate resilience concludes the study.

Weather forecasting improves with AI, but we still need humans

Weather forecasts are notoriously unreliable. Most people can relate to booking a trip or making plans expecting a sunny day, only to have it disappointingly rained out. While seven-day weather forecasts are accurate about 80 percent of the time, that figure drops to around 50 percent when extended to 10 days or more. Recent staffing cuts at the National Weather Service have already led to reduced weather balloon data collection, which experts warn could further degrade forecast accuracy. That's a minor inconvenience when it ruins a picnic, but it can be life-threatening if a forecast fails to predict a tornado or hurricane. But a series of recent advances in artificial intelligence-based forecasting models from big tech firms like Microsoft and Google might offer a silver lining. This new generation of AI systems, some of which are already being used in parts of Europe's weather centers, are faster to build and easier to update than traditional models, which can take years to develop. Early testing shows several of these systems are also more accurate than conventional models at predicting weather up to 15 days in advance. Researchers have previously noted that extending reliable forecasts from 10 to 15 days could yield "enormous socioeconomic benefits" by helping people better prepare for the impacts of extreme weather. Aurora, a new Microsoft model detailed this week in the journal Nature, outperformed the traditional models used by the European Centre for Medium-Range Weather Forecasts in more than 90 percent of the forecasts tested. It also proved more effective at predicting several extreme weather events, including typhoons and sandstorms. Though AI systems still rely on the foundational equations used in traditional models, these recent advances point to a future where researchers can respond more quickly to evolving weather patterns and deliver more accurate forecasts. All of that work, however, still depends on the continued rapid collection of accurate real-world weather data. Traditional weather forecasting is an expensive and time-consuming enterprise. The current process, in place for roughly 70 years, relies on supercomputers to solve complex mathematical equations that factor in variables like ocean currents and solar heating. Real-world weather data is then input into these equations. Eventually, the models produce several predictive outputs, which are reviewed by a human meteorologist who uses their expertise to finalize the forecast. These models can take years to build and are not easily updated. AI-based weather systems, by contrast, are smaller and more easily "fine-tuned" with newer environmental data. These models are broadly trained on large datasets of weather and climate information to recognize patterns. That pattern recognition allows them to make predictions about what could come next. In the case of Microsoft's Aurora, the "foundation model" was trained on over one million hours of data collected from satellite radar, weather stations, simulations, and other forecasts. Researchers involved in the study believe this represents the largest collection of atmospheric data ever assembled to train a forecasting model. That base model can then be tailored to predict specific types of weather events by adding additional data related to the event in question. When it came to hurricanes, Aurora was able to generate forecasts for hypothetical storms five days in advance with 15-20 percent greater accuracy than the top traditional model. It went on to outperform seven major forecasting models on all cyclone track predictions globally during the 2022-2023 season. The reported improvements weren't limited to hurricanes. Microsoft says its model accurately predicted the date and location of the devastating Typhoon Doksuri's landfall in the Philippines in July 2023 -- days before it occurred. At the time, official forecasts using traditional models had incorrectly identified the storm's landfall location. Aurora also successfully predicted the onset of a major sandstorm in Iraq one day before it happened. Researchers say the model made that prediction much faster and at a significantly lower cost than traditional forecasting methods. "Earth's climate is perhaps the most complex system we study -- with interactions spanning from quantum scales to planetary dynamics," University of Pennsylvania associate professor and paper co author Paris Perdikaris said in a statement. "With Aurora, we addressed a fundamental challenge in Earth system prediction: how to create forecasting tools that are both more accurate and dramatically more computationally efficient." The findings come just months after similarly impressive results were documented for GenCast, an AI weather forecasting model developed by Google DeepMind. In a separate Nature paper, researchers reported that GenCast significantly outperformed traditional models in medium-term forecasting accuracy. Specifically, it surpassed the Global Ensemble Prediction System, a model used by 35 countries, 97.2 percent of the time in 15-day forecasts. Like Aurora, GenCast was trained on a vast dataset of weather events, in this case spanning 40 years (from 1978 to 2018), and uses that information to predict future conditions. The model presents its forecasts probabilistically, drawing from a set of 50 or more predictions to generate its results. All of this occurs far more quickly than with traditional forecasting methods. The rapidly evolving field of AI weather forecasting is unfolding against the backdrop of job cuts and reduced capacity at the U.S. National Weather Service. These cuts -- spearheaded by Elon Musk's cost-cutting organization DOGE -- have reportedly led to the elimination of several critical weather balloon launches during tornado and hurricane season. These balloon launches, which typically occur twice daily at over 100 locations, provide meteorologists with real-time atmospheric data used to predict where severe weather events may strike. AI weather systems may be able to draw from massive training datasets to outperform traditional models in many cases, but they still rely on timely, location-specific data -- like that collected by weather balloons -- to make accurate forecasts about rapidly developing storms such as tornadoes.

Atmospheric scientists suggest that AI could be used to make 30-day weather forecasts

A team of atmospheric scientists at the University of Washington has found evidence that weather forecasters may be able to look ahead for up to 30 days when making predictions. In their study, posted on the arXiv preprint server, the group tested Google's GraphCast AI-based weather modeling and predicting system using a technique to improve initial weather conditions to improve its accuracy. Over the past half-century, weather forecasters have come to believe that a two-week forecast period is the ultimate limit. This is because of the so-called butterfly effect, in which tiny events, such as the wind created by a butterfly's wings, can lead to cascading effects, resulting in greater impacts. The butterfly effect is a thought experiment, but it is known that random events such as fires, volcanic eruptions and human activity can cause local weather changes. In this new effort, the researchers working in Washington have been testing the possibility of using AI technology to lengthen the forecasting window. The researchers conducted tests with GraphCast, an AI weather forecasting model built by Google -- it learns via training on 40 years of data from traditional forecasts and satellite imagery. They wondered if improving the accuracy of the initial conditions used to generate a forecast could improve the model's overall accuracy. The research team compared forecasts made by the model with the most recent state of the atmosphere taken from data used to train the model. They then used miscues made in short-term forecasts as a way to adjust the initial conditions and then applied them to the reanalysis data used to train the model, giving it a more accurate starting point. They then repeated the same exercise more than 1,000 times, each time making the initial conditions more accurate. The researchers then trained GraphCast using the newly revised data, and found that it improved its 10-day forecasting ability by 86% on average. It also made reasonably accurate predictions up to 33 days into the future. The researchers acknowledge that much more work is required before AI models can make accurate, long-term predictions, including testing their approach to see how well it works in the real world.

AI model beats global agencies on cyclone track forecasts

LLM trained on decades of weather data claimed to be faster, and cheaper Scientists have developed a machine learning model that can outperform official agencies at predicting tropical cyclone tracks, and do it faster and cheaper than traditional physics-based systems. Aurora, a foundation model developed by researchers from Microsoft, the University of Pennsylvania (UPenn), and several other institutions, is designed to improve the speed and accuracy of Earth system forecasts, from air quality and ocean waves to tropical cyclone tracks and high-resolution weather. Paris Perdikaris, co-lead author and associate professor of mechanical engineering and applied mechanics at UPenn, describes Aurora as a large neural network. Like ChatGPT does for text, Aurora learns from past geophysical data to predict complex physical processes, without relying explicitly on traditional physics equations. "Traditional models are designed on first physical principles, like conservation of mass, momentum and energy," he said. "Aurora, on the other hand, is not directly using those physical principles, but instead it relies on observations and data. "Aurora learns from a very diverse set of geophysical data, including forecasts, observations, and what we call analysis and re-analysis data, which is basically a reconstruction of historical weather patterns we have access to." As the researchers acknowledge in the Nature paper published Wednesday, Aurora's rapid progress depends heavily on the groundwork laid by traditional methods. "Such an accelerated timeline is only possible because of the wealth of data that is available as a result of decades of research into traditional numerical approaches." Aurora was pretrained on more than one million hours of diverse geophysical data and fine-tuned over four to eight weeks by small engineering teams, "compared with a typical development period of several years for dynamical baseline models," the paper noted. Aurora, trained only on historical data, was able to correctly forecast all hurricanes in 2023 more accurately than operational forecasting centers The model uses a combination of Perceiver-based encoders, a 3D Swin Transformer backbone, and recursive forecasting techniques, relying on multi-dimensional vector embeddings similar to those used in large language models. "Aurora, trained only on historical data, was able to correctly forecast all hurricanes in 2023 more accurately than operational forecasting centers," Perdikaris touted. The authors reported that Aurora outperformed seven operational forecasting centers on five-day tropical cyclone track predictions for all global cyclones in 2022-2023. It also surpassed state-of-the-art numerical models on 92 percent of targets in 10-day global weather forecasts at 0.1° resolution. As a foundation model, they suggest Aurora could be fine-tuned for a wide range of Earth system prediction tasks beyond weather, including air quality, ocean dynamics, and environmental extremes. "The potential implications of Aurora for the field of Earth system prediction are profound," the authors wrote. "Although in this paper we showcase the application of Aurora to four domains, it could be fine-tuned for any desired Earth system prediction task, potentially producing forecasts that outperform the current operational systems at a fraction of the cost." Potential applications range from modeling ocean currents, short- and long-term weather patterns, and vegetation cycles, to forecasting wildfires, floods, crop yields, pollination behavior, renewable energy output, and shifts in sea ice coverage. "With the ability to fine-tune Aurora to diverse application domains at only modest computational cost, Aurora represents notable progress in making actionable predictions accessible to anyone," they added. It's not the only AI model shaking up meteorology. In March, Aardvark, a novel machine learning-based weather prediction system, showed promise in outperforming traditional supercomputer-based models. The system can be trained and run on a desktop equipped with NVIDIA GPUs, generating 10-day forecasts in minutes, and at a fraction of the computational cost of current numerical weather models. ®

The AI revolution changing how we predict the weather

The era of delivering operational weather predictions by computer began for the UK Meteorological Office in 1965 with a room-sized processor nicknamed Comet. Six decades later, the country's national forecaster is part of another technological revolution, this time driven by artificial intelligence. AI is supercharging predictions for the ever-shifting patterns of cloud, precipitation and temperature mapped dynamically on a giant screen at the organisation's headquarters in the south-western city of Exeter. "We see the potential for a real step change . . . in how we forecast, which is in some ways similar to when we started using computers," says Kirstine Dale, the Met Office's chief AI officer, citing rapidly growing quantities of data, computing power to handle it and models to process it. "It's all just got bigger -- and the possibilities have also become much bigger." AI's ability to spot patterns in vast troves of data makes the swirling systems of atmospheric physics an ideal opportunity to experiment with the technology. More accurate forecasts and warnings of severe weather could enhance public safety and health, while increasing efficiency across the global economy. AI has the power to make projections of future weather patterns more accurate and more detailed, opening up new possibilities for groups from farmers to finance companies. The technology is "moving out in timescale" having already reshaped "nowcasting", or the super-accurate forecasting of the coming hours, says Richard Turner, a Cambridge university machine learning professor. "You've seen medium range -- three to 15 days -- start to get transformed. And now we're building out to sub-seasonal", or roughly two weeks to two months, says Turner, who works on AI weather models at the UK's Alan Turing Institute, the national AI centre. The opportunities have attracted substantial investments by tech companies including Google DeepMind, Nvidia, Microsoft, IBM and specialist AI weather start-ups, such as US-based Brightband and Silurian. Organisations throughout the forecasting ecosystem are preoccupied with how best to apply AI to improve our understanding of weather. They include public meteorological offices -- some, like the UK Met Office, dating back to the 19th century -- as well as universities and specialist companies such as AccuWeather, The Weather Company and DTN. These provide tailored forecasts to users in sectors from energy to construction, agriculture to transport, retailing to tourism, and to the general public via the news media. But the generally sunny outlook is darkened by the threat of shrinking access to the data on which the AI models depend. The Trump administration is seeking deep cuts in funding and staffing at the US National Oceanic and Atmospheric Administration, the federal agency dedicated to understanding and predicting changes in climate, weather, oceans and coasts. Noaa's satellites, ocean buoys, balloons and radars are an important source of data, all of which are freely available to meteorologists worldwide and fed into global forecasting models. President Donald Trump's proposed 2026 budget would cut Noaa's funding by $1.5bn, or 24 per cent. Since he took office in January, 550 employees have left its forecasting arm, the National Weather Service. This month, all five of its living former directors signed an open letter warning that the cuts have left local forecasting offices in the US severely short of staff, which could lead to a needless loss of life. At the same time, some fear that rising geopolitical tensions could threaten the free flow of public weather data on which the world's forecasters depend. Although everyone in meteorology laments the potential loss of Noaa data, some point to a countervailing factor -- the prospect of new information sources. The next generation of AI models could improve accuracy by including vast quantities of data from local weather sensors, such as thermometers and rain gauges, that are not currently incorporated into global forecasting systems. "Suddenly we're in this place where a new sensor can be set up and we can ingest that into the model very quickly," says Scott Hosking, who works on weather prediction at the Alan Turing Institute. Hosking estimates that 20 to 30 different AI weather models are at various stages of development today, some being run operationally by forecasters. "In a year's time, there will be many more," he adds. "How rapidly this has overtaken at least the weather forecasting part of our science is truly remarkable," says Peter Neilley, senior vice-president of science and forecasting operations at The Weather Company, one of the world's leading forecasting businesses. "It really occurred in the last five years and it's accelerating." Until recently, forecasting largely relied on numerical weather prediction, which involves feeding millions of real-time worldwide observations from satellites and sensors on land, sea and air into supercomputers and crunching them with physics-based equations. This process can be broken down into two steps. The first is data assimilation, which prepares an estimate of the state of the atmosphere, followed by the forecasting stage, which makes predictions about what will happen next. The pioneering generation of AI weather systems coming into operation still require computer-intensive data assimilation but they then use machine learning to run the model forward in time. Early results have been positive. The European Centre for Medium-Range Weather Forecasts, an intergovernmental organisation based in Reading, UK, says its first operational AI model, launched in February, has improved accuracy by about 20 per cent on key indicators such as predicting the path of tropical cyclones, giving valuable extra warning time. Florence Rabier, ECMWF director-general, believes that new AI technology will build on huge improvements in forecast accuracy achieved over recent decades, as computers became ever more powerful and weather data more plentiful. Predictions seven days ahead now match the quality of those five days ahead in 2000 and three days ahead in 1980. There are global ramifications, she observes. "In the late 20th century, we could predict the weather much better in the northern than in the southern hemisphere because there were so many more observations there," Rabier says. "Since the beginning of the 2000s, more advanced satellite data have become available and the accuracy gap between the hemispheres has disappeared." But a second generation of experimental "end-to-end" AI systems is emerging, which could offer even more exciting possibilities. These new iterations dispense with data assimilation and instead work directly on raw observations from satellites, weather stations and other sensors to generate both global and local forecasts. In March, a team at the Turing institute, working with ECMWF and other partners, published details of an experimental end-to-end system called Aardvark, which is so energy efficient that it can run on desktop machines rather than supercomputers. About 10 other research groups around the world in tech companies and the public sector are developing their own end-to-end models and many more are likely to join in soon, says Turing's Hosking. Their output promises to "democratise" forecasting even further, particularly in developing nations and data-sparse regions, where local observations can be added to forecasts with relatively modest computing requirements. Because AI models are trained on many years of past observations, there have been doubts about how well they will work in future, particularly as the climate changes, says Florian Pappenberger, ECMWF deputy director. But he rejects that criticism. "We have shown that a machine learning model can predict extreme and unusual events, such as record rainfall in the United Arab Emirates last year and snowfall in New Orleans this year," he says. "Machine learning learns about physics in general and not just past analogues [patterns] at a given place, so it is much more powerful than some people say." The so-called "ensemble technique" has already improved numerical weather predictions. This involves running the computer model many times from slightly different starting conditions instead of producing just a single "deterministic" forecast. This variation gives meteorologists essential information about the level of uncertainty and range of possible outcomes. But the practice is so resource intensive that it is impractical to produce ensembles with more than about 50 different forecasts, says Dion Harris, director of accelerated data centre solutions for Nvidia, the US chip powerhouse. This is where AI can be helpful, says Harris. "Using AI techniques, you can literally do thousands of ensembles, which translates into a much better understanding of the potential outcomes and helps you have an earlier indication of extreme events." If the new wave of AI models are to fulfil their potential, though, an enhanced flow of weather observations will be essential. Most of the raw data still comes from the public sector, through forecasters and satellite operators such as Noaa and the European Organisation for the Exploitation of Meteorological Satellites, and is shared freely worldwide. "The level of international data-sharing has just been fantastic. You can go and get data from Chinese satellites, for instance," says Turner at Cambridge. "The observations are all put into a pool, because they help everyone else's forecasting system, and they get people to fire reciprocal data back." Any reduction in global data availability, whether from growing geopolitical tensions or cuts to Noaa activities by the Trump administration, "is a huge worry", says Turner. "The community hasn't -- surprisingly, in my view -- woken up to this danger yet . . . Yes, there is massive concern on this and I think the cuts are very dangerous at a time when the climate really is changing." Some people in the public sector are thinking about ways to protect -- or enhance -- the way data is gathered. With the price of building, launching and operating observational satellites in low Earth orbit falling fast, private weather companies are beginning to invest in their own satellite clusters, or constellations. Tomorrow.io, a Boston start-up, says it has launched two radar satellites and seven microwave sounding satellites that see through clouds to detect rain and snow falling. The company, which has raised $300mn from investors since its foundation in 2016, plans to launch four more sounders this year and continue adding to its constellation next year. Each microwave satellite weighs just 12kg and costs less than $10mn, including launch, says Shimon Elkabetz, Tomorrow.io's chief executive. "When we started everyone said it would be too expensive to build our own constellation, but the new space economy is enabling us to do things that weren't possible before." But Elkabetz says that while private companies could help to increase the "efficiency and impact" of public sector forecasting, they cannot substitute for the huge national agencies such as Noaa. For all its promise, meteorologists are still uncertain about how AI will revolutionise forecasting. On some criteria, such as predicting the intensity of storms, the performance of AI does not yet match the best numerical weather systems. The atmosphere's intrinsic unpredictability prevents accurate and detailed day-by-day forecasting beyond two weeks or so, irrespective of the technology, says Robert Lee, a meteorologist at Reading university researching subseasonal weather patterns. "But we could predict that there will be a period of a few days with stormy conditions or a few days with cold wintry conditions." Hedge funds make money [as a result of accurate forecasts]. They hire a lot of our [meteorology] graduates Being able to see further into the future carries huge benefits. It would be particularly useful as countries become more reliant on clean energy systems that are weather-dependent, such as wind or solar. "Even if you don't know exactly which day will have the coldest weather, you want to know that you'll have a week of cold weather to be sure you don't have a shortfall of energy," Lee says. "You might want to buy gas supplies on the futures market as a precaution." At the same time, after an early heads-up of incoming wet and windy weather "you might want to sell your gas futures because there'll be more wind generation and less demand for power", he adds. "That's how some hedge funds make money. They hire a lot of our [meteorology] graduates." Advocates of AI say another huge advantage over traditional methods is that it can make very detailed predictions about specific locations. Using generative AI, Nvidia's CorrDiff system can sharpen the precision of data from conventional numerical models from 25km down to 2km. CorrDiff was originally trained on data from Taiwan, where it is now used operationally for severe weather warnings. It has been adapted to the continental US and will be rolled out in other parts of the world, Harris says. At the UK Met Office, AI chief Dale says forecasting at scales as small as 100 metres is a viable goal -- and was done for a study of hyperlocal temperature predictions published last year. "That's like street-level forecasting," she says. Greater forecast detail would enable businesses to offer new services to customers, says Neilley of The Weather Company. Better warnings of approaching thunderstorms would give airports more time to plan aircraft departures and landings. While some fear that data-intensive AI models will use ever more energy, the long-term environmental impact of AI forecasting is nuanced. Its effects will depend on the balance between the power consumption required to generate and disseminate forecasts and the energy savings that it makes possible. And just as with other sectors, the conversation about AI and greater automation also becomes an existential one. If the world starts to rely on AI to predict the weather, what role is left for meteorologists? Met Office officials doubt that the rise of AI will end the employment of people making sense of phenomena from flooding to space weather. Meteorologists will still be needed, they say -- perhaps more than ever. They will have to adjudicate differences between duelling AI models. They will maintain a vital role in putting raw forecasting numbers in context and communicating levels of risk and suggested mitigation. Raw data will still need to be collected and in some cases vetted to account for anomalies. The Met Office itself had to rescind a "record" Scottish temperature set in 2018 because the sensor was close to an ice cream van. The Met's AI chief Dale argues the technology is transforming our understanding of weather -- but as a powerful ally of long-established prediction methods, rather than an alternative. "I see an increasingly symbiotic relationship," she says. "We need them to work together on a team."

A.I. Is Poised to Revolutionize Weather Forecasting. A New Tool Shows Promise.

A Microsoft model can make accurate 10-day forecasts quickly, an analysis found. And, it's designed to predict more than weather. Weather forecasters rely on models to help them make decisions that can have life-or-death consequences, so any advantage is welcome. Artificial intelligence holds promise to deliver more accurate forecasts quickly, and tech companies including Google, Nvidia and Huawei have produced A.I.-based forecasting models. The latest entrant is Aurora, an A.I. weather model from Microsoft, and it stands out for several reasons, according to a report published Wednesday in the journal Nature. It's already in use at one of Europe's largest weather centers, where it's running alongside other traditional and A.I.-based models. The Aurora model can make accurate 10-day forecasts at smaller scales than many other models, the paper reports. And it was built to handle not only weather, but also any Earth system with data available. That means it can be trained, relatively easily, to forecast things like air pollution and wave height in addition to weather events like tropical cyclones. Users could add almost any system they like down the road; for instance, one start-up has already honed the model to predict renewable energy markets. "I'm most excited to see the adoption of this model as a blueprint that can add more Earth systems to the prediction pipeline," said Paris Perdikaris, a professor at the University of Pennsylvania who led the development of Aurora while working at Microsoft. It's also fast, able to return results in seconds as opposed to the hours that non-A.I. models can take. Traditional models, the basis of weather forecasting over the last 70 years, use layers of complex mathematical equations to represent the physical world: the sun heating the planet, winds and ocean currents swirling around the globe, clouds forming, and so on. Researchers then add real weather data and ask the computer models to predict what will happen next. Human forecasters look at results from many of these models and combine those with their own experience to tell the public what scenario is most likely. "Final forecasts are ultimately made by a human expert," Dr. Perdikaris said. (That is true for A.I.-based forecasts, too.) This system has worked well for decades. But the models are incredibly complex and require expensive supercomputers. They also take many years to build, making them difficult to update, and hours to run, slowing down the forecasting process. Artificial intelligence weather forecasting models are faster to build, run and update. Researchers feed the models on huge amounts of weather and climate data and train them to recognize patterns. Then, based on these patterns, the model predicts what comes next. But the A.I. models still need equation-based models and real-world data for their starting points, and for reality checks. "It doesn't know the laws of physics, so it could make up something completely crazy," said Amy McGovern, a computer scientist and meteorologist at the University of Oklahoma who was not involved in the study. So most, but not all, A.I. weather forecasting models still rely on data and the physics-based models in some capacity, and human forecasters need to interpret results carefully. Dr. Perdikaris and his collaborators built Aurora using this method, training it on data from physics-based models and then making purely A.I. predictions, but they didn't want it to be limited to weather. So they trained it on multiple, big Earth system data sets, creating a broad background of artificial expertise Aurora "is an important step toward more versatile forecasting systems," said Sebastian Engelke, a professor of statistics at the University of Geneva who was not involved in the study. The model's flexibility and resolution are its most novel contributions, he said. As in other areas, there's been a big push toward using A.I. for weather forecasting in the past few years, but the major A.I. forecasting models are still global, not local. Forecasts at the scale of a single storm barreling toward a city need to come from a specialized model, and those are mostly the old-school variety, at least for now. Extreme weather events like heat waves or heavy downpours are still challenging for both traditional and A.I. models to predict. A.I. forecasting models need careful calibration and human verification before they're widely used, Dr. Perdikaris said. But some are already being tested in the real world. The European Center for Medium-Range Weather Forecasting, which provides meteorological forecasts to dozens of countries, developed their own A.I. forecasting model, which they deployed in February. They use that, along with Aurora and other A.I. models, for their weather services. They've had a good experience using A.I. models so far. "It's absolutely an exciting time," said Peter Düben, who leads the European center's Earth modeling team. Other researchers are more conservative, given the checks and improvements the models need. And artificial intelligence tools come with a significant energy cost to train, though Dr. Perdikaris said this would be worth it in the long run as more people use the models. "We're all in the hype right now," said Dr. McGovern, who leads the NSF's institute that studies trust in A.I. applications to climate and weather problems. "A.I. weather is amazing. But I think there's still a long way to go." And the Trump administration's cuts to agencies including the National Oceanic and Atmospheric Administration, the National Science Foundation and the National Weather Service could stymie further improvements in A.I. forecasting tools, because federal data sets and models are critical to developing and improving A.I. models, Dr. Perdikaris said. "It's quite unfortunate, because I think it's going to slow down progress," he said.

How this AI weather model by Microsoft produces faster and more accurate forecasts

The model is the latest development in a growing cadre of AI weather models that have emerged in the past few years A new model developed by researchers at Microsoft uses artificial intelligence to produce more accurate weather forecasts -- at a much faster speed and lower cost -- than traditional models operated by government weather agencies, according to a study published Wednesday in the journal Nature. The model, named Aurora, also can forecast more environmental phenomena at a more granular level than other recently developed AI models. Aurora is the latest in a growing cadre of AI weather models that have emerged in the past few years. They have made their way into everyday use by meteorologists and, experts say, have the potential to speed up improvements in forecast accuracy while reducing reliance on expensive traditional models that are more tedious to update. Traditional weather models, such as those operated by the European Center for Medium-Range Weather Forecasts and the U.S. National Oceanic and Atmospheric Administration, generate forecasts by processing data through complex mathematical equations on large supercomputers that cost billions of dollars. AI weather models, on the other hand, make predictions by learning to recognize patterns in data, which can be done thousands of times faster on much cheaper computers as small as a desktop. Aurora was trained on more than 1 million hours of past weather and climate data to learn the processes that govern circulation through the atmosphere and oceans. With additional training on smaller datasets, the model was then fine-tuned for specific tasks such as forecasting weather, air quality and ocean waves. "Aurora represents a significant leap in AI weather modeling by demonstrating that a single foundation model can be fine-tuned for multiple Earth system tasks," said Paris Perdikaris, a co-author of the study and former principal research manager at Microsoft Research. The study showed Aurora to be 24 percent more accurate than the "European" model, a traditional model operated by the European Center and consistently ranked as the world's most accurate global weather model. Aurora beat the European model on more than 90 percent of the forecast variables tested, including wind speed and temperature, generating a 10-day global weather forecast in less than a minute compared to more than an hour for the European model. After being fine-tuned to forecast hurricanes, among the costliest and deadliest of weather disasters, Aurora produced track forecasts up to five days in advance that were 20 percent to 25 percent more accurate than top-ranked traditional weather models and official government forecasts. Training a model like Aurora on such a large and diverse set of data "is a tremendous engineering achievement," said Daniel Rothenberg, a meteorologist who builds AI weather forecasting systems but wasn't involved with Aurora's development or the new study. Some have questioned the ability of AI weather models to forecast extremes that are outside the bounds of the historical data they are trained on. However, Aurora accurately predicted a sudden increase in wind speed during a 2023 storm that produced the lowest atmospheric pressure on record for the month of November in England. "While Aurora ... demonstrates capability for extremes, I wouldn't consider this concern fully resolved," cautioned Perdikaris, now an associate professor at the University of Pennsylvania. "Reliably predicting extremes remains challenging for both AI and traditional models." Aurora also can make forecasts at smaller scales than previous AI models, which "is essential ... to accurately resolve high-impact weather events such as severe storms," the authors wrote. Aurora captures changes in weather conditions approximately every six miles, similar to the European model, while other AI models are closer to 15 miles. <b>A blizzard of AI advances in weather forecasting</b> Aurora's arrival comes just two months after another breakthrough in AI weather forecasting -- a model called Aardvark developed by researchers from the University of Cambridge with support from the Alan Turing Institute, Microsoft Research and the European Center. All weather models use data describing the current state of the atmosphere as a starting point for making a forecast. Unlike other AI weather models, which require such data to be processed through a traditional model first, Aardvark ingests data directly from satellites and other sensors. While Aardvark, like all AI weather models, still relies on traditional models for training, its ability to bypass those models when generating forecasts "represents a much greater evolutionary leap forward" than Aurora, said Rothenberg, co-founder of the AI weather start-up Brightband, which is partnering with NOAA to make government weather data more accessible for training AI models. Aurora could eventually directly consume observational data just like Aardvark, setting the stage for "future systems that could handle end-to-end forecasting -- observations to predictions -- across multiple domains at high resolution and operational scale," Perdikaris said. The rapid pace of AI weather model development in recent years has largely been driven by the private sector, with Microsoft, Google, NVIDIA and China-based Huawei all having announced global AI models equaling or surpassing the accuracy of leading traditional models. More recently, the European Center built its own global AI weather model that it says is 20 percent more accurate than the traditional European model. The AI model officially became operational in February, although its forecasts have been publicly available since 2023. Meanwhile, NOAA "does not yet have a formal plan" to develop an equivalent to the European Center's global AI model, Daryl Kleist, the acting director of NOAA's Environmental Modeling Center, said in October. NOAA did not immediately respond to a request for an update on its AI weather modeling activities and if any plans would be impacted by proposed budget cuts. But its website links to datasets for training and initializing AI models, and it highlights development of a regional AI model that can generate thunderstorm forecasts in about 30 seconds. <b>Assessing the accuracy of AI weather models</b> The accuracy of AI models compared to traditional ones varies based on the weather pattern and season, according to Matt Rogers, a meteorologist and co-founder of Commodity Weather Group, a weather risk management firm. Rogers is also a contributor to The Washington Post's Capital Weather Gang. "The accuracy has varied over the past 1.5 years we have been tracking this information," said Rogers, whose firm operates an online portal providing access to traditional and AI weather models. "There was a big rush" to follow the AI models when they were performing well, "but then pullback [during] mixed and underperformance periods." Rogers said there are still limitations among existing AI models. For example, they don't yet make forecasts for as many weather variables as traditional models. And while AI models have demonstrated increased accuracy in predicting hurricane tracks, they haven't shown the same improvement for intensity forecasts, which are critical to preparing for a landfalling storm. "Bottom line, [AI models] are not taking over in operational usage, but there are times when their value is noteworthy," Rogers said. "They are essentially another tool in the toolbox."

AI can predict the weather - but what about extreme events? - Earth.com

Artificial intelligence (AI) is already impressing scientists with its ability to forecast the weather days in advance. These new models can run faster and use far less computing power than traditional weather systems. But there's a catch: AI can only predict what it's seen before. And when it comes to extreme weather, that's a serious problem. Researchers from the University of Chicago, New York University, and the University of California Santa Cruz set out to explore this issue. The study shows that while AI can do a great job with typical day-to-day forecasting, it struggles with rare or unprecedented events. These include things like Category 5 hurricanes, once-in-a-century floods, or heat waves that break all historical records. AI models like ChatGPT learn from massive amounts of data. Weather-focused neural networks work the same way. Scientists feed them decades of past weather observations. Based on that, the models learn to predict what might happen next. When you give these models the latest weather data, they can quickly generate forecasts that rival the accuracy of traditional supercomputer models. But the problem comes when something totally unexpected happens. Pedram Hassanzadeh is an associate professor of geophysical sciences at UChicago and a corresponding author of the study. "AI weather models are one of the biggest achievements in AI in science. What we found is that they are remarkable, but not magical," said Hassanzadeh. "We've only had these models for a few years, so there's a lot of room for innovation." The concern is this: if a dangerous weather event hasn't happened in the training data, will the AI model still catch it? The researchers tested this by training an AI model with weather data that excluded all hurricanes above Category 2 strength. Then they asked the model to forecast a scenario that would typically lead to a Category 5 hurricane. The result? "It always underestimated the event. The model knows something is coming, but it always predicts it'll only be a Category 2 hurricane," said Yongqiang Sun, research scientist at UChicago and co-author of the study. This kind of error is called a false negative. In weather forecasting, it can be deadly. Overestimating a storm might cause an unnecessary evacuation, which is costly but not dangerous. Underestimating it, however, can leave people unprepared for disaster. Traditional weather models use equations that describe how the atmosphere works. They include physics, math, and knowledge of how heat, pressure, and wind interact. Neural networks don't do this. They're more like fancy autocomplete tools - offering predictions based only on what they've seen before. This difference matters. It means AI models might miss events that fall outside their training history. Scientists are beginning to use AI to explore long-term risks and future climate scenarios. But if AI can't forecast extremes, its usefulness for those tasks becomes limited. Still, there's hope. The researchers found that if the model had seen a similar extreme event - even in a different part of the world - it could make better predictions. For instance, if the AI never saw an Atlantic hurricane but had seen strong Pacific hurricanes, it could still forecast powerful Atlantic storms. "This was a surprising and encouraging finding: it means that the models can forecast an event that was unpresented in one region but occurred once in a while in another region," Hassanzadeh said. So what's the solution? The researchers believe that blending traditional physics with AI is the next step. "The hope is that if AI models can really learn atmospheric dynamics, they will be able to figure out how to forecast gray swans," Hassanzadeh said. To make that happen, scientists are exploring new techniques. One of them is called active learning. In this approach, AI models help guide physics-based simulations to create more examples of rare events. These examples can then be used to improve the AI's accuracy. Study co-author Jonathan Weare is a professor at the Courant Institute of Mathematical Sciences at New York University. "Longer simulated or observed datasets aren't going to work. We need to think about smarter ways to generate data," said Weare. "In this case, that means answering the question 'where should I place my training data to achieve better performance on extremes?' Fortunately, we think AI weather models themselves, when paired with the right mathematical tools, can help answer this question." As AI becomes a bigger part of how we forecast and prepare for extreme weather, knowing its limits is key. The technology is improving fast. But until it can truly grasp the physics of our atmosphere, it won't be able to predict everything. That's not a reason to give up on AI forecasts - it's just motivation to make them even better. The full study was published in the journal Proceedings of the National Academy of Sciences. -- - Like what you read? Subscribe to our newsletter for engaging articles, exclusive content, and the latest updates.

Microsoft AI weather forecast faster, cheaper, truer: Study

Microsoft has developed an artificial intelligence (AI) model that beats current forecasting methods in tracking air quality, weather patterns, and climate-addled tropical storms, according to findings published Wednesday. Dubbed Aurora, the new system -- which has not been commercialized -- generated 10-day weather forecasts and predicted hurricane trajectories more accurately and faster than traditional forecasting, and at lower costs, researchers reported in the journal Nature. "For the first time, an AI system can outperform all operational centers for hurricane forecasting," said senior author Paris Perdikaris, an associate professor of mechanical engineering at the University of Pennsylvania. Trained only on historical data, Aurora was able to correctly forecast all hurricanes in 2023 more accurately than operational forecasting centers, such as the US National Hurricane Center. Traditional weather predicting models are designed on first physical principles -- such as conservation of mass, momentum and energy -- and require massive computer power. The computational costs of Aurora were several hundred times lower, the study said. The experimental results follow on the heels of the Pangu-Weather AI model developed and unveiled by Chinese tech giant Huawei in 2023, and could herald a paradigm shift in how the world's major weather agencies forecast weather and potentially deadly extreme events exacerbated by global warming. 'Holy grail' "I believe that we're at the beginning of a transformation age in air system science," Perdikaris said in a video presentation distributed by Nature. "In the next five to 10 years the holy grail is how to build systems that can directly work with observations from remote sensing sources like satellites and weather stations to generate forecasts at high resolution anywhere we like." According to its designers, Aurora is the first AI model to consistently outperform seven forecasting centers in predicting the five-day trajectory of devastating cyclones. In its simulation, for example, Aurora correctly forecast four days in advance where and when Doksuri -- the most costly typhoon ever recorded in the Pacific -- would hit the Philippines. Official forecasts at the time, in 2023, had it heading north of Taiwan. Microsoft's AI model also outperformed the European Centre for Medium-Range Weather Forecasts (ECMWF) model in 92% of cases for 10-day global forecasts, on a scale of approximately 10 square kilometers (3.86 square miles). The ECMWF, which provides forecasts for 35 European countries, is considered the global benchmark for weather accuracy. In December, Google announced that its GenCast model had surpassed the European centre's accuracy in more than 97% of the 1,320 climate disasters recorded in 2019. These promising performances -- all experimental and based on observed events -- are being closely scrutinized by weather agencies. Several, including Meteo-France, are developing their own AI learning models alongside traditional digital models. "This is something we have taken very seriously," Florence Rabier, Director General of the ECMWF, told AFP. Their first "learning model", made available to member states in February, is "about 1,000 times less expensive in terms of computing time than the traditional physical model", she added. While operating as a lower resolution (30 sq km) than Aurora, the ECMWF model is already operational.

Microsoft's Aurora AI Model Sets New Standard for Weather Forecasting | AIM

From sandstorms to cyclones, Aurora predicts the weather with precision and forecasts a broad range of environmental events Traditionally reliant on physics-based models and supercomputers, forecasting is now being accelerated and enhanced by machine learning systems that can process massive datasets and detect subtle atmospheric patterns. This shift provides faster, more accurate and more localised weather and environmental forecasting. Aurora from Microsoft is an AI foundational model that predicts the weather with precision and also forecasts a broad range of environmental events, from ocean waves to air pollution. As detailed in a recent paper published in Nature, Aurora represents a significant advancement in using AI to understand and anticipate Earth system phenomena. Aurora is a large-scale AI foundation model, trained initially on over one million hours of diverse atmospheric data, including satellite imagery, radar readings, weather station data and simulation outputs. "It's not just about weather anymore," said Megan Stanley, a senior researcher on the Aurora project. According to Microsoft's blog post, Aurora beat traditional numerical models and prior AI systems in 91% of forecasting benchmarks for medium-range forecasts, up to 14 days, at a resolution of 0.25 degrees. It also outperformed major global forecasting centres in cyclone tracking, setting a new standard by correctly predicting Typhoon Doksuri's landfall in the Philippines, four days ahead of the event, while official forecasts missed the mark. Leveraging high-performance GPUs, the model can produce forecasts in seconds, around 5,000 times faster than current supercomputer-based weather systems. In one test, it accurately predicted a massive sandstorm in Iraq 24 hours in advance, despite limited data, by using its foundational understanding of atmospheric patterns. It also demonstrated high accuracy in forecasting ocean wave heights and directions, essential for maritime safety and disaster preparedness. Moreover, MSN Weather is already integrating Aurora's capabilities to provide users with more accurate hourly forecasts and expanded weather parameters. "There's a huge opportunity here, especially for countries underserved by traditional forecasting tools. Aurora allows for high-resolution, localised predictions with much lower operational costs," Stanley said. "If it truly is learning physics correctly, it can be adapted to different climate settings with confidence," she explained. "Aurora is the first of its kind -- but it won't be the last."

Microsoft reveals Aurora AI beating traditional weather forecasts

Aurora achieves predictions significantly faster, more precisely, and at operational costs several hundred times lower than traditional models. Microsoft's artificial intelligence model, Aurora, is set to revolutionize weather forecasting and earth system predictions. Developed by Microsoft and presented in a recent publication in the journal Nature, Aurora has been trained on over a million hours of geophysical data obtained from satellites, radars, weather stations, simulations, and forecasts. The extensive training enables Aurora to generate high-resolution weather predictions with unprecedented accuracy and efficiency. Aurora adopts a fundamentally different approach from traditional weather forecasting models. Instead of relying on systems built over decades, it learns patterns directly from data by identifying complex relationships in historical data of the Earth system. The method allows Aurora to make predictions that could lead to more reliable forecasts of extreme phenomena exacerbated by global warming. In 2023, Aurora outperformed all operational forecasting centers, including the U.S. National Hurricane Center, by correctly predicting the formation and direction of all hurricanes in the United States more accurately than any meteorological center. It achieved better results than seven prediction centers in 100% of the cases measured for five-day cyclone trajectories. Additionally, Aurora demonstrated the ability to generate high-resolution weather forecasts in seconds, including ten-day forecasts, and track hurricane trajectories with unprecedented accuracy and speed. "Aurora represents a significant innovation in environmental system forecasting," said Paris Perdikaris, research director at Microsoft Research AI for Science. "It is the first artificial intelligence model that functions as a single fundamental model capable of adapting to different applications, from high-resolution weather forecasting and air quality predictions to monitoring tropical cyclones and ocean waves." One of the key advantages of Aurora is its efficiency. It operates at operational costs several hundred times lower than traditional models, making environmental forecasts accessible to broader communities worldwide. Current forecasting systems rely on supercomputers and specialized teams for maintenance, which makes them inaccessible to many communities worldwide. Aurora's approach achieves high accuracy with thousands of times lower computational cost. Aurora surpassed existing models in air quality, ocean wave predictions, tropical cyclone tracking, and high-resolution weather forecasting. For high-resolution weather conditions, Aurora surpassed the performance of the leading numerical weather model IFS HRES in 92% of targets at 0.1° resolution, showing better performance in extreme events. For air quality prediction, Aurora reached or exceeded the Copernicus Atmosphere Monitoring Service in 74% of targets while being approximately 50,000 times faster. "Aurora outperforms all operational centers dedicated to hurricane forecasting," Perdikaris added. "For the first time, an AI system can surpass all operational centers in hurricane prediction." He also noted, "My team is extending this vision beyond Earth sciences to various applications in science and engineering, creating artificial intelligence systems that can not only predict but also help us understand complex physical phenomena across multiple disciplines." The development of Aurora was achieved in a remarkably short period. The experiments necessary to train Aurora lasted between four and eight weeks, while years are needed to develop benchmark models. The authors noted that the rapid progress was only possible thanks to the enormous amount of data generated by decades of research with classical numerical methods. Aurora serves as a base model for the Earth system and could be adapted for other uses beyond weather prediction, including non-meteorological applications such as agricultural productivity or pollination patterns. The model can be adjusted for any desired prediction task and offers superior results to current ones at a much lower cost. Looking ahead, researchers contemplate that Aurora could operate directly with observational data. "In the next five to ten years, the 'holy grail' will be to develop systems that can work directly with observations from remote sensing sources, such as satellites and meteorological stations, to generate high-resolution forecasts wherever needed," Perdikaris stated. "We are at the beginning of a new era in atmospheric science." Aurora's success highlights a paradigm shift in weather prediction, demonstrating the potential of AI models in forecasting Earth's systems and weather phenomena. This includes providing timely warnings about extreme events, such as hurricanes, floods, and wildfires. Aurora enables better early warnings and mitigation of natural disasters, serving as an integral tool for such warnings. The researchers emphasize that Aurora could evolve in directions. "The model can be easily scaled to generate ensembles of predictions, something key in high-uncertainty situations, such as long-term forecasts or very localized phenomena," they state. They also note that the performance ceiling of the system has not yet been reached. "Our results suggest that it is possible to further improve accuracy if trained with more diverse data or if the model size is increased." Assisted by a news-analysis system.