AI Weather Model Revolutionizes Energy Trading and Forecasting

A New AI Weather Model Is Already Changing How Energy Is Traded

At midnight every day in Bologna, Italy, rows of supercomputers inside a former tobacco factory start churning through millions of measurements to predict how the Earth's weather will change. Six hours later, energy traders all over Europe rise and refresh their browsers to get the most updated outlook. Those mainframe-generated forecasts are often the biggest factor helping them make money by knowing where and when to move energy around the power grid -- but a new model that runs on artificial intelligence is threatening to make them obsolete.

A new AI weather model is already changing how energy is traded

These advancements are crucial for managing market volatility and making informed weather-related decisions.At midnight every day in Bologna, Italy, rows of supercomputers inside a former tobacco factory start churning through millions of measurements to predict how the Earth's weather will change. Six hours later, energy traders all over Europe rise and refresh their browsers to get the most updated outlook. Those mainframe-generated forecasts are often the biggest factor helping them make money by knowing where and when to move energy around the power grid -- but a new model that runs on artificial intelligence is threatening to make them obsolete. Unlike standard weather simulations, which only crunch information from satellites, sensors and the like, the AI model from Europe's intergovernmental forecasting center also feasts on historical data. Before its release late last month, the center tested the new method against its conventional model produced in Bologna and found the AI more accurately predicted temperature, precipitation, wind and tropical cyclones, all with less computing energy. The model is poised to help traders in Europe and around the world make quicker moves in power and natural gas markets convulsed by extreme weather, geopolitics and fluctuations in renewable sources. It's a technology that could help minimize energy gluts and shortfalls in the world's fastest-warming continent, as well as provide information key to deciding where wind and solar farms should be built. "We can update our information set more often than we are used to" because of the advances in AI weather models, said Daniel Borup, chief executive officer of Danish trading firm InCommodities A/S. "That obviously leads to improvements in our predictions. It allows us to improve our job and distribute energy better." Like its traditional outlook, the European Centre for Medium-Range Weather Forecasts' new system -- the first AI model released by a major prediction center -- estimates temperatures, wind speeds and solar power two weeks in advance. But its improved accuracy means companies and policymakers can move faster on critical weather-related decisions, from canceling rail service to routing ships around storms and dispatching trucks to spread sand on icy roads, according to the center.That degree of forecasting prowess will could prove essential to managing market volatility. Earlier this month, robust generation from solar parks in Germany pushed power prices in several countries below zero. That was a reversal from earlier in the year, when a stretch of cloudy and windless weather known as a Dunkelflaute curbed renewable output and sent German electricity prices soaring. The upgrade is a radical shift away from the standard approach of using supercomputers to crunch millions of measurements to recreate a snapshot of the atmosphere's physics, and then fast-forwarding the model to predict how the weather will change. Climate and weather datasets were already structured perfectly for AI and could use machine learning techniques developed for other scientific research approaches, Florian Pappenberger, the European center's deputy director-general and lead forecaster. The forecasting center has been experimenting with machine learning techniques in earnest since 2018, but researchers' confidence in the technology's ability to make accurate weather predictions reached a critical mass in 2022, he said. "Weather and climate is a Big Data problem," he said. "We have huge amounts of data -- humongous amounts -- so it's a perfect match" for the center's new model, he added. Once the data are digested, the AI model can generate a raw forecast in three minutes versus the 30 minutes it takes the center's supercomputers to generate a conventional outlook, which typically takes six hours to finalize. While the AI model is created by the European intergovernmental group and is closely watched by traders across the continent, the forecast itself is global and used by industries and meteorologists around the world, including in the US. Twenty-odd minutes might not seem like much, but it can help companies, trading firms and government officials respond more quickly to shifts in weather -- for example, by prompting grid operators to call for more electricity ahead of a cold snap. The two-week period the forecast covers is key for traders as they make bets on how energy demand will impact prices, said Dan Harding, a meteorologist who leads research and development at the European weather analytics firm MetDesk. "It's what the markets move on most," he said. The European center's AI forecast was honed through collaborations with university scientists and research on experimental weather models developed by tech companies like Nvidia Corp., Huawei Technologies Co., Microsoft Corp. and Alphabet Inc.'s Google. Those results convinced Christian Bach, InCommodities head quant and weather intelligence lead, that AI models including the center's were outpacing conventional forecasting methods. "It was really the first indication that machine learning is going to be a big thing," he said. Another way to illustrate AI's rapid ascent in meteorology is through the European forecasting center's plan for improving its outlooks over the next decade. AI was a small piece of the puzzle in 2020, but the center's new 10-year road map predicts AI will improve nearly every aspect of its forecasting ability. The rapid rise of AI and machine-learning in meteorology has been "faster than expected," according to the plan. Data-driven models are ''already at a maturity where we can confidently expect them to play an important part in operational prediction." AI's ability to create forecasts quickly with fewer computing resources make it good fit for energy traders hungry to get more weather information more often, said Rob Hutchinson, a meteorologist who leads the energy and utilities team at the Swiss weather analytics firm Meteomatics AG. Testing from Meteomatics shows the European center's AI forecasts appear to be more accurate than conventional versions when it comes to estimating temperature around the five days ahead of time, he added. "Speed is one thing, but there are certain parameters and time horizons where there does appear to be some additional accuracy as well," he said. But Hutchinson and other meteorologists don't expect AI models to replace conventional forecasts anytime soon. The European center is releasing its AI models alongside its conventional forecasts and envisions further adoption of a hybrid system that uses the most accurate and useful elements from both approaches. "It's quite a lot of marketing hype, sticking AI in front of it and pretending it's better," Hutchinson said, "But it's much more nuanced than that. We have to let the numbers speak for themselves." That's partly because, despite its rapid improvement, AI models are still less accurate than conventional forecasts for cloud cover, dust and some weather extremes, Pappenberger said. The current AI model is also only used for a type of forecast that generates one prediction at a time. The next version of the technology will apply to a kind of forecast known as an ensemble that generates 50 predictions each time it's run. The next step, Pappenberger said, will be connecting AI models more directly with data from satellites and weather stations. In the future, AI could also tap new streams of weather information collected by non-standard sources, including cars, appliances, phones and other devices. "AI weather models have the potential to increase the frequency of forecast updates and improve performance," said Edoardo Simioni, head of trading and flexibility at Copenhagen-based electricity supplier Reel ApS. The advances in technology, he added, are "ultimately good for the market."