The story so far: On Dec. 4, Google DeepMind unveiled GenCast, an artificial intelligence (AI) model that the company said can predict the weather better than most existing tools and more days in advance. Details of the model were published in a peer-reviewed article in the journal Nature.

How do we forecast the weather?

“Forecasts … are produced by conducting multiple numerical simulations of the atmosphere,” Vassili Kitsios, senior scientist at the Commonwealth Scientific and Industrial Research Organization of Australia, wrote earlier this month. “Each simulation assumes a slightly different estimate of the current weather. This is because we don’t know exactly what the weather is like around the world right now. … By solving equations that describe the fundamental physical laws of nature, the simulations predict what will happen in the atmosphere.”

This process is called numerical weather forecasting (NWP). The best NWP forecasts require the use of powerful supercomputers as well as high-quality data about the weather at a specific location. Even then, NWPs can only predict the weather about a week in advance.

Ensemble forecasting came onto the scene in the 1990s. Scientists use an NWP model to create multiple forecasts at a specific point in time and with different starting conditions. This collection of forecasts is called an ensemble and indicates the range of meteorological possibilities.

How does GenCast work?

Google’s GenCast also uses ensemble predictions, but the options in the ensemble come from an AI model and not an NWP. Engineers at Google trained this AI model using 40 years of reanalysis data from 1979 to 2019. According to the European Center for Medium-Range Weather Forecasts (ECMWF), “Reanalysis data currently provide the most comprehensive picture of past weather and climate. They are a mix of observations with previous short-term weather forecasts, repeated using modern weather forecast models.”

GenCast was trained in two steps: Step I in 3.5 days and Step II in 1.5 days, each with 32 TPU v5 instances. “TPU” is an abbreviation for “Tensor Processing Unit,” an integrated circuit developed by Google for running machine learning models and sold through Google Cloud. In December 2023, Google Cloud launched a TPU called v5p: it contains 8,960 interconnected chips with a bandwidth of 4,800 Gbps/chip and costs $4.2 per chip-hour on demand.

Just as ChatGPT is good at identifying the next word in an unfinished sentence, GenCast is good at guessing what the weather will be like in the next moment given the weather up to a certain point in time. According to the Nature GenCast, in one study, had “greater capabilities than ENS on 97.2% of the 1,320 targets we assessed and is better at predicting extreme weather, tropical cyclone tracks and wind power production.” ENS refers to the ensemble forecasts produced by ECMWF known as considered one of the best in NWP.

Google also said that GenCast was more accurate than ENS in predicting the weather more than 36 hours in advance for 99.8% of 1,320 targets.

How does GenCast work?

The AI ​​model described in the paper had a neural network with 41,162 nodes and 2.4 lakh edges. Each node is a point in the network where some input data is accepted and manipulated and an output is generated as input to another node. An edge is a connection between nodes.

For information about how this setup processes data, see the diagram below. Caption: The globes below show a weather forecast at four points in time, one after the other. Each forecast is generated by combining existing weather data with a noisy input. GenCast’s challenge is to extract a weather forecast for the next moment from the noisy input – the globes above. To do this, the model performs the combination of a refinement (green box), creates a less noisy prediction, then combines it with the input data again, performs a second refinement, then combines the new output with the input data, performs a third refinement, etc. , until 30 refinements are completed. The final denoised output, called X1, is the final weather forecast for the next point in time. To predict the weather for the next moment, the model first accepts X1 as input and starts again with a noisy input. The green boxes show the neural networks.

The ability to denoise a noisy input is a common feature of a diffusion-type AI model, as is the case with GenCast. Other well-known apps that use diffusion models include OpenAI’s text-to-video model Sora and Stability AI’s text-to-image model Stable Diffusion, both of which are also examples of generative AI.

GenCast produces at least 50 forecasts simultaneously for the ensemble, and Google has stated that each forecast can be produced in parallel. In total, the ensemble contains forecasts for 15 days each with a spatial resolution of 0.25° x 0.25° (latitude-longitude) and a temporal resolution of 12 hours. The researchers found that this entire process took eight minutes for GenCast to run on a TPU v5 unit, much shorter than the several hours it takes supercomputers for NWP.

Will GenCast replace NWP?

GenCast’s forecasts are probabilistic rather than deterministic, i.e. “on December 25th there will be a 25% chance of rain in Chennai” rather than “on December 25th there will be 5mm of rain in Chennai”. Current NWP models and their ensembles are deterministic. Experts say probabilistic weather forecasts can better reveal the possibility of extreme weather events.

“We should make greater use of these probabilistic forecasts for extreme events instead of relying on quantitative forecasts. “Probabilistic forecasts provide more lead time that can be used for better preparation,” wrote former Indian government secretary Madhavan Rajeevan The Hindu in December 2023.

Although GenCast’s performance suggests that AI weather models will soon surpass the capabilities of NWP models, both NWP and GenCast are based on more fundamental weather data that is still collected using the laws of physics. Experts say it remains important to understand weather through these laws because weather in many parts of the world is changing rapidly, in ways that historical weather conditions cannot prepare us for.

GenCast itself requires more reanalysis data to train itself. As Google said in a public statement, “We greatly value our partnerships with weather agencies and will continue to work with them to develop AI-based methods that improve their forecasts.” In the meantime, traditional models remain essential to this work. For one thing, they provide the training data and baseline weather conditions that models like GenCast need.” The code to run GenCast is available on GitHub.

DeepMind has also been working on a model called GraphCast to develop “deterministic medium-term forecasts.” Google Research has developed a model called NeuralGCM that combines AI and NWP models to produce deterministic forecasts, as well as at least two other models to predict extreme flooding and quantify forecast uncertainty. Elsewhere, Huawei’s Pangu Weather model can predict the weather one week at a time with an accuracy comparable to NWP, but much faster. Nvidia’s FourCastNet model can already outperform a state-of-the-art NWP facility at ECMWF in predicting extreme rainfall in less than two seconds.