Extremes & hazards

Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán

Andrew J. Charlton-Perez, Helen F. Dacre, Simon Driscoll, Suzanne L. Gray, Ben Harvey, Natalie J. Harvey, Kieran M. R. Hunt, Robert W. Lee, Ranjini Swaminathan, Remy Vandaele, Ambrogio Volonté

npj Climate and Atmospheric Science · 05 December 2023 · University of Reading

PaperPDF

In one sentence. Case study of Storm Ciarán finds FourCastNet, Pangu, and GraphCast capture synoptic structure but underestimate peak winds and miss fine warning-relevant features.

Abstract

There has been huge recent interest in the potential of making operational weather forecasts using machine learning techniques. As they become a part of the weather forecasting toolbox, there is a pressing need to understand how well current machine learning models can simulate high-impact weather events. We compare forecasts of Storm Ciarán, a European windstorm that caused sixteen deaths and extensive damage in Northern Europe, made by machine learning and numerical weather prediction models. The four machine learning models considered (FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2) produce forecasts that accurately capture the synoptic-scale structure of the cyclone including the position of the cloud head, shape of the warm sector and location of warm conveyor belt jet, and the large-scale dynamical drivers important for the rapid storm development such as the position of the storm relative to the upper-level jet exit. However, their ability to resolve the more detailed structures important for issuing weather warnings is more mixed. All of the machine learning models underestimate the peak amplitude of winds associated with the storm, only some machine learning models resolve the warm core seclusion and none of the machine learning models capture the sharp bent-back warm frontal gradient. Our study shows there is a great deal about the performance and properties of machine learning weather forecasts that can be derived from case studies of high-impact weather events such as Storm Ciarán.

Labels

Topic

Extremes & hazards, Global medium-range NWP, Benchmarks, datasets & evaluation

Type

Evaluation or intercomparison of existing models

Method

Transformer / ViT, Graph neural network

Time scale

3–15 days

Domain

Regional

Data

ERA5 / reanalysis, NWP analysis or forecasts (IFS, GFS, ...)

Status

Research

Identifiers

arXiv:2312.02658 DOI:10.1038/s41612-024-00638-w OpenAlex:W4395010148

Added

2026-10-22

Labels and summary are generated by Claude and reviewed weekly. Wrong? Edit this entry.