Benchmarks, datasets & evaluation★ Highlight
Verification of AI–based environmental forecasting systems: What can we do, what do we need to do, and what are the challenges?
Why it matters. Authored by leading centres, it sets out a vision and research agenda for verifying AI weather forecasts, which is useful for evaluation practice.
In one sentence. Position paper argues AI weather forecasts need adapted verification methods, identifying where and when they are useful beyond aggregate MSE skill.
Abstract
Several institutions have released global medium–range meteorological forecasting models based on methods from machine learning, with training data provided by various reanalysis experiments. A proper and in-depth assessment of these models and the quality of their forecasts has yet to be carried out. Although in terms of simple and overall measures of skill such as mean square errors, AI-based forecasts clearly show very promising skill, we are just beginning to understand where and when these forecasts are useful and when they are not. Furthermore, while verification of meteorological forecasts has been subject to extensive (and still ongoing) research with a well established core methodology, it is not clear to what extent this methodology needs to be adapted or modified for AI–based models. Our paper aims to provide a vision on the verification of AI–based weather forecasts, identifying challenges, outlining important research questions, and laying the groundwork for a methodology to assess the quality of such forecasts.
Labels
- Topic
- Benchmarks, datasets & evaluation, Global medium-range NWP
- Type
- Position or perspective paper
- Time scale
- 3–15 days
- Domain
- Global
- Data
- ERA5 / reanalysis, NWP analysis or forecasts (IFS, GFS, ...)
- Status
- Research
- Identifiers
- DOI:10.1016/j.jemets.2026.100032 OpenAlex:W7134139927
- Added
- 2026-10-22
Labels and summary are generated by Claude and reviewed weekly. Wrong? Edit this entry.