ML4WX

Data assimilation

Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales

Peter Manshausen, Yair Cohen, Peter Harrington, Jaideep Pathak, Mike Pritchard, Piyush Garg, Morteza Mardani, Karthik Kashinath, Simon Byrne, Noah Brenowitz

Journal of Advances in Modeling Earth Systems · · NVIDIA, University of Oxford

PaperPDF

In one sentence. Score-based diffusion data assimilation at km scale assimilates 40 sparse weather stations, lowering held-out station RMSE by 10% versus HRRR.

Abstract

Data assimilation of observational data into full atmospheric states is essential for weather forecast model initialization. Recently, methods for deep generative data assimilation have been proposed which allow for using new input data without retraining the model. They could also dramatically accelerate the costly data assimilation process used in operational regional weather models. Here, in a central US testbed, we demonstrate the viability of score-based data assimilation in the context of realistically complex km-scale weather. We train an unconditional diffusion model to generate snapshots of a state-of-the-art km-scale analysis product, the High Resolution Rapid Refresh. Then, using score-based data assimilation to incorporate sparse weather station data, the model produces maps of precipitation and surface winds. The generated fields display physically plausible structures, such as gust fronts, and sensitivity tests confirm learnt physics through multivariate relationships. Preliminary skill analysis shows the approach already outperforms a naive baseline of the High-Resolution Rapid Refresh system itself. By incorporating observations from 40 weather stations, 10% lower RMSEs on left-out stations are attained. Despite some lingering imperfections such as insufficiently disperse ensemble DA estimates, we find the results overall an encouraging proof of concept, and the first at km-scale. It is a ripe time to explore extensions that combine increasingly ambitious regional state generators with an increasing set of in situ, ground-based, and satellite remote sensing data streams.

Labels

Topic
Data assimilation, Regional & km-scale models
Type
New model or method
Method
Diffusion / flow matching / score-based
Time scale
0–6 h, 6 h–3 days
Domain
Regional
Data
NWP analysis or forecasts (IFS, GFS, ...), Station / in-situ
Status
Research
Identifiers
arXiv:2406.16947 DOI:10.1029/2024ms004505 OpenAlex:W4415620306
Added
2026-10-22

Labels and summary are generated by Claude and reviewed weekly. Wrong? Edit this entry.