Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization

Lorenzo Pacchiardi, Rilwan Adewoyin, Peter Dueben, Ritabrata Dutta

Journal of Machine Learning Research, 25(45), 2024 · 15 December 2021

In one sentence. Generative networks trained with prequential scoring rules, avoiding adversarial training, improve probabilistic calibration on chaotic models and a global weather benchmark.

Abstract

Probabilistic forecasting relies on past observations to provide a probability distribution for a future outcome, which is often evaluated against the realization using a scoring rule. Here, we perform probabilistic forecasting with generative neural networks, which parametrize distributions on high-dimensional spaces by transforming draws from a latent variable. Generative networks are typically trained in an adversarial framework. In contrast, we propose to train generative networks to minimize a predictive-sequential (or prequential) scoring rule on a recorded temporal sequence of the phenomenon of interest, which is appealing as it corresponds to the way forecasting systems are routinely evaluated. Adversarial-free minimization is possible for some scoring rules; hence, our framework avoids the cumbersome hyperparameter tuning and uncertainty underestimation due to unstable adversarial training, thus unlocking reliable use of generative networks in probabilistic forecasting. Further, we prove consistency of the minimizer of our objective with dependent data, while adversarial training assumes independence. We perform simulation studies on two chaotic dynamical models and a benchmark data set of global weather observations; for this last example, we define scoring rules for spatial data by drawing from the relevant literature. Our method outperforms state-of-the-art adversarial approaches, especially in probabilistic calibration, while requiring less hyperparameter tuning.

Labels

Labels and summary are generated by Claude and reviewed weekly. Wrong? Edit this entry.