Auto-differentiable Ensemble Kalman Filters
In one sentence. Auto-differentiable ensemble Kalman filters jointly learn surrogate dynamics and assimilate states, outperforming EM and particle-filter baselines on Lorenz-96.
Abstract
Data assimilation is concerned with sequentially estimating a temporally-evolving state. This task, which arises in a wide range of scientific and engineering applications, is particularly challenging when the state is high-dimensional and the state-space dynamics are unknown. This paper introduces a machine learning framework for learning dynamical systems in data assimilation. Our auto-differentiable ensemble Kalman filters (AD-EnKFs) blend ensemble Kalman filters for state recovery with machine learning tools for learning the dynamics. In doing so, AD-EnKFs leverage the ability of ensemble Kalman filters to scale to high-dimensional states and the power of automatic differentiation to train high-dimensional surrogate models for the dynamics. Numerical results using the Lorenz-96 model show that AD-EnKFs outperform existing methods that use expectation-maximization or particle filters to merge data assimilation and machine learning. In addition, AD-EnKFs are easy to implement and require minimal tuning.
Labels
- Topic
- Data assimilation
- Type
- New model or method
- Method
- Differentiable physical model, Other
- Data
- Other
- Status
- Research
- Identifiers
- arXiv:2107.07687
- Added
- 2026-10-22
Labels and summary are generated by Claude and reviewed weekly. Wrong? Edit this entry.