A Stochastic Covariance Shrinkage Approach in Ensemble Transform Kalman Filtering
References
- Anderson, JL. 2001. ‘An ensemble adjustment Kalman filter for data assimilation’. Monthly Weather Review, 129(12): 2884–2903. DOI: 10.1175/1520-0493(2001)129<;2884:AEAKFF>2.0.CO;2
- Anderson, JL. 2012. ‘Localization and sampling error correction in ensemble Kalman filter data assimilation’. Monthly Weather Review, 140(7): 2359–2371. DOI: 10.1175/MWR-D-11-00013.1
- Arakawa, A. 1966. ‘Computational design for long-term numerical integration of the equations of fluid motion: Two-dimensional incompressible flow. Part I’. Journal of Computational Physics, 1(1): 119–143. DOI: 10.1016/0021-9991(66)90015-5
- Asch, M, Bocquet, M and Nodet, M. 2016. Data assimilation: methods, algorithms, and applications, SIAM. DOI: 10.1137/1.9781611974546
- Bishop, CH, Etherton, BJ and Majumdar, SJ. 2001. ‘Adaptive sampling with the ensemble transform kalman filter. part i: Theoretical aspects’. Monthly weather review, 129(3): 420–436. DOI: 10.1175/1520-0493(2001)129<;0420:ASWTET>2.0.CO;2
- Buhmann, MD. 1998. ‘Radial functions on compact support’. Proceedings of the Edinburgh Mathematical Society, 41(1): 33–46. DOI: 10.1017/S0013091500019416
- Burgers, G, Jan van Leeuwen, P and Evensen, G. 1998. ‘Analysis scheme in the ensemble kalman filter’. Monthly weather review, 126(6): 1719–1724. DOI: 10.1175/1520-0493(1998)126<;1719:ASITEK>2.0.CO;2
- Chen, Y, Wiesel, A, Eldar, YC and Hero, AO. 2010. ‘Shrinkage algorithms for mmse covariance estimation’. IEEE Transactions on Signal Processing, 58(10): 5016–5029. DOI: 10.1109/TSP.2010.2053029
- Chen, Y, Wiesel, A and Hero, AO. 2009. Shrinkage estimation of high dimensional covariance matrices. In ‘2009 IEEE International Conference on Acoustics, Speech and Signal Processing’. IEEE, 2937–2940. DOI: 10.1109/ICASSP.2009.4960239
- Chen, Y, Wiesel, A and Hero, AO. 2011. ‘Robust shrinkage estimation of high-dimensional covariance matrices’. IEEE Transactions on Signal Processing, 59(9): 4097–4107. DOI: 10.1109/TSP.2011.2138698
- Computational Science Laboratory. 2020. ‘ODE test problems’. URL:
https://github.com/ComputationalScienceLaboratory/ODE-Test-Problems . - Evensen, G. 1994. ‘Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics’. Journal of Geophysical Research: Oceans, 99(C5): 10143–10162. DOI: 10.1029/94JC00572
- Evensen, G. 2009. Data assimilation: the ensemble Kalman filter, Springer Science & Business Media. DOI: 10.1007/978-3-642-03711-5
- Flyer, N and Wright, GB. 2009. ‘A radial basis function method for the shallow water equations on a sphere’. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 465(2106): 1949–1976. DOI: 10.1098/rspa.2009.0033
- Gaspari, G and Cohn, SE. 1999. ‘Construction of correlation functions in two and three dimensions’. Quarterly Journal of the Royal Meteorological Society, 125(554): 723–757. DOI: 10.1002/qj.49712555417
- Hairer, E, Nørsett, SP and Wanner, G. 1991. Solving ordinary differential equations. 1, Nonstiff problems. Springer-Vlg. DOI: 10.1007/978-3-662-09947-6_1
- Hamill, TM. 2001. ‘Interpretation of rank histograms for verifying ensemble forecasts’. Monthly Weather Review, 129(3): 550–560. DOI: 10.1175/1520-0493(2001)129<;0550:IORHFV>2.0.CO;2
- Hamill, TM and Snyder, C. 2000. ‘A hybrid ensemble Kalman filter–3d variational analysis scheme’. Monthly Weather Review, 128(8): 2905–2919. DOI: 10.1175/1520-0493(2000)128<;2905:AHEKFV>2.0.CO;2
- Hunt, BR, Kostelich, EJ and Szunyogh, I. 2007. ‘Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter’. Physica D: Nonlinear Phenomena 230(1–2): 112–126. DOI: 10.1016/j.physd.2006.11.008
- Jaynes, ET. 2003. Probability theory: The logic of science, Cambridge university press. DOI: 10.1017/CBO9780511790423
- Jespersen, DC. 1974. ‘Arakawa’s method is a finite-element method’. Journal of computational physics, 16(4): 383–390. DOI: 10.1016/0021-9991(74)90047-3
- Kalman, RE. 1960. ‘A new approach to linear filtering and prediction problems’. Journal of basic Engineering, 82(1): 35–45. DOI: 10.1115/1.3662552
- Kalnay, E. 2003. Atmospheric modeling, data assimilation and predictability, Cambridge university press. DOI: 10.1017/CBO9780511802270
- Law, K, Stuart, A and Zygalakis, K. 2015. Data assimilation: a mathematical introduction, Vol. 62, Springer. DOI: 10.1007/978-3-319-20325-6
- Ledoit, O and Wolf, M. 2004. ‘A well conditioned estimator for large dimensional covariance matrices’. Journal of Multivariate Analysis, 88(2): 365–411. DOI: 10.1016/S0047-259X(03)00096-4
- Lorenz, EN. 1996. Predictability: A problem partly solved. In ‘Proc. Seminar on predictability’. Vol. 1.
- Macdonald, CB. 2003.
Constructing high-order Runge-Kutta methods with embedded strong-stability-preserving pairs , PhD thesis, Theses (Dept. of Mathematics)/Simon Fraser University. - Mou, C, Liu, H, Wells, DR and Iliescu, T. 2019. ‘Data-driven correction reduced order models for the quasi-geostrophic equations: A numerical investigation’. arXiv preprint.
http://arxiv.org/abs/1908.05297 . - Neta, B, Giraldo, F and Navon, I. 1997. ‘Analysis of the turkel–zwas scheme for the two-dimensional shallow water equations in spherical coordinates’. Journal of Computational Physics, 133(1): 102–112. DOI: 10.1006/jcph.1997.5657
- Nino-Ruiz, ED, Sandu, A and Deng, X. 2015.
A parallel ensemble Kalman filter implementation based on modified Cholesky decomposition . In ‘Proceedings of the 6th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems’. Vol. Supercomputing 2015 of ScalA ’15, Austin, Texas. DOI: 10.1145/2832080.2832084 - Nino-Ruiz, E and Sandu, A. 2015. ‘Ensemble Kalman filter implementations based on shrinkage covariance matrix estimation’. Ocean Dynamics, 65(11): 1423–1439. DOI: 10.1007/s10236-015-0888-9
- Nino-Ruiz, E and Sandu, A. 2017. ‘An ensemble Kalman filter implementation based on modified Cholesky decomposition for inverse covariance matrix estimation’. SIAM Journal on Scientific Computing submitted. URL:
https://arxiv.org/abs/1605.08875 . - Nino-Ruiz, E and Sandu, A. 2018. ‘Efficient parallel implementation of DDDAS inference using an ensemble Kalman filter with shrinkage covariance matrix estimation’. Cluster Computing, 1–11. URL:
https://link.springer.com/article/10.1007/s10586-017-1407-1 . DOI: 10.1007/s10586-017-1407-1 - Petersen, KB, Pedersen, MS, et al. 2008. ‘The matrix cookbook’. Technical University of Denmark, 7(15): 510.
- Petrie, R. 2008. ‘Localization in the ensemble Kalman filter’. MSc Atmosphere, Ocean and Climate University of Reading.
- Popov, AA and Sandu, A. 2019. ‘A bayesian approach to multivariate adaptive localization in ensemblebased data assimilation with time-dependent extensions’. Nonlinear Processes in Geophysics, 26(2): 109–122. DOI: 10.5194/npg-26-109-2019
- Popov, AA and Sandu, A. 2020. ‘An explicit probabilistic derivation of inflation in a scalar ensemble Kalman filter for finite step, finite ensemble convergence’.
- Reich, S and Cotter, C. 2015. Probabilistic forecasting and Bayesian data assimilation, Cambridge University Press. DOI: 10.1017/CBO9781107706804
- Roberts, S, Popov, AA and Sandu, A. 2019. ‘ODE test problems: a MATLAB suite of initial value problems’. CoRR abs/1901.04098. URL:
http://arxiv.org/abs/1901.04098 . - Ruiz, EN, Sandu, A and Anderson, J. 2014. ‘An efficient implementation of the ensemble Kalman filter based on an iterative Sherman–Morrison formula’. Statistics and Computing, 1–17. DOI: 10.1007/s11222-014-9454-4
- Sakov, P and Oke, PR. 2008a. ‘A deterministic formulation of the ensemble Kalman filter: an alternative to ensemble square root filters’. Tellus A, 60(2): 361–371. DOI: 10.1111/j.1600-0870.2007.00299.x
- Sakov, P and Oke, PR. 2008b. ‘Implications of the form of the ensemble transformation in the ensemble square root filters’. Monthly Weather Review, 136(3): 1042–1053. DOI: 10.1175/2007MWR2021.1
- San, O and Iliescu, T. 2015. ‘A stabilized proper orthogonal decomposition reduced-order model for large scale quasigeostrophic ocean circulation’. Advances in Computational Mathematics, 41(5): 1289–1319. DOI: 10.1007/s10444-015-9417-0
- Stoica, P, Li, J, Zhu, X and Guerci, JR. 2008. ‘On using a priori knowledge in space-time adaptive processing’. IEEE Transactions on Signal Processing, 56(6): 2598–2602. DOI: 10.1109/TSP.2007.914347
- Zhang, Y, Liu, N and Oliver, DS. 2010. ‘Ensemble filter methods with perturbed observations applied to nonlinear problems’. Computational Geosciences, 14(2): 249–261. DOI: 10.1007/s10596-009-9149-7
- Zhu, X, Li, J and Stoica, P. 2011. ‘Knowledge-Aided Space-Time Adaptive Processing’. IEEE Transaction on Aerospace And Electronic Systems, 47(2): 1325–1336. DOI: 10.1109/TAES.2011.5751261
- Zubair, HB. 2009.
Efficient Multigrid Methods based on Improved Coarse Grid Correction Techniques . PhD thesis, Delft University of Technology, Netherlands.
DOI: https://doi.org/10.16993/tellusa.214 | Journal eISSN: 3035-9554
Language: English
Page range: 159 - 171
Submitted on: Jul 12, 2022
Accepted on: Feb 2, 2023
Published on: Apr 11, 2023
Published by: Stockholm University Press
In partnership with: Paradigm Publishing Services
Keywords:
© 2023 Andrey A. Popov, Adrian Sandu, Elias D. Nino-Ruiz, Geir Evensen, published by Stockholm University Press
This work is licensed under the Creative Commons Attribution 4.0 License.