State-of-the-art stochastic data assimilation methods for high-dimensional non-Gaussian problems
References
- Ades
, M.and van Leeuwen, P. J.. An exploration of the equivalent weights particle filter. Q. J. R. Meteorol. Soc., 820– 840. - Anderson
, J.. A local least squares framework for ensemble filtering. Mon. Wea. Rev., 634– 642. - Anderson
, J. L.. An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 2884– 2903. - Anderson
, J. L.. Exploring the need for localization in ensemble data assimilation using a hierarchical ensemble filter. Physica D, 99– 111. - Anderson
, J. L.. Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus, 72– 83. - Anderson
, J. L.. A non-Gaussian ensemble filter update for data assimilation. Mon. Wea. Rev.( 11), 4186– 4198. - Anderson
, J. L.. Localization and sampling error correction in ensemble Kalman filter data assimilation. Mon. Wea. Rev., 2359– 2371. - Anderson
, J. L.and Anderson, S. L.. A Monte Carlo implementation of the non-linear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 2741– 2758. - Bannister
, R. N.. A review of operational methods of variational and ensemble-variational data assimilation. Q. J. R. Meteorol. Soc., 607– 633. - Bengtsson
, T., Snyder, C.and Nychka, D.. Toward a nonlinear ensemble filter for high-dimensional systems. J. Geophys. Res., 8775– 8785. - Bengtsson
, T., Bickel, P.and Li, B.. Curse-of-dimensionality revisited: collapse of the particle filter in very large scale systems. IMS Collections: Prob. Stat. Essays Honor David A. Freedman, 316– 334. - Bishop
, C. H.and Hodyss, D.. Ensemble covariances adaptively localized with ECO-RAP. Part 2: a strategy for the atmosphere. Tellus, 97– 111. - Bishop
, C. H., Etherton, B. J.and Majumdar, S. J.. Adaptive sampling with the ensemble transform Kalman filter. Part I: theoretical aspects. Mon. Wea. Rev., 420– 436. - Blockley
, E. W., Martin, M. J., McLaren, A. J., Ryan, A. G., Waters, A., and co-authors. . Recent development of the Met Office operational ocean forecasting system: an overview and assessment of the new global FOAM forecasts. Geosci. Mod. Dev., 2613– 2638. - Bocquet
, M.. Ensemble Kalman filtering without the intrinsic need for inflation. Nonl. Proc. Geophy., 735– 750. - Bocquet
, M., Pires, C. A.and Wu, L.. Beyond Gaussian statistical modelling in geophysical data assimilation. Mon. Wea. Rev., 2997– 3023. - Bolić
, M., Djurić, P. M.and Hong, S.. New resampling algorithms for particle filters. Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP ’03). 2003 IEEE International Conference, Vol. 2, IEEE, pp. II--589– 592. - Brankart
, J.-M., Testut, C.-E., Brasseur, P.and Verron, J.. Implementation of a multivariate data assimilation scheme for isopycnic coordinate ocean models: application to a 1993–1996 hindcast of the North Atlantic ocean circulation. J. Geophys. Res.( C3), 3074. - Browne
, P. A.and Wilson, S.. A simple method for integrating a complex model into an ensemble data assimilation system using MPI. Env. Modell. Software, 122– 128. - Brusdal
, K., Brankart, J. M., Halberstadt, G., Evensen, G., Brasseur, P., and co-authors. . A demonstration of ensemble based assimilation methods with a layered OGCM from the perspective of operational ocean forecasting systems. J. Mar. Syst., 253– 289. - Buehner
, M.and Charron, M.. Spectral and spatial localization of background-error correlations for data assimilation. Q. J. R. Meteorol. Soc., 615– 630. - Burgers
, G., van Leeuwen, P. J.and Evensen, G.. Analysis scheme in the ensemble Kalman filter. Mon. Wea. Rev.( 6), 1719– 1724. - Campbell
, W. F., Bishop, C. H.and Hodyss, D.. Vertical covariance localization for satellite radiances in ensemble Kalman filters. Mon. Wea. Rev., 282– 290. - Chorin
, A. J.and Tu, X.. Implicit sampling for particle filters. PNAS, 17249– 17254. - Chorin
, A. J., Morzfeld, M.and Tu, X.. Interpolation and iteration for nonlinear filters. Commun. Appl. Math. Comput. Sci., 221– 240. - Chustagulprom
, N., Reich, S.and Reinhardt, M.. A hybrid ensemble transform filter for nonlinear and spatially extended dynamical systems. SIAM/ASA J. Uncert. Quant., 552– 591. - Cohn
, S. E., Da Silva, A., Guo, J., Sienkiewicz, M.and Lamich, D.. Assessing the effects of data selection with the DAO physical-space statistical analysis system. Mon. Wea. Rev., 2913– 2926. - Desroziers
, G., Berre, L., Chapnik, B.and Poli, P.. Diagnosis of observation, background and analysis-error statistics in observation space. Q. J. R. Meteorol. Soc., 3385– 3396. - Doucet
, A., Godsill, S.and Andrieu, C.. On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput., 197– 208. - Doucet
, A., de Freitas, N., Gordon, N.. Sequential Monte-Carlo Methods in Practice. Springer-Verlag, New York. - Evensen
, G.. Open boundary conditions for the extended Kalman filter with a quasi-geostrophic ocean model. J. Geophys. Res.( C9), 16529– 16546. - Evensen
, G.. Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte-Carlo methods to forecast error statistics. J. Geophys. Res., 10143– 10162. - Evensen
, G.. The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dyn., 343– 367. - Flowerdew
, J.. Towards a theory of optimal localisation. Tellus A, 25257. - Frei
, M.and Künsch, H. R.. Bridging the ensemble Kalman and particle filters. Biometrica, 781– 800. - Gaspari
, G.and Cohn, S.. Construction of correlation functions in two and three dimensions. Q. J. R. Meteorol. Soc., 723– 757. - Golub
, G. H.and Van Loan, C. F.. Matrix computations, 3rd ed., Baltimore and London. - Gordon
, N. J., Salmond, D. J.and Smith, A. F. M.. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEEE Proc., 107– 113. - Greybush
, S. J., Kalnay, E., Miyoshi, T., Ide, K.and Hunt, B. R.. Balance and ensemble Kalman filter localization techniques. Mon. Wea. Rev., 511– 522. - Gropp
, W., Lusk, E.and Skjellum, A.. Using MPI: Portable Parallel Programming with the Message-Passing Interface, Cambridge, Massachusetts. - Hamill
, T. M.. Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 550– 560. - Hamill
, T. M.. Ensemble-based atmospheric data assimilation . In: Predictability of Weather and Climate(eds. T.and R.). Cambridge University Press, New York, pp. 124– 156. chapter 6. - Hamill
, T. M., Whitaker, J. S.and Snyder, C.. Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 2776– 1790. - Hoteit
, I., Pham, D.-T., Triantafyllou, G.and Korres, G.. A new approximate solution of the optimal nonlinear filter for data assimilation in meteorology and oceanography. Mon. Wea. Rev., 317– 334. - Houtekamer
, P. L.and Mitchell, H. L.. Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 796– 811. - Houtekamer
, P. L.and Mitchell, H. L.. A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 123– 137. - Houtekamer
, P. L.and Zhang, F.. Review of the ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 4489– 4532. - Hunt
, B. R., Kostelich, E. J.and Szunyogh, I.. Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter. Physica D, 112– 126. - Ide
, K., Courtier, P., Ghil, M.and Lorenc, A. C.. Unified notation for data assimilation: pperational, l sequential and variational. J. Meteor. Soc. Jpn., 181– 189. - Janjić
, T., Nerger, L., Albertella, A., Schroeter, J.and Skachko, S.. On domain localization in ensemble-based Kalman filter algorithms. Mon. Wea. Rev., 2046– 2060. - Jazwinski
, A. H.. Stochastic Processes and Filtering Theory, New York. - Kalman
, R. E.. A new approach to linear filtering and prediction problems. Trans. AMSE --J. Basic Eng.( D), 35– 45. - Kepert
, J. D.. Localisation, balance and choice of analysis variable in an ensemble Kalman filter. Q. J. R. Meteorol. Soc.( 642), 1157– 1176. - Kepert
, J. D.. Covariance localisation and balance in an ensemble Kalman filter. Q. J. R. Meteorol. Soc., 1157– 1176. - Keppenne
, C. L.and Rienecker, M. M.. Initial testing of a massively parallel ensemble Kalman filter with the Poseidon isopycnal ocean circulation model. Mon. Wea. Rev., 2951– 2965. - Keppenne
, C. L.and Rienecker, M. M.. Assimilation of temperature into an isopycnal ocean general circulation model using a parallel ensemble Kalman filter. J. Mar. Syst., 363– 380. - Kirchgessner
, P., Nerger, L.and Bunse-Gerstner, A.. On the choice of an optimal localization radius in ensemble Kalman filter methods. Mon. Wea. Rev., 2165– 2175. - Law
, K. J. H.and Stuart, A. M.. Evaluating data assimilation algorithms. Mon. Wea. Rev., 3757– 3782. - Lawson
, W. G.and Hansen, J. A.. Implications of stochastic and deterministic filters as ensemble-based data assimilation methods in varying regimes of error growth. Mon. Wea. Rev., 1966– 1989. - Lei
, J.and Bickel, P.. A moment matching ensemble filter for nonlinear non-Gaussian data assimilation. Mon. Wea. Rev., 3964– 3973. - Lei
, L.and Anderson, J.. Empirical localization of observations for serial ensemble Kalman filter data assimilation in an atmospheric general circulation model. Mon. Wea. Rev., 1835– 1851. - Lermusiaux
, P. F. J.. Adaptive modelling, adaptive data assimilation and adaptive sampling. Physica D, 172– 196. - Lermusiaux
, P. F. J.and Robinson, A. R.. Data assimilation via error subspaces statistical estimation, part I: theory and schemes. Mon. Wea. Rev., 1385– 1407. - Lermusiaux
, P. F. J., Robinson, A. R., Haley, P. J.and Leslie, W. G.. Advanced interdisciplinary data assimilation: filtering and smoothing via error subspace statistical estimation. Proceedings of “The OCEANS 2002 MTS/IEEE conference, Holland”, Vol. 230, pp. 795– 802. - Livings
, D.. Aspects of the ensemble Kalman filter [Master’s thesis], UK. - Livings
, D., Dance, S. L.and Nichols, N. K.. Unbiased ensemble square root filters. Physica D, 1021– 1028. - Lorenc
, A. C.. The potential of the ensemble Kalman filter for NWP - a comparison with 4D-Var. Q. J. R. Meteorol. Soc., 3183– 3203. - Metref
, S., Cosme, E., Snyder, C.and Brasseur, P.. A non-Gaussian analysis scheme using rank histograms for ensemble data assimilation. Nonlin. Processes Geophys., 869– 885. - Metref
, S., Cosme, E., Snyder, C.and Brasseur, P.. A non-Gaussian analysis scheme using rank histogram for ensemble data assimilation. Nonlinear Proc. Geophys., 869– 885. - Minamide
, M.and Zhang, F.. Adaptive observation error inflation for assimilating all-sky satellite radiance. Mon. Wea. Rev., 1063– 1081. - Miyoshi
, T.and Yamane, S.. Local ensemble transform Kalman filter with an AGCM at a T159/L48 resolution. Mon. Wea. Rev., 3841– 3861. - Morzfeld
, M., Tu, X., Atkins, E.and Chorin, A. J.. A random map implementation of implicit filters. J. Comput. Phys., 2049– 2066. - Morzfeld
, M., Hodyss, D.and Snyder, C.. What the collapse of the ensemble Kalman filter tells us about particle filters. Tellus A, 1– 14. - Nakano
, S., Ueno, G.and Higuchi, T.. Merging particle filter for sequential data assimilation. Nonlinear Process. Geophys., 395– 408. - Nerger
, L.. Parallel Filter Algorithms for Data Assimilation in OceanographyNumber 487 in Reports on Polar and Marine Research, Alfred Wegener Institute for Polar and Marine Research, Bremerhaven, Germany, [PhD Thesis] . University of Bremen, Germany. - Nerger
, L.. On serial observation processing on localized ensemble Kalman filters. Mon. Wea. Rev., 1554– 1567. - Nerger
, L.and Hiller, W.. Software for ensemble-based data assimilation systems - implementation strategies and scalability. Comput. Geosci., 110– 118. - Nerger
, L., Hiller, W.and Schröter, J.. PDAF -- the parallel data assimilation framework: experiences with Kalman filtering . In: Use of High Performance Computing in Meteorology: Proceedings of the Eleventh ECMWF Workshop on the Use of High Performance Computing in Meteorology, Reading, UK, 25--29 October 2004(eds. W.and G.). World Scientific, Singapore, pp. 63– 83. - Nerger
, L., Hiller, W.and Schröter, J.. A comparison of error subspace Kalman filters. Tellus, 715– 735. - Nerger
, L., Danilov, S., Hiller, W.and Schröter, J.. Using sea level data to constrain a finite-element primitive-equation ocean model with a local SEIK filter. Ocean Dyn., 634– 649. - Nerger
, L., Janjić, T., Schroeter, J.and Hiller, W.. A unification of ensemble square root filters. Mon. Wea. Rev., 2335– 2345. - Nerger
, L., Janjić, T., Schröter, J.and Hiller, W.. A regulated localization scheme for ensemble-based Kalman filters. Q. J. Roy. Meteor. Soc., 802– 812. - Ott
, E., Hunt, B. R., Szunyogh, I., Zimin, A. V., Kostelich, E. J., and co-authors. . A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 415– 428. - Patil
, D. J., Hunt, B. R., Kalnay, E., Yorke, J. A.and Ott, E.. Local low dimensionality of atmospheric dynamics. Phys. Rev. Lett.( 26), 5878– 5881. - Penny
, S.and Miyoshi, T.. A local particle filter for high-dimensional geophysical systems. Nonlinear Processes Geophys., 391– 405. - Perianez
, A., Reich, H.and Potthast, R.. Optimal localization for ensemble Kalman filter systems. J. Meteorol. Soc. Jpn., 585– 597. - Petrie
, R. E.and Dance, S. L.. Ensemble-based data assimilation and the localisation problem. Weather( 3), 65– 69. - Pham
, D. T.. Stochastic methods for sequential data assimilation in strongly nonlinear systems. Mon. Wea. Rev., 1194– 1207. - Pham
, D. T., Verron, J.and Gourdeau, L.. Singular evolutive Kalman filters for data assimilation in oceanography. C. R. Acad. Sci. Ser. II( 4), 255– 260. - Pham
, D. T., Verron, J.and Roubaud, M. C.. A singular evolutive extended Kalman filter for data assimilation in oceanography. J. Mar. Syst., 323– 340. - Poterjoy
, J.. A localized particle filter for high-dimensional nonlinear systems. Mon. Wea. Rev., 59– 76. - Poterjoy
, J.and Anderson, J. L.. Efficient sssimilation of simulated observations in a high-dimensional geophysical system using a localized particle filter. Mon. Wea. Rev., 2007– 2020. - Reich
, S.. A nonparametric ensemble transform method for Bayesian inference. SIAM Journal on Scientific Computing( 35), 2013– 2024. - Sakov
, P.and Oke, P. R.. Implications of the form of the ensemble transformation in the ensemble square root filters. Mon. Wea. Rev., 1042– 1053. - Sakov
, P., Counillon, F., Bertino, L., Lisaeter, K. A., Oke, P. R.and co-authors. . TOPAZ4: an ocean-sea ice data assimilation system for the North Atlantic and Arctic. Ocean Sci., 633– 656. - Silverman
, B. W.. Density estimation for statistics and data analysis, New Work. - Snyder
, C., Bengtsson, T., Bickel, P.and Anderson, J.. Obstacles to high-dimensional particle filtering. Mon. Wea. Rev., 4629– 4640. - Snyder
, C., Bengtsson, T.and Morzfeld, M.. Performance bounds for particle filters using the optimal proposal. Mon. Wea. Rev.( 11), 4750– 4761. - Stordal
, A. S., Karlsen, H. A., Nævdal, G., Skaug, H. J.and Valles, B.. Bridging the ensemble Kalman filter and particle filters: the adaptive Gaussian mixture filter. Comput. Geosci., 293– 305. - Testut
, C.-E., Brasseur, P., Brankart, J.-M.and Verron, J.. Assimilation of sea-surface temperature and altimetric observationsduring 1992–1993 inot an eddy permitting primitive equation model of the North Atlantic ocean. J. Mar. Sys., 291– 316. - Tippett
, M. K., Anderson, J. L., Bishop, C. H., Hamill, T. M.and Whitaker, J. S.. Ensemble square root filters. Mon. Wea. Rev., 1485– 1490. - Tödter
, J.and Ahrens, B.. A Second-Order Exact Ensemble Square Root Filter for Nonlinear Data Assimilation. Mon. Wea. Rev.( 4), 1347– 1367. - Tödter
, J., Kirchgessner, P., Nerger, L.and Ahrens, B.. Assessment of a nonlinear ensemble transform filter for high-dimensional data assimilation. Mon. Wea. Rev., 409– 427. - Tong
, X. T., Majda, A. J.and Kelly, D.. Nonlinear stability and ergodicity of ensemble based Kalman filters. Nonlinearity, 657– 691. - van Leeuwen
, P. J.. A variance-minimizing filter for nonlinear dynamics. Mon. Wea. Rev., 2071– 2084. - van Leeuwen
, P. J.. Nonlinear ensemble data assimilation for the ocean. Recent Developments in data assimilation for atmosphere and ocean, ECMWF Seminar 8--12 September 2003, Reading, United Kingdom, pp. 265– 286. - van Leeuwen
, P. J.. Particle filtering in geophysical systems. Mon. Wea. Rev., 4089– 4114. - van Leeuwen
, P. J.. Nonlinear data assimilation in Geosciences: an extremely efficient particle filter. Q. J. R. Meteorol. Soc., 1991– 1999. - van Leeuwen
, P. J.. Efficient nonlinear data-assimilation in geophysical fluid dynamics. Comput. Fluids, 52– 58. - van Leeuwen
, P. J., Cheng, Y.and Reich, S.. Nonlinear data assimilation. Frontiers in Applied Dynamical Systems: Reviews and Tutorials 2. - van Leeuwen
, P. J.and Evensen, G.. Data assimilation and inverse methods in terms of a probabilistic formulation. Mon. Wea. Rev., 2898– 2913. - Verlaan
, M.and Heemink, A. W.. Nonlinearity in data assimilation applications: a practical method for analysis. Mon. Wea. Rev.( 6), 1578– 1589. - Vossepoel
, F. C.and Van Leeuwen, P. J.. Parameter estimation using a particle method: inferring mixing coefficients from sea-level observations. Mon. Wea. Rev., 1006– 1020. - Wang
, Y., Jung, Y., Supine, T. A.and Xue, M.. A hybrid MPI-OpenMP parallel algorithm and performance analysis for an ensemble square root filter designed for multiscale observations. J. Atm. and Oce. Tech., 1382– 1397. - Whitaker
, J. S.and Hamill, T. M.. Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 1913– 1924. - Wikle
, C. K.and Berliner, L. M.. A Bayesian tutorial for data assimilation. Physica D, 230( 1): 1– 16. - Xiong
, X., Navon, I. M.and Uzunoglu, B.. A note on the particle filter with posterior Gaussian resampling. Tellus A( 4), 456– 460. - Zhen
, Y.and Zhang, F.. A probabilistic approach to adaptive covariance localization for serial ensemble square root filters. Mon. Wea. Rev., 4499– 4518. - Zhu
, M., van Leeuwen, P. J.and Amezcua, J.. Implicit equal-weights particle filter. Q. J. R. Meteorol. Soc. page personal communication. - Zupanski
, M.. Maximum likelihood ensemble filter: Theoretical aspects. Mon. Wea. Rev., 1710– 1726. - Zupanski
, M., Michael, N. I.and Zupanski, D.. The maximum likelihood ensemble filter as a non-differentiable minimization algorithm. Q. J. R. Meteorol. Soc., 1039– 1050.
DOI: https://doi.org/10.1080/16000870.2018.1445364 | Journal eISSN: 3035-9554
Language: English
Page range: 1445364 - 1445364
Submitted on: Jul 5, 2017
Accepted on: Feb 19, 2018
Published on: Jan 1, 2018
Published by: Stockholm University Press
In partnership with: Paradigm Publishing Services
© 2018 Sanita Vetra-Carvalho, Peter Jan Van Leeuwen, Lars Nerger, Alexander Barth, M. Umer Altaf, Pierre Brasseur, Paul Kirchgessner, Jean-Marie Beckers, published by Stockholm University Press
This work is licensed under the Creative Commons Attribution 4.0 License.