Using global Bayesian optimization in ensemble data assimilation: parameter estimation, tuning localization and inflation, or all of the above
References
- Abbas, M. , Ilin, A. , Solonen, A. , Hakkarainen, J. , Oja, E. and co-authors. 2016. Empirical evaluation of Bayesian optimization in parametric tuning of chaotic systems. Int. J. Uncertainty Quantif. 6 , 467–485. doi:10.1615/Int.J.UncertaintyQuantification.2016016645
- Anderson, J. 2012. Localization and sampling error correction in ensemble Kalman filter data assimilation. Mon. Weather Rev. 140 , 2359–2371. doi:10.1175/MWR-D-11-00013.1
- Anderson, J. 2016. Reducing correlation sampling error in ensemble Kalman filter data assimilation. Mon. Weather Review 144 , 913–925. doi:10.1175/MWR-D-15-0052.1
- Anderson, J. and Anderson, S. 1999. A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev. 127 , 2741–2758. doi:10.1175/1520-0493(1999)127<;2741:AMCIOT>2.0.CO;2
- Andrieu, C. , Doucet, A. and Holenstein, R. 2010. Particle Markov chain Monte Carlo methods. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 72 , 269–342. doi:10.1111/j.1467-9868.2009.00736.x
- Bocquet, M. and Sakov, P. 2013. Joint state and parameter estimation with an iterative ensemble Kalman smoother. Nonlin. Processes Geophys. 20 , 803–818. doi:10.5194/npg-20-803-2013
- Bonavita, M. , Isaksen, L. and Hólm, E. 2012. On the use of EDA background-error variances in the ECMWF 4D-Var. Q. J. R. Meteorol. Soc. 138 , 1540–1559. doi:10.1002/qj.1899
- Buehner, M. 2012. Evaluation of a spatial/spectral covariance localization approach for atmospheric data assimilation. Mon. Wea. Rev. 140 , 617–636. doi:10.1175/MWR-D-10-05052.1
- Buehner, M. and Shlyaeva, A. 2015. Scale-dependent background-error covariance localisation. Tellus A: Dyn. Meteorol. Oceanogr. 67, 28027. doi:10.3402/tellusa.v67.28027
- Byrd, R. H. , Lu, P. , Nocedal, J. and Zhu, C. 1995. A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16 , 1190–1208. doi:10.1137/0916069
- Carrassi, A. , Bocquet, M. , Bertino, L. and Evensen, G. 2018. Data assimilation in the geosciences: An overview of methods, issues, and perspectives. Wiley Interdiscip. Rev. Clim. Change 9 , e535. doi:10.1002/wcc.535
- Chopin, N. , Jacob, P. E. and Papaspiliopoulos, O. 2013. SMC2: An efficient algorithm for sequential analysis of state space models. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 75 , 397–426. doi:10.1111/j.1467-9868.2012.01046.x
- De La Chevrotière, M. and Harlim, J. 2017. A data-driven method for improving the correlation estimation in serial ensemble Kalman filters. Mon. Wea. Rev. 145 , 985–1001. doi:10.1175/MWR-D-16-0109.1
- Destouches, M. , Montmerle, T. , Michel, Y. and Ménétrier, B. 2020. Estimating optimal localization for sampled background-error covariances of hydrometeor variables. Q J. R. Meteorol. Soc. 147(734), 74–93.
- Doucet, A. , Freitas, N. d. , & Gordon, N. (Eds.). 2001. Sequential Monte Carlo Methods in Practice . New York, New York: Springer.
- El Gharamti, M. , Raeder, L. , Anderson, J. and Wang, X. 2019. Comparing adaptive prior and posterior inflation for ensemble filters using an atmospheric general circulation model. Mon. Weather Rev. 147 , 2535–2553. doi:10.1175/MWR-D-18-0389.1
- Evensen, G. 2009a. Data Assimilation: The Ensemble Kalman Filter . Springer Science & Business Media, Dordrecht, Heidelberg, London, New York.
- Evensen, G. 2009b. The ensemble Kalman filter for combined state and parameter estimation. IEEE Control Syst. 29 , 83–104. doi:10.1109/MCS.2009.932223
- Farchi, A. and Bocquet, M. 2018. Review article: Comparison of local particle filters and new implementations. Nonlinear Processes Geophys. Discuss, 25 (4), 765–807.
- Flowerdew, J. 2015. Towards a theory of optimal localisation. Tellus A: Dyn. Meteorol. Oceanogr. 67 , 25257. doi:10.3402/tellusa.v67.25257
- Flowerdew, J. 2017. Initial trials of convective-scale data assimilation with a cheaply tunable ensemble filter. Q. J. R. Meteorol. Soc. 143 , 1670–1684. doi:10.1002/qj.3038
-
Frazier, P. I.
and
Wang, J.
2016.
Bayesian optimization for materials design . In: Lookman T., Alexander F., Rajan K. (eds) Information Science for Materials Discovery and Design . Springer, vol 225. Springer, Cham. - Hamill, T. , Whitaker, J. S. , Anderson, J. and Snyder, C. 2009. Comments on “Sigma-point Kalman filter data assimilation methods for strongly nonlinear systems”. J. Atmos. Sci. 66 , 3498–3500. doi:10.1175/2009JAS3245.1
- Hamill, T. , Whitaker, J. and Snyder, C. 2001. Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev. 129 , 2776–2790. doi:10.1175/1520-0493(2001)129<;2776:DDFOBE>2.0.CO;2
- Houtekamer, P. and Mitchell, H. 2001. A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev. 129 , 123–137. doi:10.1175/1520-0493(2001)129<;0123:ASEKFF>2.0.CO;2
- Hunt, B. , Kostelich, E. and Szunyogh, I. 2007. Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Phys. D. 230 , 112–126. doi:10.1016/j.physd.2006.11.008
- Joe, S. and Kuo, F. Y. 2008. Constructing Sobol sequences with better two-dimensional projections. SIAM J. Sci. Comput. 30 , 2635–2654. doi:10.1137/070709359
- Jones, D. R. , Schonlau, M. and Welch, W. J. 1998. Efficient global optimization of expensive black-box functions. J. Global Optim. 13 , 455–492. doi:10.1023/A:1008306431147
- Kotsuki, S. , Ota, Y. and Miyoshi, T. 2017. Adaptive covariance relaxation methods for ensemble data assimilation: Experiments in the real atmosphere. Q. J. R. Meteorol. Soc. 143 , 2001–2015. doi:10.1002/qj.3060
- Lei, L. , Anderson, J. and Romine, G. 2015. Empirical localization functions for ensemble kalman filter data assimilation in regions with and without precipitation. Mon. Weather Rev. 143 , 3664–3679. doi:10.1175/MWR-D-14-00415.1
- Letham, B. , Karrer, B. , Ottoni, G. and Bakshy, E. 2019. Constrained Bayesian optimization with noisy experiments. Bayesian Anal. 14 , 495–519.
- Li, H. , Kalnay, E. and Miyoshi, T. 2009. Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Q. J. R. Meteorol. Soc. 135 , 523–533. doi:10.1002/qj.371
- Lorenc, A. 2003. The potential of the ensemble Kalman filter for NWP – A comparison with 4D-Var. Q. J. R. Meteorol. Soc. 129 , 3183–3203. doi:10.1256/qj.02.132
- Lorenc, A. 2017. Improving ensemble covariances in hybrid variational data assimilation without increasing ensemble size. Q. J. R. Meteorol. Soc. 143 , 1062–1072. doi:10.1002/qj.2990
- Lorenz, E. N. 1963. Deterministic nonperiodic flow. J. Atmos. Sci. 20 , 130–141. doi:10.1175/1520-0469(1963)020<;0130:DNF>2.0.CO;2
- Lorenz, E. N. 1996. Predictability: A problem partly solved. In: Proceedings Seminar on Predictability, Vol. 1. Reading, United Kingdom.
- Lunderman, S. , Fioroni, G. , McCormick, R. , Nimlos, M. , Rahimi, M. and co-authors. 2018. Screening fuels for autoignition with small-volume experiments and gaussian process classification. Energy Fuels 32 , 9581–9591. doi:10.1021/acs.energyfuels.8b02112
- Luo, X. and Hoteit, I. 2011. Robust ensemble filtering and its relation to covariance inflation in the ensemble Kalman filter. Mon. Weather Rev. 139 , 3938–3953. doi:10.1175/MWR-D-10-05068.1
- Ménétrier, B. , Montmerle, T. , Michel, Y. and Berre, L. 2015. Linear filtering of sample covariances for ensemble-based data assimilation. Part I: Optimality criteria and application to variance filtering and covariance localization. Monthly Weather Review 143 , 1622–1643. doi:10.1175/MWR-D-14-00157.1
-
Moosavi, A.
,
Attia, A.
and
Sandu, A.
2019.
Tuning covariance localization using machine learning . In: Computational Science – ICCS 2019 (eds. J. M. F. Rodrigues , P. J. S. Cardoso , J. Monteiro , R. Lam , V. V. Krzhizhanovskaya and co-authors.). Springer International Publishing, Cham, pp. 199–212. - Morzfeld, M. and Hodyss, D. 2019. Gaussian approximations in filters and smoothers for data assimilation. Tellus A: Dyn. Meteorol. Oceanogr. 71 , 1600344. doi:10.1080/16000870.2019.1600344
- Nerger, L. 2015. On serial observation processing in localized ensemble Kalman filters. Mon. Weather Rev. 143 , 1554–1567. doi:10.1175/MWR-D-14-00182.1
- Pirot, G. , Krityakierne, T. , Ginsbourger, D. and Renard, P. 2019. Contaminant source localization via Bayesian global optimization. Hydrol. Earth Syst. Sci. 23 , 351–369.
- Popov, A. and Sandu, A. 2019. A Bayesian approach to multivariate adaptive localization in ensemble-based data assimilation with time-dependent extensions. Nonlin. Processes Geophys. 26 , 109–122. doi:10.5194/npg-26-109-2019
- Posselt, D. , van den Heever, S. , Stephens, G. L. and Igel, M. 2012. Changes in the interaction between tropical convection, radiation and the large scale circulation in a warming environment. J. Clim. 35 , 557–571.
-
Rasmussen, C. E.
2004.
Gaussian processes in machine learning . In: Bousquet O., von Luxburg U., Rätsch G. (eds) Advanced Lectures on Machine Learning . Springer, Berlin, Heidelberg. p. 193. - Ruckstuhl, Y. and Janjić, T. 2018. Parameter and state estimation with ensemble Kalman filter based algorithms for convective-scale applications. Q. J. R. Meteorol. Soc. 144 , 826–841. doi:10.1002/qj.3257
- Shlyaeva, A. , Whitaker, J. and Snyder, C. 2019. Model-space localization in serial ensemble filters. J. Adv. Model. Earth Syst. 11 , 1627–1636. doi:10.1029/2018MS001514
-
Snoek, J.
,
Larochelle, H.
and
Adams, R. P.
2012.
Practical Bayesian optimization of machine learning algorithms . In: Bartlett, P. (ed) Advances in Neural Information Processing Systems , pp. 2951–2959. Red Hook, NY: Curran Associates Inc. - Talagrand, O. and Courtier, P. 1987. Variational assimilation of meteorological observations with the adjoint vorticity equation. I: Theory. Q. J. R. Meteorol. Soc. 113 , 1311–1328. doi:10.1002/qj.49711347812
- Vukicevic, T. and Posselt, D. 2008. Analysis of the impact of model nonlinearities in inverse problem solving. J. Atmos. Sci. 65 , 2803–2823. doi:10.1175/2008JAS2534.1
- Williams, C. K. and Rasmussen, C. E. 2006. Gaussian Processes for Machine Learning . Vol. 2, No. 3. MIT Press, Cambridge, MA.
- Zhen, Y. and Zhang, F. 2014. A probabilistic approach to adaptive covariance localization for serial ensemble square root filters. Mon. Weather Rev. 142 , 4499–4518. doi:10.1175/MWR-D-13-00390.1
DOI: https://doi.org/10.1080/16000870.2021.1924952 | Journal eISSN: 3035-9554
Language: English
Page range: 1924952 - 1924952
Published on: Jan 1, 2021
Published by: Stockholm University Press
In partnership with: Paradigm Publishing Services
Keywords:
© 2021 Spencer Lunderman, Matthias Morzfeld, Derek J. Posselt, published by Stockholm University Press
This work is licensed under the Creative Commons Attribution 4.0 License.