Filter Likelihood as an Observation-Based Verification Metric in Ensemble Forecasting
References
- Ben Bouallegue, Z, Haiden, T, Weber, NJ, Hamill, TM and Richardson, DS. 2020. “Accounting for representativeness in the verification of ensemble precipitation forecasts.” Monthly Weather Review, 148(5): 2049–2062. DOI: 10.1175/MWR-D-19-0323.1
- Bormann, N and Bauer, P. 2010. “Estimates of spatial and interchannel observationerror characteristics for current sounder radiances for numerical weather prediction. I: Methods and application to ATOVS data.” Quarterly Journal of the Royal Meteorological Society, 136(649): 1036–1050. DOI: 10.1002/qj.616
- Bowler, NE, Arribas, A, Mylne, KR, Robertson, KB and Beare, SE. 2008. “The MOGREPS short-range ensemble prediction system.” Quarterly Journal of the Royal Meteorological Society, 134(632): 703–722.
https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.234 . DOI: 10.1002/qj.234 - Buizza, R. 2019. “Introduction to the special issue on “25 years of ensemble forecasting”.” Quarterly Journal of the Royal Meteorological Society, 145: 1–11. DOI: 10.1002/qj.3370
- Buizza, R, Leutbecher, M and Isaksen, L. 2008. “Potential use of an ensemble of analyses in the ECMWF Ensemble Prediction System.” Quarterly Journal of the Royal Meteorological Society, 134(637): 2051–2066.
https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.346 . DOI: 10.1002/qj.346 - Buizza, R and Richardson, D. 2017. “25 years of ensemble forecasting at ECMWF.” ECMWF Newsletter, 153: 20–31.
- Candille, G and Talagrand, O. 2008. “Impact of observational error on the validation of ensemble prediction systems.” Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 134(633): 959–971. DOI: 10.1002/qj.268
- Christensen, HM. 2020. “Constraining stochastic parametrisation schemes using highresolution simulations.” Quarterly Journal of the Royal Meteorological Society, 146(727): 938–962. DOI: 10.1002/qj.3717
- Dawid, AP and Sebastiani, P. 1999. “Coherent dispersion criteria for optimal experimental design.” Annals of Statistics, 65–81. DOI: 10.1214/aos/1018031101
- Duc, L and Saito, K. 2018. “Verification in the presence of observation errors: Bayesian point of view.” Quarterly Journal of the Royal Meteorological Society, 144(713): 1063–1090. DOI: 10.1002/qj.3275
- Ekblom, M, Tuppi, L, Shemyakin, V, Laine, M, Ollinaho, P, Haario, H and Järvinen, H. 2020. “Algorithmic tuning of spread-skill relationship in ensemble forecasting systems.” Quarterly Journal of the Royal Meteorological Society, 146(727): 598–612. DOI: 10.1002/qj.3695
- Evensen, G. 2009. Data Assimilation: The Ensemble Kalman Filter. 2nd ed. Springer.
- Ferro, C. 2014. “Fair scores for ensemble forecasts.” Quarterly Journal of the Royal Meteorological Society, 140(683): 1917–1923. DOI: 10.1002/qj.2270
- Ferro, CAT. 2017. “Measuring forecast performance in the presence of observation error.” Quarterly Journal of the Royal Meteorological Society, 143(708): 2665–2676. DOI: 10.1002/qj.3115
- Geer, AJ, Bauer, P and English, SJ. 2012. “Assimilating AMSU-A temperature sounding channels in the presence of cloud and precipitation.” ECMWF Technical Memorandum, 670: 41. Also published as ECMWF/EUMETSAT Fellowship Programme Research Report No.24,
https://www.ecmwf.int/node/9514 . - Gneiting, T, Balabdaoui, F and Raftery, AE. 2007. “Probabilistic forecasts, calibration and sharpness.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(2): 243–268. DOI: 10.1111/j.1467-9868.2007.00587.x
- Gneiting, T and Raftery, AE. 2007. “Strictly proper scoring rules, prediction, and estimation.” Journal of the American statistical Association, 102(477): 359–378. DOI: 10.1198/016214506000001437
- Hakkarainen, J, Ilin, A, Solonen, A, Laine, M, Haario, H, Tamminen, J, Oja, E and Järvinen, J. 2012. “On closure parameter estimation in chaotic systems.” Nonlinear processes in Geophysics, 19(1): 127–143. DOI: 10.5194/npg-19-127-2012
- Hakkarainen, J, Solonen, A, Ilin, A, Susiluoto, J, Laine, M, Haario, H and Järvinen, H. 2013. “A dilemma of the uniqueness of weather and climate model closure parameters.” Tellus A: Dynamic Meteorology and Oceanography, 65(1): 20147. DOI: 10.3402/tellusa.v65i0.20147
- Hamill, TM. 2001. “Interpretation of rank histograms for verifying ensemble forecasts.” Monthly Weather Review, 129(3): 550–560. DOI: 10.1175/1520-0493(2001)129<;0550:IORHFV>2.0.CO;2
- Hersbach, H. 2000. “Decomposition of the continuous ranked probability score for ensemble prediction systems.” Weather and Forecasting, 15(5): 559–570. DOI: 10.1175/1520-0434(2000)015<;0559:DOTCRP>2.0.CO;2
- Houtekamer, PL, Deng, X, Mitchell, HL, Baek, S-J and Gagnon, N. 2014. “Higher Resolution in an Operational Ensemble Kalman Filter.” Monthly Weather Review, 142(3): 1143–1162.
https://journals.ametsoc.org/view/journals/mwre/142/3/mwr-d-13-00138.1.xml . DOI: 10.1175/MWR-D-13-00138.1 - Houtekamer, PL, Mitchell, HL and Deng, X. 2009. “Model Error Representation in an Operational Ensemble Kalman Filter.” Monthly Weather Review, 137(7): 2126–2143.
https://journals.ametsoc.org/view/journals/mwre/137/7/2008mwr2737.1.xml . DOI: 10.1175/2008MWR2737.1 - Isaksen, L, Bonavita, M, Buizza, R, Fisher, M, Haseler, J, Leutbecher, M and Raynaud, L. 2010. “Ensemble of data assimilations at ECMWF,” 636: 45.
https://www.ecmwf.int/node/10125 . - Kay, JK and Kim, HM. 2014. “Characteristics of Initial Perturbations in the Ensemble Prediction System of the Korea Meteorological Administration.” Weather and Forecasting, 29(3): 563–581.
https://journals.ametsoc.org/view/journals/wefo/29/3/waf-d-13-00097_1.xml . DOI: 10.1175/WAF-D-13-00097.1 - Köhler, D, Ekblom, M, Tuppi, L, Ollinaho, P and Järvinen, H. 2023. “Impact of model tuning on spread-skill relationship in ensemble forecasts.” Manuscript in preparation.
- Leutbecher, M. 2019. “Ensemble size: How suboptimal is less than infinity?” Quarterly Journal of the Royal Meteorological Society, 145: 107–128. DOI: 10.1002/qj.3387
- Leutbecher, M and Haiden, T. 2021. “Understanding changes of the continuous ranked probability score using a homogeneous Gaussian approximation.” Quarterly Journal of the Royal Meteorological Society, 147(734): 425–442. DOI: 10.1002/qj.3926
- Leutbecher, M, Lock, S-J, Ollinaho, P, Lang, STK, Balsamo, G, Bechtold, P, Bonavita, M, et al. 2017. “Stochastic representations of model uncertainties at ECMWF: state of the art and future vision.” Quarterly Journal of the Royal Meteorological Society, 143(707): 2315–2339.
https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.3094 . DOI: 10.1002/qj.3094 - Leutbecher, M and Palmer, TN. 2008. “Ensemble forecasting.” Journal of computational physics, 227(7): 3515–3539. DOI: 10.1016/j.jcp.2007.02.014
- McNally, T. 2006. Bias estimation and correction for satellite data assimilation. na.
https://www.ecmwf.int/sites/default/files/elibrary/2005/15832-bias-estimation-and-correction-satellite-data-assimilation.pdf . - Miyoshi, T and Sato, Y. 2007. “Assimilating Satellite Radiances with a Local Ensemble Transform Kalman Filter (LETKF) Applied to the JMA Global Model (GSM).” SOLA, 3: 37–40. DOI: 10.2151/sola.2007-010
- NWP-SAF. 2021. Radiance Simulator. na.
https://nwp-saf.eumetsat.int/site/software/radiance-simulator/ . - Ollinaho, P, Carver, GD, Lang, STK, Tuppi, L, Ekblom, M and Järvinen, H. 2021. “Ensemble prediction using a new dataset of ECMWF initial states–OpenEnsemble 1.0.” Geoscientific Model Development, 14(4): 2143–2160. DOI: 10.5194/gmd-14-2143-2021
- Ollinaho, P, Lock, S-J, Leutbecher, M, Bechtold, P, Beljaars, A, Bozzo, A, Forbes, RM, Haiden, T, Hogan, RJ and Sandu, I. 2017. “Towards process-level representation of model uncertainties: stochastically perturbed parametrizations in the ECMWF ensemble.” Quarterly Journal of the Royal Meteorological Society, 143(702): 408–422.
https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.2931 . DOI: 10.1002/qj.2931 - Saetra, ∅, Hersbach, H, Bidlot, J-R and Richardson, DS. 2004. “Effects of observation errors on the statistics for ensemble spread and reliability.” Monthly Weather Review, 132(6): 1487–1501. DOI: 10.1175/1520-0493(2004)132<;1487:EOOEOT>2.0.CO;2
- Särkkä, S. 2013. Bayesian Filtering and Smoothing.
Institute of Mathematical Statistics Textbooks . Cambridge University Press. DOI: 10.1017/CBO9781139344203 - Saunders, R, Hocking, J, Turner, E, Rayer, P, Rundle, D, Brunel, P, Vidot, J, et al. 2018. “An update on the RTTOV fast radiative transfer model (currently at version 12).” Geoscientific Model Development, 11(7): 2717–2737. DOI: 10.5194/gmd-11-2717-2018
- Schweppe, FC. 1965. “Evaluation of likelihood functions for Gaussian signals.” IEEE Transactions on Information Theory, 11(1): 61–70. DOI: 10.1109/TIT.1965.1053737
- Siegert, S, Ferro, CAT, Stephenson, DB and Leutbecher, M. 2019. “The ensemble-adjusted Ignorance Score for forecasts issued as normal distributions.” Quarterly Journal of the Royal Meteorological Society, 145: 129–139. DOI: 10.1002/qj.3447
- Solonen, A and Järvinen, H. 2013. “An approach for tuning ensemble prediction systems.” Tellus A: Dynamic Meteorology and Oceanography, 65(1): 20594. DOI: 10.3402/tellusa.v65i0.20594
- Toth, Z and Kalnay, E. 1993. “Ensemble Forecasting at NMC: The Generation of Perturbations.” Bulletin of the American Meteorological Society, 74(12): 2317–2330.
https://journals.ametsoc.org/view/journals/bams/74/12/1520-0477_1993_074_2317_efantg_2_0_co_2.xml . DOI: 10.1175/1520-0477(1993)074<;2317:EFANTG>2.0.CO;2 - Toth, Z and Kalnay, E. 1997. “Ensemble Forecasting at NCEP and the Breeding Method.” Monthly Weather Review, 125(12): 3297–3319.
https://journals.ametsoc.org/view/journals/mwre/125/12/1520-0493_1997_125_3297_efanat_2.0.co_2.xml . DOI: 10.1175/1520-0493(1997)125<;3297:EFANAT>2.0.CO;2 - Vannitsem, S, Wilks, DS and Messner, J. 2018. Statistical postprocessing of ensemble forecasts. Elsevier.
- Virtanen, P, Gommers, R, Oliphant, TE, Haberland, M, Reddy, T, Cournapeau, D, Burovski, E, et al. 2020. “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python.” Nature Methods, 17: 261–272. DOI: 10.1038/s41592-019-0686-2
- Wei, M, Toth, Z, Wobus, R and Zhu, Y. 2008. “Initial perturbations based on the ensemble transform (ET) technique in the NCEP global operational forecast system.” Tellus A: Dynamic Meteorology and Oceanography, 60(1): 62–79. DOI: 10.1111/j.1600-0870.2007.00273.x
- Wei, M, Toth, Z, Wobus, R, Zhu, Y, Bishop, CH and Wang, X. 2006. “Ensemble Transform Kalman Filter-based ensemble perturbations in an operational global prediction system at NCEP.” Tellus A: Dynamic Meteorology and Oceanography, 58(1): 28–44. DOI: 10.1111/j.1600-0870.2006.00159.x
- Yamaguchi, M, Lang, STK, Leutbecher, M, Rodwell, MJ, Radnoti, G and Bormann, N. 2016. “Observation-based evaluation of ensemble reliability.” Quarterly Journal of the Royal Meteorological Society, 142(694): 506–514. DOI: 10.1002/qj.2675
- Zhang, Z and Krishnamurti, TN. 1999. “A Perturbation Method for Hurricane Ensemble Predictions.” Monthly Weather Review, 127(4): 447–469.
https://journals.ametsoc.org/view/journals/mwre/127/4/1520-0493_1999_127_0447_apmfhe_2.0.co_2.xml . DOI: 10.1175/1520-0493(1999)127<;0447:APMFHE>2.0.CO;2
DOI: https://doi.org/10.16993/tellusa.96 | Journal eISSN: 3035-9554
Language: English
Page range: 69 - 87
Submitted on: May 13, 2022
Accepted on: Dec 2, 2022
Published on: Feb 6, 2023
Published by: Stockholm University Press
In partnership with: Paradigm Publishing Services
Keywords:
© 2023 Madeleine Ekblom, Lauri Tuppi, Olle Räty, Pirkka Ollinaho, Marko Laine, Heikki Järvinen, published by Stockholm University Press
This work is licensed under the Creative Commons Attribution 4.0 License.