Verification and intercomparison of mesoscale ensemble prediction systems in the Beijing 2008 Olympics Research and Development Project

Masaru Kunii; Kazuo Saito; Hiromu Seko; Masahiro Hara; Tabito Hara; Munehiko Yamaguchi; Jiandong Gong; Martin Charron; Jun Du; Yong Wang; Dehui Chen

doi:10.1111/j.1600-0870.2011.00512.x

Verification and intercomparison of mesoscale ensemble prediction systems in the Beijing 2008 Olympics Research and Development Project

Volume 63 (2011): Issue 3

By: Masaru Kunii, Kazuo Saito, Hiromu Seko, Masahiro Hara, Tabito Hara, Munehiko Yamaguchi, Jiandong Gong, Martin Charron, Jun Du, Yong Wang and Dehui Chen

Open Access

|Jan 2011

Abstract

During the period around the Beijing 2008 Olympic Games, the Beijing 2008 Olympics Research and Development Project (B08RDP) was conducted as part of the World Weather Research Program short-range weather forecasting research project. Mesoscale ensemble prediction (MEP) experiments were carried out by six organizations in nearreal time, in order to share their experiences in the development of MEP systems. The purpose of this study is to objectively verify these experiments and to clarify the problems associated with the current MEP systems through the same experiences.

Verification was performed using the MEP outputs interpolated into a common verification domain with a horizontal resolution of 15 km. For all systems, the ensemble spreads grew as the forecast time increased, and the ensemble mean improved the forecast errors compared with individual control forecasts in the verification against the analysis fields. However, each system exhibited individual characteristics according to the MEP method.

Some participants used physical perturbation methods. The significance of these methods was confirmed by the verification. However, the mean error (ME) of the ensemble forecast in some systems was worse than that of the individual control forecast. This result suggests that it is necessary to pay careful attention to physical perturbations.

References

Brier , G ., 1950 . Verification of forecasts expressed in terms of probabil-ity. Mon . Wea. Re v ., 78 , 1 – 3 .
Search in Google Scholar Back to article
Buizza , R. , Tribbia , J ., Molteni , E and Palmer , T. N. , 1993. Computation of optimal unstable structures for a numerical weather prediction model. Tellus , 45A , 388 - 407 .
Search in Google Scholar Back to article
Casati , B. , Wilson , L. J. , Stephenson , D. B. , Nurmi , R , Ghelli , A. and co-authors , 2008. Forecast verification: current status and future di-rections. Meteor AppL , 15 , 3 - 18 .
Search in Google Scholar Back to article
Charron , M. , Pellerin , G. , Spacek , L. , Houtekamer , P. L. , Gagnon , N. and co-authors . 2010 . Toward random sampling of model error in the Canadian ensemble prediction system . Mon. Wea. Rev ., 138 , 1877– 1901 .
Search in Google Scholar Back to article
Chen D H , Xue , J. S. and Yang , X. S ., 2008 . New generation of multi-scale NVVP system (GRAPES): general scientific design . China Sci. Bull ., 53 ( 22 ): 3433 – 3445
Search in Google Scholar Back to article
Cote , J. , Gravel , S. , Methot , A. , Patoine, A. Roch , M. and co-authors. 1998a. The operational CMC-MRB Global Environmental Multiscale (GEM) model , Part I: design considerations and formulation. Mon. Wea. Rev ., 126 , 1373-139 5 .
Search in Google Scholar Back to article
Cote , J. , Desmarais , J.-G. Gravel, S. Methot , A. Patoine , A. and co-authors. 1998b. The operational CMC-MRB Global Environmen-tal Multiscale (GEM) model , part II: results. Mon. Wea. Rev ., 126 , 1397-141 8 .
Search in Google Scholar Back to article
Derkova. , M. and Bellus , M . 2007 . Various applications of the blending by digital filter technique in the ALADIN numerical weather predic-tion system. Meteorologic14 casopis , 10 , 27 - 36. Available online at http://wwwsc-lace.eu.
Search in Google Scholar Back to article
Du , J . 2004 . Hybrid ensemble prediction system: a new ensem-bling approach. Preprints, Symposium on the 50th Anniversary of Operational Numerical Weather Prediction , University of Mary-land, College Park, Maryland, June 14-17, 2004, Amer. Me-teor. Soc., CD-ROM (paper p4.2, 5pp). Available online at http://www.emc.ncep.noaa.gov/mmb/SREF/reference.html.
Search in Google Scholar Back to article
Ebisuzalci , W. and Kalnay , E . 1991 . Ensemble experiments with a new lagged average forecasting scheme. WMO Research Activities in At-mospheric and Oceanic Modeling Rep . 15 , Geneva, Switzerland , 6.31 - 6.32 .
Search in Google Scholar Back to article
Evensen , G ., 1994 . Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statis-tics. J. Geophys. Res ., 99 ( C5 ), 10 143-10 162 .
Search in Google Scholar Back to article
Harvey , L. O. , Hammond , K. R. , Lusk , C. M. and Mross , E. F. 1992. The application of signal detection theory to weather forecasting behavior. Mon. Wea. Rev ., 120 , 863 - 883 .
Search in Google Scholar Back to article
Hou , D. , Kalnay , E. and Drogemeier , K . 2001 . Objective verification of the SAMEX98 ensemble forecasts. Mon . Wea. Re v ., 129 , 73 – 91 .
Search in Google Scholar Back to article
Janjié , Z. I. , Gerrity Jr., J. P . and Nickovic , S. 2001 . An alternative ap-proach to nonhydrostatic modeling . Mon. Wea. Rev ., 129 , 1164 - 1178 .
Search in Google Scholar Back to article
Keenan , T. , Joe , P. , Wilson , J. , Collier , C. , Golding , B. and co-authors . 2003 . The Sydney 2000 World Weather Research Programme Forecast Demonstration Project. Bull. A. Met. Soc ., 84 , 1041 - 1054 .
Search in Google Scholar Back to article
Kunii , M. , Saito , K. and Seko , H . 2010 . Mesoscale data assimilation experiment in the WWRP BO8RDP . SOLA , 6 , 33 – 36 .
Search in Google Scholar Back to article
Mullen , S. L. , and Baumhefner , D. P . 1989 . The impact of initial con-dition uncertainty on numerical simulations of large-scale explosive cyclogenesis. Mon . Wea. Re v ., 117 , 2800 – 2821 .
Search in Google Scholar Back to article
Murphy , A. H ., 1973 . A new vector partition of the probability score . J. AppL Meteor , 12 , 595 – 600 .
Search in Google Scholar Back to article
Murphy , A. H ., 1988 . Skill scores based on the mean square error and their relationships to the correlation coefficient. Mon . Wea. Re v ., 116 , 2417 – 2424 .
Search in Google Scholar Back to article
Richardson , D. S ., 2000 . Skill and relative economic value of the ECMWF ensemble prediction system . Q. J. R. Meteor Soc ., 126 , 649 – 667 .
Search in Google Scholar Back to article
Saito , K. , Fujita , T. , Yamada , Y. , Ishida , J. , Kumagai , Y. and co-authors . 2006 . The operational JMA nonhydrostatic mesoscale model. Mon. Wea. Rev ., 134 , 1266 - 1298 .
Search in Google Scholar Back to article
Saito , K. , Ishida , J. , Aranami , K. , Hara , T. , Segawa , T. and co-authors . 2007 . Nonhydrostatic atmospheric models and operational develop-ment at .TMA. J. Meteor Soc. Japan , 85B , 271 - 304 .
Search in Google Scholar Back to article
Saito , K. , Seko , H. , Kunii , M. and Hara , M . 2008 . Mesoscale ensem-ble prediction experiment for WWRP Beijing Olympic 2008 RDP: 2007 preliminary experiment. CAS/JSC WGNE Research Activities in Atmospheric and Oceanic Modelling , 37 , 1.23 - 1.24 .
Search in Google Scholar Back to article
Saito , K. , Kunii , M. , Hara , M. , Seko , H. , Hara , T. and co-authors . 2010 . WWRP Beijing Olympics 2008 Forecast Demon-stration/Research and Development Project (B08FDP/RDP). Tech. Rep. MRI , 62 , 1 - 146. Available online at http://www.mri-jma.go. jp/Publish/Technical/DATANOL_62/62_en.html.
Search in Google Scholar Back to article
Saito , K. , Hara , M. , Kunii , M. , Seko , H. and Yamaguchi , M . 2011 . Com-parison of initial perturbation methods for the mesoscale ensemble prediction system of the Meteorological Research Institute for the WWRP Beijing 2008 Olympics Research and Development Project (B08RDP) . Tellus , 63A .
Search in Google Scholar Back to article
Skamarock , W. C. , Klemp , J. B. , Dudhia , J. , Gill , D. O. , Barker , D. M. and co-authors. 2005. A description of the Advanced Research WRF, Version 2. NCAR Technical Note.
Search in Google Scholar Back to article
Toth , Z. , and Kalnay , E . 1993 . Ensemble forecasting at NMC: the gen-eration of perturbations . Bull. Am. Meteor. Soc ., 74 , 2317 – 2330 .
Search in Google Scholar Back to article
Toth , Z. , and Kalnay , E . 1997 . Ensemble forecasting at NCEP: the breeding method. Mon . Wea. Re v ., 125 , 3297 – 3318 .
Search in Google Scholar Back to article
Wei , M. , Toth , Z. , Wobus , R. and Zhu , Y . 2008 . Initial perturbations based on the ensemble transform (ET) technique in the NCEP global operational forecast system . Tellus , 60A , 62 – 79 .
Search in Google Scholar Back to article
Yamaguchi , M. , Sakai , R. , Kyoda , M. Komori , T. and Kadowaki , T. 2009. Typhoon ensemble prediction system developed at the Japan Meteorological Agency. Mon. Wea. Rev ., 137 , 2592 - 2604 .
Search in Google Scholar Back to article
Yeh , K.-S. , Cote , J. , Gravel , S. , Methot , A. , Patoine , A. and co-authors . 2002 . The CMC-MRB global environmental multiscale (GEM) model, Part HL nonhydrostatic formulation. Mon. Wea. Rev ., 130 , 339 - 356 .
Search in Google Scholar Back to article
Zhou , B. and Du , J . 2010 . Fog prediction from a multimodel mesoscale ensemble prediction system . Weather and Forecasting , 25 , 302 – 322 .
Search in Google Scholar Back to article

Articles in this issue

DOI: https://doi.org/10.1111/j.1600-0870.2011.00512.x | Journal eISSN: 3035-9554

Journal RSS Feed

Language: English

Page range: 531 - 549

Submitted on: Apr 20, 2010

Accepted on: Jan 13, 2011

Published on: Jan 1, 2011

Published by: Stockholm University Press

In partnership with: Paradigm Publishing Services

© 2011 Masaru Kunii, Kazuo Saito, Hiromu Seko, Masahiro Hara, Tabito Hara, Munehiko Yamaguchi, Jiandong Gong, Martin Charron, Jun Du, Yong Wang, Dehui Chen, published by Stockholm University Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Volume 63 (2011): Issue 3