Data compression in the presence of observational error correlations

A.M. Fowler

doi:10.1080/16000870.2019.1634937

Data compression in the presence of observational error correlations

Tellus

Volume 71 (2019): Issue 1

By: A.M. Fowler

Open Access

|Jan 2019

Berger, H. and Forsythe, M. 2004. Satellite wind superobbing. Met Office Forecasting Research Technical Report No. 451. http://research.metoffice.gov.uk/research/nwp/publications/papers/technical_reports
Search in Google Scholar Back to article
Bergman, K. H. and Bonner, W. D. 1976. Analysis error as a function of observation density for satellite temperature soundings with spatially correlated errors. Mon. Wea. Rev. 104, 1308–1316. doi:10.1175/1520-0493(1976)104<;1308:AEAAFO>2.0.CO;2
Open DOI Search in Google Scholar Back to article
Bormann, N. and Bauer, P. 2010. Estimates of spatial and interchannel observation-error characteristics for current sounder radiances for numerical weather prediction. I: methods and application to ATOVS data. Q. J. R. Meteorol. Soc. 136, 1036–1050. doi:10.1002/qj.616
Open DOI Search in Google Scholar Back to article
Bormann, N., Saarinen, S., Kelly, G. and Thépaut, J.-N. 2003. The spatial structure of observation errors in atmospheric motion vectors from geostationary satellite data. Mon. Wea. Rev. 131, 706–718. doi:10.1175/1520-0493(2003)131<;0706:TSSOOE>2.0.CO;2
Open DOI Search in Google Scholar Back to article
Bormann, N., Bonavita, M., Dragani, R., Eresmaa, R., Matricardi, M. and co-authors. 2016. Enhancing the impact of IASI observations through an updated observation-error covariance matrix. Q. J. R. Meteorol. Soc. 142, 1767–1780. doi:10.1002/qj.2774
Open DOI Search in Google Scholar Back to article
Campbell, W. F., Satterfield, E. A., Ruston, B. and Baker, N. L. 2017. Accounting for correlated observation error in a dual formulation 4D-variational data assimilation system. Mon. Wea. Rev. 145, 1019. doi:10.1175/MWR-D-16-0240.1
Open DOI Search in Google Scholar Back to article
Cardinali, C., Pezzulli, S. and Andersson, E. 2004. Influence-matrix diagnostics of a data assimilation system. Q. J. R Meteorol. Soc. 130, 2767–2786. doi:10.1256/qj.03.205
Open DOI Search in Google Scholar Back to article
Collard, A. D., McNally, A. P., Hilton, F. I., Healy, S. B. and Atkinson, N. C. 2010. The use of principal component analysis for the assimilation of high-resolution infrared sounder observations for numerical weather prediction. Q. J. R. Meteorol. Soc. 136, 2038–2050. URL https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.701. doi:10.1002/qj.701
Open DOI Search in Google Scholar Back to article
Cordoba, M., Dance, S. L., Kelly, G. A., Nichols, N. K. and Waller, J. A. 2017. Diagnosing atmospheric motion vector observation errors for an operational high resolution data assimilation system. Q. J. R. Meteorol. Soc. 143, 333–341. doi:10.1002/qj.2925
Open DOI Search in Google Scholar Back to article
Dando, M. L., Thorpe, A. J. and Eyre, J. R. 2007. The optimal density of atmospheric sounder observations in the Met Office NWP system. Q. J. R. Meteorol. Soc. 133, 1933–1943. doi:10.1002/qj.175
Open DOI Search in Google Scholar Back to article
Evensen, G. 1994. Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res. 99, 10143–10162. doi:10.1029/94JC00572
Open DOI Search in Google Scholar Back to article
Fowler, A., Dance, S. and Waller, J. 2018. On the interaction of observation and a-priori error correlations. Q. J. R. Meteorol. Soc. 144, 48–62. doi:10.1002/qj.3183
Open DOI Search in Google Scholar Back to article
Fowler, A. M. 2017. A sampling method for quantifying the information content of IASI channels. Mon.Wea. Rev. 145, 709–725. doi:10.1175/MWR-D-16-0069.1
Open DOI Search in Google Scholar Back to article
Goldberg, M. D., Qu, Y., McMillin, L. M., Wolf, W., Zhou, L. and Divakarla, M. 2003. AIRS near-real-time products and algorithms in support of operational numerical weather prediction. IEEE Trans. Geosci. Remote Sensing 41, 379–389. doi:10.1109/TGRS.2002.808307
Open DOI Search in Google Scholar Back to article
Golub, G. H. and Van Loan, C. F. 1996. Matrix Computations. 3rd ed. The John Hopkins University Press, Baltimore, MD.
Search in Google Scholar Back to article
Gray, R. 2006. Toeplitz and Circulant Matrices: A Review. Foundations and Trends in Communications and Information Theory: Vol 2: No. 3, pp 155–239. doi:10.1561/0100000006
Search in Google Scholar Back to article
Haben, S. A., Lawless, A. S. and Nichols, N. K. 2011. Conditioning of incremental variational data assimilation, with application to the Met Office system. Tellus A 64, 782–792.
Search in Google Scholar Back to article
Hamill, T. M., Whitaker, J. S. and Snyder, C. 2001. Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev. 129, 2776–2790. doi:10.1175/1520-0493(2001)129<;2776:DDFOBE>2.0.CO;2
Open DOI Search in Google Scholar Back to article
Hilton, F., Collard, A., Guidard, V., Randriamampianina, R. and Schwaerz, M. 2009. Assimilation of IASI radiances at European NWP centres. In: Proceedings of Workshop on the assimilation of IASI data in NWP, ECMWF, Reading, UK, 6-8 May 2009.
Search in Google Scholar Back to article
Houtekamer, P. L. and Zhang, F. 2016. Review of the ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev. 144, 4489–4532. doi:10.1175/MWR-D-15-0440.1
Open DOI Search in Google Scholar Back to article
Huang, H.-L. and Purser, R. J. 1996. Objective measures of the information density of satellite data. Meteorl. Atmos. Phys. 60, 105–117. doi:10.1007/BF01029788
Open DOI Search in Google Scholar Back to article
Hunt, B. R., Kostelich, E. J. and Szunyogh, I. 2007. Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter. Physica D 230, 112–126. doi:10.1016/j.physd.2006.11.008
Open DOI Search in Google Scholar Back to article
Janjic, T., Bormann, N., Bocquet, M., Carton, J. A., Cohn, S. E. and co-authors. 2017. On the representation error in data assimilation. Q. J. R. Meteorol. Soc. 144, 1257–1278. doi:10.1002/qj.3130
Open DOI Search in Google Scholar Back to article
Kalnay, E. 2003. Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, New York,.
Search in Google Scholar Back to article
Kent, T., Bokhove, O. and Tobias, S. 2017. Dynamics of an idealized fluid model for investigating convective-scale data assimilation. Tellus A 69, 1369332. URL doi:10.1080/16000870.2017.1369332
Open DOI Search in Google Scholar Back to article
Khare, S. P. and Anderson, J. L. 2006. An examination of ensemble filter based adaptive observation methodologies. Tellus 58A, 179–195.
Search in Google Scholar Back to article
Li, Z., Ballard, S. P. and Simonin, D. 2018. Comparison of 3D-Var and 4D-Var data assimilation in an NWP-based system for precipitation nowcasting at the Met Office. Q. J. R. Meteorol. Soc. 144, 404–413. URL https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.3211. doi:10.1002/qj.3211
Open DOI Search in Google Scholar Back to article
Liu, Z.-Q. and Rabier, F. 2002. The interaction between model resolution, observation resolution and observational density in data assimilation: a one-dimensional study. Q. J. R. Meteorol. Soc. 128, 1367–1386. doi:10.1256/003590002320373337
Open DOI Search in Google Scholar Back to article
Liu, Z.-Q. and Rabier, F. 2003. The potential of high-density observations for numerical weather prediction: a study with simulated observations. Q. J. R. Meteorol. Soc. 129, 3013–3035. doi:10.1256/qj.02.170
Open DOI Search in Google Scholar Back to article
Lorenz, E. N. 1995. Predictability – a problem partly solved. In: Seminar Proceedings I, Reading, UK, ECMWF, pp. 1–18.
Search in Google Scholar Back to article
Lorenz, E. N. and Emanuel, K. A. 1998. Optimal sites for supplementary weather observations: simulation with a small model. J. Atmos. Sci. 55, 399–414. doi:10.1175/1520-0469(1998)055<;0399:OSFSWO>2.0.CO;2
Open DOI Search in Google Scholar Back to article
Majumdar, S. J. 2016. A review of targeted observations. Bull. Am. Meteor. Soc. 97, 2287–2303. URL doi:10.1175/BAMS-D-14-00259.1
Open DOI Search in Google Scholar Back to article
Migliorini, S. 2013. Information-based data selection for ensemble data assimilation. Q. J. R. Meteorol. Soc. 139, 2033–2054. doi:10.1002/qj.2104
Open DOI Search in Google Scholar Back to article
Migliorini, S. 2015. Optimal ensemble-based selection of channels from advanced sounders in the presence of cloud. Mon. Wea. Rev. 143, 3754–3773. URL doi:10.1175/MWR-D-14-00249.1
Open DOI Search in Google Scholar Back to article
Miyoshi, T., Lien, G.-Y., Satoh, S., Ushio, T., Bessho, K. and co-authors. 2016. Big data assimilation” toward post-petascale severe weather prediction: An overview and progress. Proc. IEEE 20, 2155–2179.
Search in Google Scholar Back to article
Oke, P. R., Sakov, P. and Corney, S. P. 2007. Impacts of localisation in the EnKF and EnOI: experiments with a small model. Ocean Dyn. 57, 32–45. doi:10.1007/s10236-006-0088-8
Open DOI Search in Google Scholar Back to article
Rabier, F., Jarvinen, H., Klinker, E., Mahfouf, J.-F. and Simmons, A. 2000. The ECMWF operational implementation of four-dimensional variational data assimilation. I: experimental results and simplified physics. Q J. R. Meteorol. Soc. 126, 1143–1170. doi:10.1002/qj.49712656415
Open DOI Search in Google Scholar Back to article
Rabier, F., Fourrié, N., Chafa I, D. and Prunet, P. 2002. Channel selection methods for infrared atmospheric sounding interferometer radiances. Q J. R Meteorol. Soc. 128, 1011–1027. doi:10.1256/0035900021643638
Open DOI Search in Google Scholar Back to article
Rainwater, S., Bishop, C. H. and Campbell, W. F. 2015. The benefits of correlated observation errors for small scales. Q. J. R. Meteorol. Soc. 141, 3439–3445. doi:10.1002/qj.2582
Open DOI Search in Google Scholar Back to article
Rawlins, F., Ballard, S. P., Bovis, K. J., Clayton, A. M., Li, D. and co-authors. 2007. The Met Office global four-dimensional variational data assimilation scheme. Q. J. R. Meteorol. Soc. 133, 347–362. doi:10.1002/qj.32
Open DOI Search in Google Scholar Back to article
Rodgers, C. D. 2000. Inverse Methods for Atmospheric Sounding. World Scientific Publishing, Singapore.
Search in Google Scholar Back to article
Seaman, R. S. 1977. Absolute and differential accuracy of analyses achievable with specified observational network characteristics. Mon. Wea. Rev. 105, 1211–1222. doi:10.1175/1520-0493(1977)105<;1211:AADAOA>2.0.CO;2
Open DOI Search in Google Scholar Back to article
Stewart, L. M., Dance, S. L., Nichols, N. K., Eyre, J. R. and Cameron, J. 2014. Estimating interchannel observation-error correlations for IASI radiance data in the Met Office system. Q. J. R. Meteorol. Soc. 140, 1236–1244. doi:10.1002/qj.2211
Open DOI Search in Google Scholar Back to article
Tabeart, J. M., Dance, S. L., Haben, S. A., Lawless, A. S., Nichols, N. K. and co-authors. 2018. The conditioning of least squares problems in variational data assimilation. Numer. Linear Algebra Appl. 0, e2165. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/nla.2165.
Search in Google Scholar Back to article
van Leeuwen, P. J. 2015. Representation errors and retrievals in linear and nonlinear data assimilation. Q. J. R. Meteorol. Soc 141, 612–1623.
Search in Google Scholar Back to article
Waller, J. A., Dance, S. L., Lawless, A. S. and Nichols, N. K. 2014. Estimating correlated observation error statistics using an ensemble transform Kalman filter. Tellus A 66, 23294. doi:10.3402/tellusa.v66.23294
Open DOI Search in Google Scholar Back to article
Waller, J. A., Ballard, S. P., Dance, S. L., Kelly, G., Nichols, N. K. and co-authors. 2016a. Diagnosing horizontal and inter-channel observation error correlations for SEVIRI observations using observation-minus-background and observation-minus-analysis statistics. Remote Sens. 8, 581. doi:10.3390/rs8070581
Open DOI Search in Google Scholar Back to article
Waller, J. A., Simonin, D., Dance, S. L., Nichols, N. K. and Ballard, S. P. 2016b. Diagnosing observation error correlations for Doppler radar radial winds in the Met Office UKV model using observation-minus-background and observation-minus-analysis statistics. Mon. Wea. Rev. 144, 3533. doi:10.1175/MWR-D-15-0340.1
Open DOI Search in Google Scholar Back to article
Weston, P. P., Bell, W. and Eyre, J. R. 2014. Accounting for correlated error in the assimilation of high-resolution sounder data. Q. J. R. Meteorol. Soc. 140, 2420–2429. doi:10.1002/qj.2306
Open DOI Search in Google Scholar Back to article
Xu, Q. 2007. Measuring information content from observations for data assimilation: relative entropy versus Shannon entropy difference. Tellus 59A, 198–209.
Search in Google Scholar Back to article
Xu, Q., Wei, L. and Healy, S. 2009. Measuring information content from observations for data assimilation: connection between different measures and application to radar scan design. Tellus 61A, 144–153.
Search in Google Scholar Back to article