Spatial-temporal fractions verification for high-resolution ensemble forecasts

Experiments with two ensemble systems of resolutions 10 km (MF10km) and 2 km (MF2km) were designed to examine the value of cloud-resolving ensemble forecast in predicting precipitation on small spatio-temporal scales. Since the verification was performed on short-term precipitation at high resolution, uncertainties from small-scale processes caused the traditional verification methods to be inconsistent with the subjective evaluation. An extended verification method based on the Fractions Skill Score (FSS) was introduced to account for these uncertainties. The main idea is to extend the concept of spatial neighbourhood in FSS to the time and ensemble dimension. The extension was carried out by recognising that even if ensemble forecast is used, small-scale variability still exists in forecasts and influences verification results. In addition to FSS, the neighbourhood concept was also incorporated into reliability diagrams and relative operating characteristics to verify the reliability and resolution of two systems.

The extension of FSS in time dimension demonstrates the important role of temporal scales in short-term precipitation verification at small spatial scales. The extension of FSS in ensemble space is called the ensemble FSS, which is a good representative of FSS for ensemble forecast in comparison with the FSS of ensemble mean. The verification results show that MF2km outperforms MF10km in heavy rain forecasts. In contrast, MF10km was slightly better than MF2km in predicting light rains, suggesting that the horizontal resolution of 2 km is not necessarily enough to completely resolve convective cells.