Verification of an Ensemble Prediction System against Observations

Candille, G.; Côté, C.; Houtekamer, P. L.; Pellerin, G.
July 2007
Monthly Weather Review;Jul2007, Vol. 135 Issue 7, p2688
Academic Journal
A verification system has been developed for the ensemble prediction system (EPS) at the Canadian Meteorological Centre (CMC). This provides objective criteria for comparing two EPSs, necessary when deciding whether or not to implement a new or revised EPS. The proposed verification methodology is based on the continuous ranked probability score (CRPS), which provides an evaluation of the global skill of an EPS. Its reliability/resolution partition, proposed by Hersbach, is used to measure the two main attributes of a probabilistic system. Also, the characteristics of the reliability are obtained from the two first moments of the reduced centered random variable (RCRV), which define the bias and the dispersion of an EPS. Resampling bootstrap techniques have been applied to these scores. Confidence intervals are thus defined, expressing the uncertainty due to the finiteness of the number of realizations used to compute the scores. All verifications are performed against observations to provide more independent validations and to avoid any local systematic bias of an analysis. A revised EPS, which has been tested at the CMC in a parallel run during the autumn of 2005, is described in this paper. This EPS has been compared with the previously operational one with the verification system presented above. To illustrate the verification methodology, results are shown for the temperature at 850 hPa. The confidence intervals are computed by taking into account the spatial correlation of the data and the temporal autocorrelation of the forecast error. The revised EPS performs significantly better for all the forecast ranges, except for the resolution component of the CRPS where the improvement is no longer significant from day 7. The significant improvement of the reliability is mainly due to a better dispersion of the ensemble. Finally, the verification system correctly indicates that variations are not significant when two theoretically similar EPSs are compared.


Related Articles

  • Discrete Response Multilevel Models for Repeated Measures: An Application to Voting Intentions Data. Barbosa, Maria Ferrao; Goldstein, Harvey // Quality & Quantity;Aug2000, Vol. 34 Issue 3, p323 

    Repeated measures data can be modelled as a two-level model where occasions (level one units) are grouped by individuals (level two units). Goldstein et al. (1994) proposed a multilevel time series model when the response variable follows a Normal distribution and the measurements are taken with...

  • SOBRE A ESTIMAÇÃO DE INTERVALOSDE CONFIANÇA PARA OSQUANTISDE VARIÁVEIS ALEATÓRIAS HIDROLÓGICAS. Silva, Artur Tiago; Naghettini, Mauro; Portela, Maria Manuela // Recursos Hídricos;Nov2011, Vol. 32 Issue 2, p63 

    The research presented in this paper analyses some theoretical and practical aspects of constructing confidence intervals for the quantiles of hydrological random variables. Such intervals assume an important role in the design and operation of water resources systems, as they summarize the...

  • Distributional properties of georeferenced random variables based on the eigenfunction spatial filter. Griffith, Daniel A. // Journal of Geographical Systems;Oct2004, Vol. 6 Issue 3, p263 

    The eigenfunction spatial filter derives from the Moran Coefficient that indexes spatial autocorrelation. Mean, variance and statistical distribution characterizations and descriptions of georeferenced random variables and their interrelationships are derived in terms of the eigenfunction...

  • GENERATION OF STANDARD NORMAL RANDOM VARIABLES. Rao, Kollu Ranga; Boiroju, Naveen Kumar; Reddy, M. Krishna // Indian Journal of Scientific Research;Dec2011, Vol. 2 Issue 4, p83 

    In this paper, we discuss generation of random variables from standard normal distribution. We apply inverse transform method to approximation of cumulative normal distribution. We require atleast two uniform random variables to generate a single standard normal variable, but in the proposed...

  • Calculating the accuracy of hierarchical estimation. STRIJBOSCH, L. W. G.; MOORS, J. J. A. // IMA Journal of Management Mathematics;Jul2010, Vol. 21 Issue 3, p303 

    Instead of forecasting demand for individual items separately, hierarchical forecasting is often used: total demand is forecasted for a collection of items; this total forecast then is broken down to produce the desired individual demand forecasts. To allow analytical analyses, we considered in...

  • Confidence intervals for rate ratios between geographic units. Zhu, Li; Pickle, Linda W.; Pearson, James B. // International Journal of Health Geographics;12/15/2016, Vol. 15, p1 

    Background: Ratios of age-adjusted rates between a set of geographic units and the overall area are of interest to the general public and to policy stakeholders. These ratios are correlated due to two reasons--the first being that each region is a component of the overall area and hence there is...

  • Performance Characteristics of a Pseudo-Operational Ensemble Kalman Filter. Torn, Ryan D.; Hakim, Gregory J. // Monthly Weather Review;Oct2008, Vol. 136 Issue 10, p3947 

    The 2-yr performance of a pseudo-operational (real time) limited-area ensemble Kalman filter (EnKF) based on the Weather Research and Forecasting Model is described. This system assimilates conventional observations from surface stations, rawinsondes, the Aircraft Communications Addressing and...

  • Influence of the compiler on multi-CPU performance of WRFv3. Langkamp, T.; Böhner, J. // Geoscientific Model Development;Jul2011, Vol. 4 Issue 3, p611 

    The article presents a study which examines the influence of compilers on the multi-central processing unit (CPU) performance of the Weather Research and Forecasting system version 3 (WRFv3). It concludes that the use of compilers and the Message Passing Interface library versions, a software...

  • Group-membership recognition methods based on confidence boundaries and their testing in clinical oncology. Petunin, Yu.; Klyushin, D.; Golubeva, E.; Naleskina, L.; Kunskaya, L.; Chekhun, V. // Cybernetics & Systems Analysis;Jul2009, Vol. 45 Issue 4, p517 

    Confidence intervals are constructed for an unknown correlation coefficient and mathematical expectation using 3 s1 and 2 s1 rules. The intervals make it possible to solve the problem of statistical analysis of the dependence between different random variables. The results obtained are used to...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics