Effects of Observation Errors on the Statistics for Ensemble Spread and Reliability

Saetra, Øyvind; Hersbach, Hans; Bidlot, Jean-Raymond; Richardson, David S.
June 2004
Monthly Weather Review;Jun2004, Vol. 132 Issue 6, p1487
Academic Journal
The effects of observation errors on rank histograms and reliability diagrams are investigated using a perfect model approach. The three-variable Lorenz-63 model was used to simulate an idealized ensemble prediction system (EPS) with 50 perturbed ensemble members and one control forecast. Observation errors at verification time were introduced by adding normally distributed noise to the true state at verification time. Besides these simulations, a theoretical analysis was also performed. One of the major findings was that rank histograms are very sensitive to the presence of observation errors, leading to overpopulated upper- and lowermost ranks. This sensitivity was shown to grow for larger ensemble sizes. Reliability diagrams were far less sensitive in this respect. The resulting u-shaped rank histograms can easily be misinterpreted as indicating too little spread in the ensemble prediction system. To account for this effect when real observations are used to assess an ensemble prediction system, normally distributed noise based on the verifying observation error can be added to the ensemble members before the statistics are calculated. The method has been tested for the ECMWF ensemble forecasts of ocean waves and forecasts of the geopotential at 500 hPa. The EPS waves were compared with buoy observations from the Global Telecommunication System (GTS) for a period of almost 3 yr. When the buoy observations were taken as the true value, more than 25% of the observations appeared in the two extreme ranks for the day 3 forecast range. This number was reduced to less than 10% when observation errors were added to the ensemble members. Ensemble forecasts of the 500-hPa geopotential were verified against the ECMWF analysis. When analysis errors were neglected, the maximum number of outliers was more than 10% for most areas except for Europe, where the analysis errors are relatively smaller. Introducing noise to the ensemble members, based on estimates of analysis errors, reduced the number of outliers, particularly in the short range, where a peak around day 1 more or less vanished.


Related Articles

  • Audit sampling with nonsampling errors of the first type. BONETT, DOUGLAS G.; CLUTE, RONALD C. // Contemporary Accounting Research;Spring1990 Part 1, Vol. 6 Issue 2 Pt. 1, p432 

    Standard statistical auditing procedures rest upon the assumption that statistical nonsampling errors do not exist. Three distinct types of statistical nonsampling errors have been identified in the literature. This paper presents new theoretical results regarding the problem of audit sampling...

  • THE ACCURACY MEASUREMENT OF HARDWARE TOOLS METROLOGICAL CHARACTERISTICS FOR SPECTRAL FRACTIONS SIGNAL IDENTIFICATION. Kharchenko, Volodymyr; Larin, Vitaliy; Savitskaya, Iana // Proceedings of National Aviation University;2014, Vol. 61 Issue 4, p76 

    The article gives the accuracy figures quantitative assessment by the example of hardware tools basic main metrological characteristics determination, which allows provide the trustworthiness of signals spectral components in the real time mode. The influence of random error fractions is...

  • Establishing Some Measures of Absolute and Relative Reliability of a Motor Tests. Šerbetar, Ivan // Croatian Journal Educational / Hrvatski Casopis za Odgoj I Obraz;2015 Special Edition 1, Vol. 17, p37 

    Relative and absolute reliability are discussed in this paper on the bases of some empirical motor data. Relative reliability was assessed via the calculation of intraclass correlation coefficients (ICC; Shrout & Fleiss, 1979; Nunnally & Berstein, 1994). Absolute reliability was assessed by...

  • DEPONTO: A Reusable Dependability Domain Ontology. Sanislav, Teodora // TEM Journal;Aug2015, Vol. 4 Issue 3, p223 

    This paper proposes a dependability reusable ontology for knowledge representation. The fundamental knowledge related to dependability follows its taxonomy. Thus, this paper gives an analysis of what is the dependability domain ontology andof its components. The dependability domain ontology...

  • Erratum to: Reliability of the ADI-R for the Single Case-Part II: Clinical Versus Statistical Significance. Cicchetti, Domenic; Lord, Catherine; Koenig, Kathy; Klin, Ami; Volkmar, Fred // Journal of Autism & Developmental Disorders;Jan2015, Vol. 45 Issue 1, p274 

    A correction to the article "Reliability of the ADI-R for the Single Case-Part II: Clinical Versus Statistical Significance" that was published in the online resource' of the November 26, 2013 issue is presented.

  • Double Sampling with Ranked Set Selection in the Second Phase with Nonresponse: Analytical Results and Monte Carlo Experiences. Agarwal, Gaajendra K.; Allende, Sira M.; Bouza, Carlos N. // Journal of Probability & Statistics;2012, p1 

    This paper is devoted to the study of the behavior of the use of double sampling for dealing with nonresponses, when ranked set sample is used. The characteristics of the sampling strategies are derived. The structure of the errors generated the need of studying of the optimality of the...

  • APPLICATION OF HLM TO DATA WITH MULTILEVEL STRUCTURE. Valente, Vítor; Oliveira, Teresa A. // Discussiones Mathematicae: Probability & Statistics;2011, Vol. 31 Issue 1/2, p87 

    Many data sets analyzed in human and social sciences have a multilevel or hierarchical structure. By hierarchy we mean that units of a certain level (also referred micro units) are grouped into, or nested within, higher level (or macro) units. In these cases, the units within a cluster tend to...

  • Rounded data analysis based on multi-layer ranked set sampling. Li, Wei; Bai, Zhi // Acta Mathematica Sinica;Dec2011, Vol. 27 Issue 12, p2507 

    Observations of sampling are often subject to rounding, but are modeled as though they were unrounded. This paper examines the impact of rounding errors on parameter estimation with multi-layer ranked set sampling. It shows that the rounding errors seriously distort the behavior of covariance...

  • ADDITIVE FUNCTIONS AND THEIR APPLICATION TO UNCERTAIN INFORMATION. Kämpke, Thomas // Annals of Operations Research;1991, Vol. 32 Issue 1-4, p51 

    For additive functions over ordered sets, a minimum representation is given in case the ordered set has a particular deviation property. Additive functions are known to be special supermodular functions on power sets and in probability calculus, they are related to, for example, capacities and...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics