Information-Based Skill Scores for Probabilistic Forecasts

Ahrens, Bodo; Walser, André
January 2008
Monthly Weather Review;Jan2008, Vol. 136 Issue 1, p352
Academic Journal
The information content, that is, the predictive capability, of a forecast system is often quantified with skill scores. This paper introduces two ranked mutual information skill (RMIS) scores, RMISO and RMISY, for the evaluation of probabilistic forecasts. These scores are based on the concept of mutual information of random variables as developed in information theory. Like the ranked probability skill score (RPSS)—another and often applied skill score—the new scores compare cumulative probabilities for multiple event thresholds. The RMISO quantifies the fraction of information in the observational data that is explained by the forecasts. The RMISY quantifies the amount of useful information in the forecasts. Like the RPSS, the new scores are biased, but they can be debiased with a simple and robust method. This and additional promising characteristics of the scores are discussed with ensemble forecast assessment experiments.


Related Articles

  • Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation. Gneiting, Tilmann; Raftery, Adrian E.; Westveld III, Anton H.; Goldman, Tom // Monthly Weather Review;May2005, Vol. 133 Issue 5, p1098 

    Ensemble prediction systems typically show positive spread-error correlation, but they are subject to forecast bias and dispersion errors, and are therefore uncalibrated. This work proposes the use of ensemble model output statistics (EMOS), an easy-to-implement postprocessing technique that...

  • Increasing the Reliability of Reliability Diagrams. Bröcker, Jochen; Smith, Leonard A. // Weather & Forecasting;Jun2007, Vol. 22 Issue 3, p651 

    The reliability diagram is a common diagnostic graph used to summarize and evaluate probabilistic forecasts. Its strengths lie in the ease with which it is produced and the transparency of its definition. While visually appealing, major long-noted shortcomings lie in the difficulty of...

  • On Using “Climatology” as a Reference Strategy in the Brier and Ranked Probability Skill Scores. Mason, Simon J. // Monthly Weather Review;Jul2004, Vol. 132 Issue 7, p1891 

    The Brier and ranked probability skill scores are widely used as skill metrics of probabilistic forecasts of weather and climate. As skill scores, they compare the extent to which a forecast strategy outperforms a (usually simpler) reference forecast strategy. The most widely used reference...

  • Evaluation of probabilistic precipitation forecast determined from WRF forecasted amounts. Tanessong, Roméo; Igri, P.; Vondou, Derbetini; Tamo, P.; Kamga, F. // Theoretical & Applied Climatology;May2014, Vol. 116 Issue 3-4, p649 

    This paper examines the connection between the probability of precipitation and forecast amounts from Weather Research and Forecasting (WRF) model runs over Central and West Africa. A one season period (June-September 2010) was used to investigate the quantitative precipitation...

  • Verification of Categorical Probability Forecasts. Zhang, H.; Casey, T. // Weather & Forecasting;Feb2000, Vol. 15 Issue 1, p80 

    This paper compares a number of probabilistic weather forecasting verification approaches. Forecasting skill scores from linear error in probability space and relative operating characteristics are compared with results from an alternative approach that first transforms probabilistic forecasts...

  • Distributions-Oriented Verification of Probability Forecasts for Small Data Samples. Bradley, A. Allen; Hashino, Tempei; Schwartz, Stuart S. // Weather & Forecasting;Oct2003, Vol. 18 Issue 5, p903 

    The distributions-oriented approach to forecast verification uses an estimate of the joint distribution of forecasts and observations to evaluate forecast quality. However, small verification data samples can produce unreliable estimates of forecast quality due to sampling variability and...

  • Extended Prediction of North Indian Ocean Tropical Cyclones. Belanger, James I.; Webster, Peter J.; Curry, Judith A.; Jelinek, Mark T. // Weather & Forecasting;Jun2012, Vol. 27 Issue 3, p757 

    This analysis examines the predictability of several key forecasting parameters using the ECMWF Variable Ensemble Prediction System (VarEPS) for tropical cyclones (TCs) in the North Indian Ocean (NIO) including tropical cyclone genesis, pregenesis and postgenesis track and intensity projections,...

  • Sensitivity of WRF Forecasts for South Florida to Initial Conditions. Etherton, Brian; Santos, Pablo // Weather & Forecasting;Aug2008, Vol. 23 Issue 4, p725 

    This study presents results from an experiment conducted to measure the impact of locally initializing a numerical weather prediction model on that model’s ability to predict precipitation and other surface parameters. The study consisted of quantifying the impact of initializing the...

  • Predictability of Heavy Precipitation in the Waikato River Basin of New Zealand. Dravitzki, Stacey; McGregor, James // Monthly Weather Review;Jul2011, Vol. 139 Issue 7, p2184 

    This paper investigates the predictability of heavy precipitation in the economically important Waikato River basin of New Zealand. A 2-yr archive of Global Forecast System (GFS) model data to ++180 h for the period August 2005--August 2007 forms the basis of the study. GFS model predictions of...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics