Precipitation Seasonal Hindcast Skill

Seasonal skill score based on the historical performance of each calibrated NMME model and their multimodel ensemble (1991-2020).

Skill is mapped by calendar month for seasonal lead times. Lead 1 = months 2-4, Lead 2 = months 3-5, Lead 3 = months 4-6, Lead 4 = months 5-7 after the forecast is issued. Forecasts skill scores combine start times by calendar month and across years 1991 to 2020. The observational reference datasets are CMAP-URD for precipitation and GHCN-CAMS for temperature. These were computed in May 2023, using the calibrated MME comprising the NCEP-CFSv2, CanSIPS-IC3, COLA-RSMAS-CCSM4, and GFDL-SPEAR, and NASA-GEOSS2S models. The climatological base period for normal is 1991-2020.

These skill scores diagnostics maps give a sense of where and when (issued which months of the year and for which seasonal lead times) the probabilistic seasonal forecasts have the potential to provide useful information, based on hindcasting.

Skill scores definitions:

• RPSS: Ranked Probability Skill Scores (RPSS; Epstein (1969); Murphy (1969, 1971); Weigel et al. (2007)) are used to quantify the extent to which the calibrated tercile-category predictions are improved compared to climatological frequencies. RPSS values tend to be small, even for skillful forecasts. The approximate relationship between RPSS and correlation being such that a RPSS value of 0.1 corresponds to a correlation of about 0.44 (Tippett et al. 2010).

References:

• Epstein, E.S., 1969: A Scoring System for Probability Forecasts of Ranked Categories. J. Appl. Meteor., 8, 985–987
• Murphy, A.H., 1969: On the “Ranked Probability Score”. J. Appl. Meteor., 8, 988–989
• Murphy, A.H., 1971: A Note on the Ranked Probability Score. J. Appl. Meteor., 10, 155–156
• Tippett, M.K., A.G. Barnston, and T. DelSole, 2010: Comments on “Finite Samples and Uncertainty Estimates for Skill Measures for Seasonal Prediction”. Mon. Wea. Rev., 138, 1487–1493
• Weigel, A.P., M.A. Liniger, and C. Appenzeller, 2007: The Discrete Brier and Ranked Probability Skill Scores. Mon. Wea. Rev., 135, 118–124

Dataset Documentation

Forecasts Skill Scores: Global 1˚ Multi-Model Ensemble forecasts skill scores per month of the year over the period 1982-2010 available here.

Instructions

Helpdesk

Contact help@iri.columbia.edu with any technical questions or problems with this Map Room.