Subseasonal skill score based on the historical performance of each model and their multi-model ensemble.

The different skill scores are mapped by calendar month. The forecasts lead times are combined over the weeks 2-3 and the weeks 3-4 from the forecast start time (i.e., 14-day long periods respectively 8 to 21 days and 15 to 28 days after the forecast is issued). Forecasts skill scores combine start times by calendar month and across years 1999 to 2010.

The probabilistic forecasts shown here are obtained from the statistical calibration of three models (ECMWF, NCEP CFSv2 and CMA) from the Subseasonal to Seasonal (S2S) Prediction Project database (Vitart et al, 2017) which are combined with equal weight to form multi-model ensemble precipitation tercile probabilities forecasts. Individual model forecasts are calibrated separately for each point, start and lead using Extended Logistic Regressions (ELR; Vigaud et al, 2017) based on the historical performance of each model, and thus provide reliable intra-seasonal climate information in regards to a wide range of climate risk of concerns to the decision making communities and for which subseasonal forecasts are particularly well suited.

These skill scores diagnostics maps give a sense of where and when (issued which months of the year and for which weekly lead times) subseasonal forecasts may have the potential to provide useful information.

The actual forecasts, of which these skill scores are measuring the historical performance, are to be found in the Experimental Precipitation Subseasonal Forecast Maproom.

**Skill scores definitions:**

**RPSS**: Ranked Probability Skill Scores (RPSS; Epstein (1969); Murphy (1969, 1971); Weigel et al. (2007)) are used to quantify the extent to which the calibrated predictions are improved compared to climatological frequencies. RPSS values tend to be small, even for skillful forecasts. The approximate relationship between RPSS and correlation being such that a RPSS value of 0.1 corresponds to a correlation of about 0.44 (Tippett et al. 2010).**Spearman Ranked Correlation**: the Spearman Anomalies Correlation Coefficient corresponds to the ranked correlation between MME forecasts and observed anomalies, which is particularly appropriate to verify probabilistic forecast**ACC**: the Anomalies Correlation Coefficient is the correlation between MME forecasts and observed anomalies.

**References:**

- Epstein, E.S., 1969: A Scoring System for Probability Forecasts of Ranked Categories. J. Appl. Meteor., 8, 985–987
- Murphy, A.H., 1969: On the “Ranked Probability Score”. J. Appl. Meteor., 8, 988–989
- Murphy, A.H., 1971: A Note on the Ranked Probability Score. J. Appl. Meteor., 10, 155–156
- Tippett, M.K., A.G. Barnston, and T. DelSole, 2010: Comments on “Finite Samples and Uncertainty Estimates for Skill Measures for Seasonal Prediction”. Mon. Wea. Rev., 138, 1487–1493
- Vitart, F., C. Ardilouze, A. Bonet, A. Brookshaw, M. Chen, C. Codorean, M. Déqué, L. Ferranti, E. Fucile, M. Fuentes, H. Hendon, J. Hodgson, H. Kang, A. Kumar, H. Lin, G. Liu, X. Liu, P. Malguzzi, I. Mallas, M. Manoussakis, D. Mastrangelo, C. MacLachlan, P. McLean, A. Minami, R. Mladek, T. Nakazawa, S. Najm, Y. Nie, M. Rixen, A.W. Robertson, P. Ruti, C. Sun, Y. Takaya, M. Tolstykh, F. Venuti, D. Waliser, S. Woolnough, T. Wu, D. Won, H. Xiao, R. Zaripov, and L. Zhang, 2017: The Subseasonal to Seasonal (S2S) Prediction Project Database. Bull. Amer. Meteor. Soc., 98, 163–173
- Vigaud, N., A.W. Robertson, and M.K. Tippett, 2017: Multimodel Ensembling of Subseasonal Precipitation Forecasts over North America. Mon. Wea. Rev., 145, 3913–3928
- Weigel, A.P., M.A. Liniger, and C. Appenzeller, 2007: The Discrete Brier and Ranked Probability Skill Scores. Mon. Wea. Rev., 135, 118–124

**Forecasts Skill Scores:** Global 1˚ Multi-Model Ensemble forecasts skill scores per month of the year over
the period 1999-2010 available here.

Contact help@iri.columbia.edu with any technical questions or problems with this Map Room.