Comparison of Scores in Model Comparison MSSS COR
- Slides: 16
Comparison of Scores in Model Comparison (MSSS, COR, RMSE and ROC) Tomoaki OSE (MRI/JMA) With help of Yukiko NARUSE (CPD/JMA) and use of JMA/MRI CGCM and AGCM Hindcasts
MSSS, COR and RMSE (from SVS_LRF of WMO manual)
Dependence of MODEL comparisons on SCORES FOR RAIN ANOMALY OF NH SUMMER PREDICTION FROM NH WINTER
Originally defined MSSS is not useful at current Seasonal Forecasts CGCM MSSS AGCM MSSS CGCM BIAS AGCM BIAS
Better COR does not necessarily correspond to better MSSS. CGCM MSSS AGCM MSSS CGCM COR AGCM COR
MSSS is largely influenced by model SDs. CGCM MSSS AGCM MSSS CGCM SDmdl/ SDanal AGCM SDmdl/ SDanal
The change of RMSE is more similar to the change of MSSS than COR. CGCM RMSE AGCM RMSE CGCM COR AGCM COR
Comparison of SCOREs: ROC map seems to be more similar to COR map. CGCM ROC CGCM MSSS CGCM COR CGCM RMSE
Summary • It is confirmed that better COR does not necessarily indicate better MSSS in model comparison. • Because MSSS is definitely dependent on model SD (and/or BIAS) in addition to COR. • The change of RMSE shows similar characteristics to the change of MSSS. • Small RMSE regions do not correspond to the regions for good seasonal forecasts. • COR map seems to show more similar distribution to ROC map than MSSS map.
Why COR for SIF modelers? • Temporal correlation is the most fundamental skill measure for SI forecasts predictability. The square of correlation is the ratio of forecast signal variance to the total variance. • BIAS, SD, MSSS and RMSE can be corrected by linear transform using OBS while correlation is invariant. • Correlation is widely used and most users are familiar with it and easy to understand. • In order to evaluate the value of probability forecast for particular user’s application, more sophisticated skill measure may be needed anyway for each.
Dependence of MODEL comparisons on SCORES FOR T 2 M ANOMALY OF NH WINTER PREDICTION FROM NH SUMMER
Comparison of SCOREs: ROC map seems to be more similar to COR map. CGCM ROC CGCM MSSS CGCM COR CGCM RMSE
Dependence of MODEL comparisons on SCORES FOR RAIN ANOMALY OF NH WINTER PREDICTION FROM NH SUMMER
Comparison of SCOREs: ROC map seems to be more similar to COR map. CGCM ROC CGCM MSSS CGCM COR CGCM RMSE
Dependence of MODEL comparisons on SCORES FOR T 2 M ANOMALY OF NH SUMMER PREDICTION FROM NH WINTER
Comparison of SCOREs: ROC map seems to be more similar to COR map. CGCM ROC CGCM MSSS CGCM COR CGCM RMSE
- Scale scores convey more information than index scores.
- Punctuation formulas
- Comparison test series
- P25 z score
- Gwar sdsu
- Pre act scores
- Fldoe learning gains
- Ielts presentation
- Vemat scores
- Lexile range 1170l-1320l
- Armed services vocational aptitude battery definition
- Virginia sol scoring chart
- Reading inventory scores by grade level
- Reading inventory scores
- Rias memory
- Reading lexile levels
- Grouped data frequency distribution table