Preliminary report on verification scores for malaria and
- Slides: 37
Preliminary report on verification scores for malaria and the climate drivers DEMETER to ENSEMBLES Andy Morse, Cyril Caminade and Anne Jones Department of Geography, University of Liverpool, United Kingdom A. P. Morse@liv. ac. uk WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
Running Order • Introduction • Malaria verification – ENSEMBLES stream II – limited area • Climate verification – ENSEMBLES stream II - all Africa • Summary and Way Forward WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
Introduction • Normally use BSS and ROCSS to work with probabilistic data • Running RMSE, ACC and MSSS deterministically on ensemble mean • Developed/modified scripts for skill scores in NCAR Command Language (NCL) – share & cross validate? • Work not complete – will report via ENSEMBLES website WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
Introduction – ROCSS malaria modelling Tier-2 malaria runs - ROC Skill Scores Above Median Event May 4 -6 JAS Nov 4 -6 FMA DEMETER driven LMM. Areas of high interannual variability were selected and persisted forecast skill was removed from the scores. Jones, A. and Morse, A. (2007) CLIVAR Exchanges, 43 WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
Validation of Liverpool Malaria Model (LMM) runs forced by ENSEMBLES stream 2 seasonal forecasts WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Table of experiments: LMM dynamic malaria mode forced with daily Tx and pptn. The domain: Botswana: [27. 5°S-17. 5°S] [17. 5°E-30°E] Expid name Project Institution/Model Period Start date (lead months) Ensemble members ecmf ENSEMBLES 9 ENSEMBLES Nov 1 (14) 9 lfpw ENSEMBLES Nov 1 (14) 9 ingv ENSEMBLES Nov 1 (7) 9 ifmk ENSEMBLES 19602006 19702005 19602005 Nov 1 (14) egrr ECMWF IFSHOPE UK Met-Office Had. Gem 2 Meteo-France ARPEGE 4/OPA INGV ECHAM 5/OPA 8. 2 Kiel ECHAM 5/OM 1 Nov 1 (14) 9 WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Scores: • ACC: anomaly correlation coefficients • Addresses the question: How well did the forecast anomalies correspond to the observed anomalies? • RMSE: Root mean square errors • • • Answers the question: What is the average magnitude of the forecast errors? MSSS: Mean square skill score with WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Results 1961 -2000 NDJF: RMSE for Malaria Incidence over Botswana (NDJF 1961 -2000). The reference dataset is a companion run forced by ERA 40 reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Results 1961 -2000 NDJF: ACC for Malaria Incidence over Bostwana (NDJF 1961 -2000). The reference dataset is a companion run forced by ERA 40 reanalysis, and the analysis is applied to the ensemble means. The dotted area display WGSIP 12 – Miami - January 2009 –correlations Impacts verification - Andy significant at the 95%Morse confidence interval
ENSEMBLES –stream 2 Results 1961 -2000 NDJF: MSSS for Malaria Incidence over Botswana (NDJF 1961 -2000). The reference dataset is a companion run forced by ERA 40 reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Results 1981 -2000 NDJF: RMSE for Malaria Incidence over Botswana (NDJF 1981 -2000). The reference dataset is a companion run forced by ERA 40 reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Results 1981 -2000 NDJF: ACC for Malaria Incidence over Botswana (NDJF 1981 -2000). The reference dataset is a companion run forced by ERA 40 reanalysis, and the analysis is applied to the ensemble means. The dotted area display significant correlations at the 95% confidence interval WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Results 1981 -2000 NDJF: MSSS for Malaria Incidence over Bostwana (NDJF 1981 -2000). The reference dataset is a companion run forced by ERA 40 reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
Validation climate drivers ENSEMBLES stream 2 1981 -2000 seasonal forecasts WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Results 1981 -2000 NDJF: RMSE for tasmax (NDJF 1981 -2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Results 1981 -2000 NDJF: ACC for tasmax (NDJF 1981 -2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Results 1981 -2000 NDJF: MSSS for tasmax (NDJF 1981 -2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Results 1981 -2000 NDJF: RMSE for prlr (NDJF 1981 -2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Results 1981 -2000 NDJF: ACC for prlr (NDJF 1981 -2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 2 Results 1981 -2000 DJF: MSSS for prlr (NDJF 1981 -2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
Summary & Way Forward? • Experience in BSS and ROCSS (probabilistic) • Question non probabilistic nature/use of RMSE, ACC and MSSS for EPS verification • All data should come with sets of verification plots e. g. ECMWF DEMETER http: //www. ecmwf. int/research/demeter/d/charts/verification • Library of scripts? Or portal to where they exsist WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
Questions? WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
Validation climate drivers ENSEMBLES stream 1 1991 -2001 seasonal forecasts also have for DEMETER equivalent years WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 DJF: RMSE for tas (DJF 1991 -2000). The reference dataset is a ERA 40 reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 DJF: RMSE for tas (DJF 1991 -2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 DJF: ACC for tas (DJF 1991 -2000). The reference dataset is a ERA 40 reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 DJF: ACC for tas (DJF 1991 -2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 DJF: MSSS for tas (DJF 1991 -2000). The reference dataset is a ERA 40 reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 DJF: MSSS for tas (DJF 1991 -2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 DJF: RMSE for prlr (DJF 1991 -2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 DJF: ACC for prlr (DJF 1991 -2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 DJF: MSSS for prlr (DJF 1981 -2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 JAS: RMSE for prlr (JAS 1991 -2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 JAS: ACC for prlr (JAS 1991 -2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
ENSEMBLES –stream 1 Results 1991 -2000 JAS: MSSS for prlr (JAS 1991 -2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
WGSIP 12 – Miami - January 2009 – Impacts verification - Andy Morse
- Typologies are nominal composite measures
- Expenditure verification report
- Trypanosoma bruzi
- University of pecs faculty of health sciences
- Malaria and dengue
- Malaria and dengue
- Waterfall strategy
- Preliminary cooking and flavoring
- Enlargement of a lymphoid organ in the luq
- What is hyposplenism
- Dot
- Primary prevention of poliomyelitis
- Obat malaria dhp
- Schüffner's dots vs maurer's
- Malaria prophylaxis
- Apicomplex
- Etiologia de malaria
- Gold standard malaria
- Imep full form in malaria
- Malaria in uganda facts
- Falciparum malaria icd 10
- Complicated malaria
- Malaria in pregnancy definition
- Malaria in pregnancy definition
- Epidemiological triad
- Makalah surveilans malaria
- Malaria
- Malaria
- Malaria references
- Malaria life cycle
- Ciclo biológico plasmodium
- History of malaria
- Black water malaria
- Schistomomiasis
- Malaria sequestration
- Black water fever cause
- Malaria
- Malaria