Federal Department of Home Affairs FDHA Federal Office

  • Slides: 27
Download presentation
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology Meteo. Swiss

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology Meteo. Swiss Priority project « Advanced interpretation and verification of very high resolution models »

Topics 1. Advanced postprocessing of weather parameters 2. Verification of very high resolution models,

Topics 1. Advanced postprocessing of weather parameters 2. Verification of very high resolution models, incl. fuzzy verification methods 3. Hydrological applications COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 2

3. Hydrological applications Hydrology (precipitation adaptation): Presentation by A. Mazur Snow parametrisation: Presentation by

3. Hydrological applications Hydrology (precipitation adaptation): Presentation by A. Mazur Snow parametrisation: Presentation by E. Machulskaya COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 3

1. Recognition of weather elements • Done last year: recognition of thunderstorms with the

1. Recognition of weather elements • Done last year: recognition of thunderstorms with the boosting algorithm: • Choice of predictors Perler, Kohli, Walser COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 4

1. Kalman filtering of COSMO LEPS V. Stauch, poster outside COSMO General meeting ¦

1. Kalman filtering of COSMO LEPS V. Stauch, poster outside COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 5

2. Verification of very high resolution models Goals • • • 1 -3 km

2. Verification of very high resolution models Goals • • • 1 -3 km scale (VHR) Focus on precipitation Is VHR (~2 km) better than HR (~7 km)? Model intercomparison Generate products related to the verification Way to define the scores could depend on the application (value) • Use synop, (high resolution rainguage network), radar, evt. composition of all (gridded observations) COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 6

Motivation Which rain forecast would you rather use? Mesoscale model (5 km) 21 Mar

Motivation Which rain forecast would you rather use? Mesoscale model (5 km) 21 Mar 2004 Global model (100 km) 21 Mar 2004 Sydney RMS=13. 0 Observed 24 h rain Sydney RMS=4. 6 B. Ebert COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 7

Motivation: precipitation pattern 2 km 7 km COSMO General meeting ¦ Cracow, September 2008

Motivation: precipitation pattern 2 km 7 km COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 8

Fuzzy Verification F. Ament Method Raw Data Fuzzyfication Score Example result Average X Upscaling

Fuzzy Verification F. Ament Method Raw Data Fuzzyfication Score Example result Average X Upscaling X X X Equitable threat score x X X X x Fractional coverage Fraction Skill Score (Roberts and Lean, 2005) X X X x X X X Skill score with reference to worst forecast x Verification on coarser scales than model scale: “Do not require a point wise match!“ COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 9

Expected behaviour of scores From Nigel Roberts (2005) COSMO General meeting ¦ Cracow, September

Expected behaviour of scores From Nigel Roberts (2005) COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 10

Application of scores to a perfect forecast All scores should equal ! But, in

Application of scores to a perfect forecast All scores should equal ! But, in fact, 5 out of 12 do not! COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 11

Requested theoretical properties of scores J J J Avoid « leaking » scores Use

Requested theoretical properties of scores J J J Avoid « leaking » scores Use illustrative and understandable scores Score should give a real information of the forecast quality on the different scales Monotonic behavior concerning • scale (best values for large scales) • frequency of occurrence (best values for high frequencies of occurrence) Represent some significant characteristics of the PDF (obs and forecast) COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 12

Requested practical properties of scores J J J Agreement between subjective and objective judgment

Requested practical properties of scores J J J Agreement between subjective and objective judgment Possible help in decision making Correspond to the needs of the users Should be able to provide a comparison between 2 km and 7 km models (also global models) Should not use a matching between prediction and observation because it would not allow the generation of univocal products COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 13

Chosen scores Our best candidates: Upscaling and Fraction skill score Corresponding products • Upscaling

Chosen scores Our best candidates: Upscaling and Fraction skill score Corresponding products • Upscaling mean around a point / station • Fraction skill score probability to exceed some threshold in a neighbourhood COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 14

Fuzzy Verification: COSMO-DE – COSMO-EU 90 Fraction skill score COSMO-DE (2. 8 km) =

Fuzzy Verification: COSMO-DE – COSMO-EU 90 Fraction skill score COSMO-DE (2. 8 km) = 20 7 Difference COSMO-EU (7 km) - 33 90 = 58 33 20 Spatial scale (km) Upscaling - 58 Spatial scale (km) JJA 2007, Verification against Swiss Radar Composite, 3 hourly accumulations 7 Threshold (mm/3 h) bad Threshold (mm/3 h) good COSMO-EU better COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch COSMO-DE better 15

Fuzzy Verification COSMO-2 – COSMO-7 90 Fraction skill score COSMO-2 (2. 2 km) =

Fuzzy Verification COSMO-2 – COSMO-7 90 Fraction skill score COSMO-2 (2. 2 km) = 20 7 Difference COSMO-7 (7 km) - 33 90 = 58 33 20 7 Threshold (mm/3 h) bad Threshold (mm/3 h) good COSMO-7 better COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch COSMO-2 better 16 Spatial scale (km) Upscaling - 58 Spatial scale (km) JJA 2007, Verification against Swiss Radar Composite, 3 hourly accumulations

Monthly dependency cut-off 03 h, accumulation 03 h COSMO-2 - COSMO-7 COSMO-DE - COSMO-EU

Monthly dependency cut-off 03 h, accumulation 03 h COSMO-2 - COSMO-7 COSMO-DE - COSMO-EU June July August COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 17

Quarterly summaries of „Fuzzy“-scores FSS Autumn 2007 U. Damrath COSMO General meeting ¦ Cracow,

Quarterly summaries of „Fuzzy“-scores FSS Autumn 2007 U. Damrath COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 18

Monthly summaries of „Fuzzy“-scores FSS July 2007 COSMO General meeting ¦ Cracow, September 2008

Monthly summaries of „Fuzzy“-scores FSS July 2007 COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 19

Analysis of precipitation in boxes We devised a verification methodology by aggregating observed and

Analysis of precipitation in boxes We devised a verification methodology by aggregating observed and predicted precipitation in boxes of 1°x 1° (labelled boxes in the map) The choice of the size and position of the areas has been performed according to different rules: Average number of stations in each area ( SON 2007) X • boxes have to be enough large in order to contain a high number of observation points (ranging from 20 to over 100, depending on location and period of time considered) • boxes have to be homogeneous as much as possible in terms of geographic-territorial characteristics M. -S. Tesini C. Cacciamani 20

Box 2 aut 2007 90 th percentile of “climatological” pdf 19 mm/24 23 mm/24

Box 2 aut 2007 90 th percentile of “climatological” pdf 19 mm/24 23 mm/24 25 mm/24 21

Consideration on “day-by-day” behaviour • COSMO-I 7 seems to be more realistic than ECMWF

Consideration on “day-by-day” behaviour • COSMO-I 7 seems to be more realistic than ECMWF in reproducing the intra-box variability. • However, COSMO-I 7 presents both a large number of false alarms and high “spikes”. On the other hand, ECMWF presents a greater number of missed alarms, especially for high thresholds. • According to most standard verification measures, COSMO-I 7 forecast would have poor quality, but it might be very valuable to the forecaster since it provides information on the distribution and variability of the rain field over the considered region. 22

Neighbourhood method P. Kaufmann • Cylindrical neighbourhood with fading zone • Settings at Meteo.

Neighbourhood method P. Kaufmann • Cylindrical neighbourhood with fading zone • Settings at Meteo. Swiss: • COSMO-7 (6. 6 km): rxy= 5, rf= 5, rt=3 • COSMO-2 (2. 2 km): rxy=10, rf=10, rt=1 • Effective radius: • COSMO-7: ~50 km • COSMO-2: ~35 km COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch t y x 23

12 July: high probabilities match well with precipitation pattern Probability of 12 h sum

12 July: high probabilities match well with precipitation pattern Probability of 12 h sum above 35 mm 06 – 18 UTC 18 – 06 UTC 24 h sum 06 – 06 UTC next day COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 24

15 August: high probabilities match well precipitation pattern Probability of 12 h sum above

15 August: high probabilities match well precipitation pattern Probability of 12 h sum above 35 mm 06 – 18 UTC 18 – 06 UTC 24 h sum 06 – 06 UTC next day COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 25

17 July: completely missed event Probability of 12 h sum above 35 mm 06

17 July: completely missed event Probability of 12 h sum above 35 mm 06 – 18 UTC 18 – 06 UTC 24 h sum 06 – 06 UTC next day COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 26

Conclusions on verification of very high resoution models • Results of Upscaling and Fraction

Conclusions on verification of very high resoution models • Results of Upscaling and Fraction skill score are reasonable. • Scores increase with box size, but it is difficult to extract optimal size by looking at one single model. • Overall better results for very high-res models • This benefits of very high-res models is rather to see in situations where precipitation variability is large: convection , orography, summer… • …and at scales of 30 to 50 km • Products can be generated • Regional means (not new) • Probability to exceed threshold in neighborhood • Or possibly the whole pdf? COSMO General meeting ¦ Cracow, September 2008 Pierre. Eckert[at]meteoswiss. ch 27