Eidgenssisches Departement des Innern EDI Bundesamt fr Meteorologie

Eidgenössisches Departement des Innern EDI Bundesamt für Meteorologie und Klimatologie Meteo. Schweiz Weather type dependant fuzzy verification of precipitation Tanja Weusthoff COSMO General Meeting, Offenbach, 07. -11. 09. 2009

Fuzzy Verification fcst obs • „multi-scale, multi-intensity approach“ • Two methods • Upscaling (UP) • Fraction Skill Score (FSS) • present output scale dependent • standard setting: 3 h accumulations, Jun-Nov 2007, vs Swiss radar data COSMO-2 (2. 2 km): leadtimes 03 -06 COSMO-7 (6. 6 km): leadtimes 03 -06, 06 -09, 09 -12, 12 -15 Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff increasing box size • „Fuzzy verification toolbox“ of B. Ebert increasing threshold 2

… Upscaling (UP) (Atger, 2001) 1. Principle: Define box around region of interest and calculate the average of observation and forecast data within this box. 2. Contingency Table Event No-Event Rave if if Rave ≥ threshold Rave < threshold observation yes no yes Hit False Alarm no Miss Correct negative forecast 3. Equitable Threat Score (ETS) Q: Which fraction of observed yes - events was correctly forecast? Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff 3

… Fraction Skill Score (FSS) (Roberts and Lean, 2005) 1. Principle: Define box around region of interest and determine the fraction pj and oj of grid points with rain rates above a given threshold. 2. Probabilities X X X 0 < pj < 1 = fraction of fcst grid points > threshold 0 < oj < 1 = fraction of obs grid points > threshold x X X X x 3. Skill Score for Probabilities Q: On which spatial scales does the forecast resemble the observation? FBS worst no colocation of non-zero fractions Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff 4

… Fraction Skill Score (FSS) (Roberts and Lean, 2005) 4. Useful Scales useful scales are marked in bold in the graphics Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff 5

COSMO-2 = COSMO-7 - Upscaling Fractions skill score Upscaling and Fractions Skill Score bad Difference = good Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff COSMO-7 better COSMO-2 better 7

COSMO-2 vs. COSMO-7, bootstrapping Values = Score of COSMO-2 abs(median) / 0. 5(q 95 -q 05) [ 10 , Inf ] [ 5 , 10 [ Size of numbers = abs(Median) / 0. 5(q 95 -q 05) measure for significance of differences [2, 5[ [1, 2[ [0, 1[ COSMO-2 better D I F F E R E N C E S COSMO-7 better Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff q(50%) q(5%) q(95%) ¦ COSMO-2 – COSMO-7 ¦ 5% 95% 8

Sensitivities Only 00 and 12 UTC model runs COSMO-2 & COSMO-7: leadtimes 3 -6, 6 -9, 9 -12, 12 -15 absolute values of COSMO-2 slightly lower, but still the same pattern of differences COSMO-2 better D I F F E R E N C E S COSMO-7 better Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff 9

Weather type verification: COSMO-2 Upscaling Window size: 3 gp, 6. 6 km Window size: 27 gp, 60 km Fractions Skill Score Window size: 3 gp, 6. 6 km Window size: 27 gp, 60 km Threshold [mm/3 h] # cases 4 7 5 5 7 45 31 23 15 34 7 Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff 10

Smallest spatial scale [gridpoints] where the forecast has been useful regarding to FSS „useful scales“ definition. COSMO-2, gridpoints* 2. 2 km dx 0. 1 0. 2 0. 5 1. 0 2. 0 5. 0 10 20 7 NE 45 45 45 / / / NE 15 / / / / 4 N 45 45 / / / N / / / / 23 NW 3 3 9 9 27 27 / / NW 3 3 5 5 9 15 / / 5 SE 1 1 9 / / / SE 1 1 1 3 9 / / / 7 S 1 1 1 9 / / S 1 1 1 3 3 9 / / 45 SW 1 1 3 9 9 45 / / SW 3 3 5 9 9 / / / 5 E 9 15 27 / / 45 E 15 / / / / 31 W 1 1 3 9 15 / / / W 3 3 3 5 9 / / / 15 F 15 15 27 27 45 / / / F 9 9 15 15 / / 34 H 27 27 27 45 / / H / 15 / / / 7 L 3 9 15 27 45 / / / L 3 5 9 9 15 / / / 2 0. 5 16 14 10 0. 4 <0. 1 # cases + + - COSMO-7, gridpoints* 6. 6 km 16 14 10 7 5 <0. 1 % obs gridpts >= thresh (whole period) 7 5 2

Frequency of Weather Classes, June – November 2007 45 34 31 23 15 4 7 5 5 7 Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff 7 13

Northerly Winds (NE, N) COSMO-2 (wc) vs. COSMO-2 (all) 11 days COSMO-2 vs. COSMO-7 COSMO-2 (wc) better D I F F E R E N C E S COSMO-2 (all) better COSMO-2 in northerly wind situations clearly worse than over whole period. Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff COSMO-7 better 14

Southerly Winds (SE, S) COSMO-2 (wc) vs. COSMO-2 (all) 12 days COSMO-2 vs. COSMO-7 COSMO-2 (wc) better D I F F E R E N C E S COSMO-2 (all) better COSMO-2 in southerly wind situations clearly better than over whole period. Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff COSMO-7 better 15

Northwesterly winds (NW) COSMO-2 (wc) vs. COSMO-2 (all) 23 days COSMO-2 vs. COSMO-7 COSMO-2 (wc) better COSMO-2 better D I F F E R E N C E S COSMO-2 (all) better COSMO-2 in nothwesterly wind situations at large thresholds clearly better than over whole period. Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff COSMO-7 better 16

Flat (F) COSMO-2 (wc) vs. COSMO-2 (all) 15 days COSMO-2 vs. COSMO-7 COSMO-2 (wc) better COSMO-2 better D I F F E R E N C E S COSMO-2 (all) better COSMO-2 in flat pressure situations clearly worse than over whole period. Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff COSMO-7 better 17

Summary & Outlook • Fuzzy verification of the 6 months period in 2007 has shown that COSMO-2 generally performed better than COSMO-7 on nearly all scales • part of this superiority is caused by the higher update frequency, but using same model runs still shows the same pattern of differences • the results for different weather types show large variations • • • best results were found for southerly winds and winds from Northwest and West, Northeasterly winds as well as Flat pressure situations lead to worse perfomance of both models operational Fuzzy verification is about to start, including Upscaling and Fraction Skill Score Weather type dependant fuzzy verification | COSMO GM, 07. -11. 09. 2009 Tanja Weusthoff 18