SPC PreImplementation Evaluation of the GFS Israel Jirak

SPC Pre-Implementation Evaluation of the GFS Israel Jirak, Andy Dean, and Steven Weiss February 29, 2016

SPC GFS Evaluation Overview • Thanks to the Global Modeling Branch at EMC for making data from retrospective runs available on WCOSS • The evaluation by SPC focused primarily on comparing the 00 Z operational and parallel forecasts: o Objective comparison of low-level moisture and stability characteristics between the GFS and GFSp for 2015 o Objective verification of proxies for thunderstorms and severe weather environments for 2015 o Subjective examination of forecasts for a few severe weather events in 2015 and 2016, especially focused on instability and the vertical thermodynamic profile 2

SPC GFS Evaluation Statistical Evaluation • Objective comparison of low-level moisture and stability characteristics between the GFS and GFSp for 2015 • Objective verification of proxies for thunderstorms and severe weather environments for 2015 3

SPC GFS Evaluation 2 -m Dewpoint Bias: Jan-Dec 2015 over CONUS compared to sfc. OA • For 2015, the low frequency bias of 2 -m dewpoint is reduced in the GFSp at all forecast hours, but a high bias appears for values ≥ 70°F forecasts valid at 00 Z. 4

SPC GFS Evaluation SBCAPE Bias (500 J/kg): Jan-Dec 2015 over CONUS compared to sfc. OA • For 2015, the low frequency bias of SBCAPE ≥ 500 J/kg is slightly improved at all forecast hours. 5

SPC GFS Evaluation SBCAPE Bias (1000 J/kg): Jan-Dec 2015 over CONUS compared to sfc. OA • Improvement in bias for SBCAPE ≥ 1000 J/kg as well, with similar bias pattern for GFS and GFSp. Forecasts valid from 15 -21 Z have very low frequency bias. 6

SPC GFS Evaluation SBCAPE Bias (2000 J/kg): Jan-Dec 2015 over CONUS compared to sfc. OA • For SBCAPE ≥ 2000 J/kg, the frequency bias for the GFSp is improved, but still very low (around 0. 5 on average). 7

SPC GFS Evaluation SBCAPE Bias (3000 J/kg): Jan-Dec 2015 over CONUS compared to sfc. OA • The frequency bias is most improved in the GFSp for SBCAPE values ≥ 3000 J/kg, though still well below 1. 8

SPC GFS Evaluation Proxy Verification • Comparison valid for 00 Z cycle: 20150115 -20151231 • Comparison done using 1 degree forecast grids of GFS over the CONUS for 6 -hr forecasts out to 180 hrs • Thunderstorm Proxy o MUCAPE > 100 J/kg and 6 -hr conv. precip > 0. 01” o Compared against observed CG lightning on 1° grid • Severe Proxy o Supercell comp. param (SCP) > 1, 6 -hr conv precip > 0. 01” o Compared against SPC mesoanalysis severe proxy (SCP >= 1 coincident with observed CG lightning) on 1° grid 9

SPC GFS Evaluation Thunderstorm Proxy Verification GFSp FAR Bias POD • GFSp has reduced high bias. However, lower POD with a similar FAR in the GFSp results in a slightly lower CSI than for the GFS. CSI 10

SPC GFS Evaluation Severe Proxy Verification GFSp FAR Bias • GFSp has a reduced high bias (near 1), but also has a slightly lower POD than the GFS. Overall, the CSI for the GFSp is very slightly lower than or similar to the GFS. POD CSI 11

SPC GFS Evaluation Case Studies • A variety of severe weather events from 2015 and early 2016 were examined to determine the practical impact of the GFSp changes on severe weather fields and products looked at by SPC forecasters: o 6 May 2015 Enhanced Risk for Southern Nebraska into Northern Texas o 13 July 2015 Moderate Risk for Illinois, Indiana, and Western Kentucky o 22 August 2015 Enhanced Risk for central Minnesota into southeastern Nebraska o 21 January 2016 Enhanced Risk for Louisiana and southern Mississippi o 15 February 2016 Slight Risk for southern Louisiana, Mississippi, and Alabama 12

06 May 2015 Enhanced Risk Overview � Numerous hail and tornado reports extending from northern Texas into southern Nebraska. 13

06 May 2015 Enhanced Risk SBCAPE, Shear: 06/00 F 018 Valid 18 Z 06 May – D 1 GFS 2000 J/kg GFSp 2000 J/kg Dashed white line is the approximate contour of MUCAPE = 2000 J/kg as estimated from SPC mesoanalysis (RAP-based) • Forecasts of SBCAPE for the GFS and GFSp are similar for this case, though too 14 low and not far enough west relative to SPC mesoanalysis, which is based on the

06 May 2015 Enhanced Risk LMN Obs/Forecast Sounding: 06/17 & 06/00 F 018 Valid 18 Z 06 May – D 1 GFS OBS SBCAPE = 2076 J/kg EFF. SRH = 115 m 2 s-2 GFSp SBCAPE = 2527 J/kg EFF. SRH = 255 m 2 s-2 SBCAPE = 2127 J/kg EFF. SRH = 125 m 2 s-2 • The thermodynamic profiles are good for this case. The southeasterly winds at the surface are not captured by the GFS or GFSp, however, which results in an underestimate of storm-relative helicity (SRH). 15

13 July 2015 Moderate Risk Overview � Numerous reports in the Midwest, mostly on the periphery of the Moderate Risk area. 16

13 Jul 2015 Moderate Risk SBCAPE, Shear: 12/00 F 042 Valid 18 Z 13 Jul – D 2 GFS 2000 J/kg GFSp 2000 J/kg Dashed white line is the approximate contour of MUCAPE = 2000 J/kg as estimated from SPC mesoanalysis (RAP-based) • GFSp more appropriately has larger areal extent of extreme instability values across the region than the GFS although both models are too far east with the 17

13 Jul 2015 Moderate Risk SBCAPE, Shear: 13/00 F 018 Valid 18 Z 13 Jul – D 1 GFS 2000 J/kg GFSp 2000 J/kg Dashed white line is the approximate contour of MUCAPE = 2000 J/kg as estimated from SPC mesoanalysis (RAP-based) • Day 1 forecast is similar to Day 2 with GFSp revealing higher instability. 18

13 Jul 2015 Moderate Risk DVN Obs/Forecast Sounding: 13/18 & 12/00 F 018 Valid 18 Z 13 Jul – D 1 GFS OBS SBCAPE = 5778 J/kg EFF. SRH = 93 m 2 s-2 GFSp SBCAPE = 5370 J/kg EFF. SRH = 189 m 2 s-2 SBCAPE = 6668 J/kg EFF. SRH = 92 m 2 s-2 • Near the instability axis, thermodynamic profiles are reasonable, though the GFS is too warm and the GFSp is too moist near the surface. Both models have veered surface winds, resulting in SRH values that are too low. 19

13 Jul 2015 Moderate Risk SBCAPE, Shear: 13/00 F 018 Valid 18 Z 13 Jul – D 1 GFS 2000 J/kg GFSp 2000 J/kg Dashed white line is the approximate contour of MUCAPE = 2000 J/kg as estimated from SPC mesoanalysis (RAP-based) • Now looking farther west on the instability gradient. Notice an eastward bias of the unstable warm sector, which still indicates overmixing in the central plains. 20

13 Jul 2015 Moderate Risk LMN Obs/Forecast Sounding: 13/20 & 12/00 F 021 Valid 21 Z 13 Jul – D 1 GFS OBS SBCAPE = 2556 J/kg EFF. SRH = 38 m 2 s-2 GFSp SBCAPE = 6030 J/kg EFF. SRH = 88 m 2 s-2 SBCAPE = 3797 J/kg EFF. SRH = 47 m 2 s-2 • The GFS is too hot and much too dry near the surface, while the GFSp is an improvement. However, it is still off by ~2000 J/kg compared to observed 21

22 Aug 2015 Enhanced Risk Overview � Significant overforecast of severe weather potential from central Minnesota into southeastern Nebraska. 22

22 Aug 2015 Enhanced Risk SBCAPE, Shear: 22/00 F 018 Valid 18 Z 22 Aug – D 1 GFS 2000 J/kg GFSp 2000 J/kg Dashed white line is the approximate contour of MUCAPE = 2000 J/kg as estimated from SPC mesoanalysis (RAP-based) • Very big difference in 18 -h instability forecasts between GFS and GFSp in central 23 Minnesota.

22 Aug 2015 Enhanced Risk MPX Obs/Forecast Sounding: 22/19 & 22/00 F 018 Valid 18 Z 22 Aug – D 1 GFS OBS SBCAPE = 2393 J/kg EFF. SRH = 29 m 2 s-2 GFSp SBCAPE = 2600 J/kg EFF. SRH = 128 m 2 s-2 SBCAPE = 2414 J/kg EFF. SRH = 42 m 2 s-2 • The profiles for the GFS and GFSp look similar in the Minneapolis, MN, area. 24

22 Aug 2015 Enhanced Risk SBCAPE, Shear: 22/00 F 018 Valid 18 Z 22 Aug – D 1 GFS 2000 J/kg GFSp 2000 J/kg Dashed white line is the approximate contour of MUCAPE = 2000 J/kg as estimated from SPC mesoanalysis (RAP-based) • Farther to the southwest, there are large differences in instability. 25

22 Aug 2015 Enhanced Risk MPX Obs/Forecast Sounding: 22/19 & 22/00 F 018 Valid 18 Z 22 Aug – D 1 GFS SBCAPE = 3417 J/kg EFF. SRH = 142 m 2 s-2 GFSp SBCAPE = 2164 J/kg EFF. SRH = 0 m 2 s-2 • The GFS is too warm while the GFSp is too cool. Given the clouds and precip in the area and resultant lack of severe weather, GFSp representation of instability was more appropriate. 26

21 January 2016 Enhanced Risk • Severe Storms Lower Mississippi Valley/Gulf Coast • Several Tornadoes Over MS and LA o EF-2 in Lamar County, MS (30 mile path length) Storm Reports Tornado Damage in Mississippi 27

12 z LCH Observed Sounding Valid 12 z 21 Jan 2016 MUCAPE 1694 J/kg Mean Q 11. 5 G/kg LCH RAOB exhibits several EMLs with inversions near 700 mb and 500 mb and elevated moisture above ground 28

12 z GFS 00 -h LCH Sounding Valid 12 z 21 Jan 2016 MUCAPE 518 J/kg (error ~ 1100 J/kg) Mean Q 10 G/kg GFS analysis sounding smooths inversions/EMLs and is too dry above the surface => limited CAPE 29

12 z GFSp 00 -h LCH Sounding Valid 12 z 21 Jan 2016 MUCAPE 718 J/kg (error ~900 J/kg) Mean Q 10. 3 G/kg GFSp analysis sounding also smooths inversions/EMLs and is too dry above the surface => limited CAPE 30

12 z SHV Observed Sounding Valid 12 z 21 Jan 2016 MUCAPE 1494 J/kg SHV RAOB exhibits elevated instability above nearsurface stable layer 31

12 z GFS 00 -h SHV Sounding Valid 12 z 21 Jan 2016 MUCAPE 606 J/kg (error ~900 J/kg) GFS analysis sounding weakens inversion strength and max theta-e at nose of inversion => lower CAPE 32

12 z GFSp 00 -h SHV Sounding Valid 12 z 21 Jan 2016 MUCAPE 393 J/kg (error ~1100 J/kg) GFSp analysis sounding weakens inversion strength further => even less CAPE 33

RAP and GFS MUCAPE Analyses Valid 12 z 21 Jan 2016 RAP MUCAPE GFS MUCAPE Green Shading Denotes > 1000 J/kg 34

RAP and GFSp MUCAPE Analyses Valid 12 z 21 Jan 2016 RAP MUCAPE GFSp MUCAPE Green Shading Denotes > 1000 J/kg 35

RAP and GFSp MUCAPE Analyses Valid 12 z 21 Jan 2016 GFS MUCAPE GFSp MUCAPE Green Shading Denotes > 1000 J/kg Little Practical Difference Between GFS and GFSp 36

18 z LCH Observed Sounding Valid 18 z 21 Jan 2016 MUCAPE 2541 J/kg LCH RAOB exhibits low-level moisture increase since 12 Z and larger CAPE value 37

LCH: GFS 06 -h Fcst-Raob Overlay Valid 18 z 21 Jan 2016 MUCAPE 1332 J/kg (error ~1200 J/kg) GFS forecast is too cool/dry near surface and near surface winds are veered too much compared to RAOB 38

LCH: GFSp 06 -h Fcst-Raob Overlay Valid 18 z 21 Jan 2016 MUCAPE 1307 J/kg (error ~1200 J/kg) GFSp forecast is very similar to GFS 39

RAP Analysis & GFS Forecast Valid 18 z 21 Jan 2016 RAP MUCAPE Analysis GFS MUCAPE 6 -h Forecast Green Shading Denotes > 1000 J/kg Purple Shading Denotes > 2000 J/kg GFS Forecast Less Supportive of Severe Weather 40

RAP Analysis & GFSp Forecast Valid 18 z 21 Jan 2016 RAP MUCAPE Analysis GFSp MUCAPE 6 -h Forecast Green Shading Denotes > 1000 J/kg Purple Shading Denotes > 2000 J/kg GFSp Forecast Less Supportive of Severe Weather 41

GFS and & GFSp 6 -h Forecasts Valid 18 z 21 Jan 2016 GFS MUCAPE 6 -h Forecast GFSp MUCAPE 6 -h Forecast Green Shading Denotes > 1000 J/kg Little Practical Difference Between GFS and GFSp 42

15 February 2016 Slight Risk • Severe Storms Across Lower Mississippi Valley/Gulf Coast • Primarily Tornadoes/Wind Damage LA, MS, AL, FL o EF-2/EF-3 Tornadoes in southern MS and Northwest FL Storm Reports Tornado Damage in Florida 43

00 z LCH Observed Sounding Valid 00 z 15 Feb 2016 MUCAPE 105 J/kg Mean Q 10. 4 G/kg RAOB exhibits stable layer ~800 mb with minimal CAPE 44

GFS 00 -h LCH Sounding Valid 00 z 15 Feb 2016 MUCAPE 208 J/kg Mean Q 10. 8 G/kg GFS 00 -h sounding is slightly more moist and unstable but generally corresponds well to observed raob 45

GFSp 00 -h LCH Sounding Valid 00 z 15 Feb 2016 MUCAPE 77 J/kg Mean Q 10. 0 G/kg GFS 00 -h sounding is slightly less moist and unstable but generally corresponds well to observed raob 46

12 z LCH Observed Sounding Valid 12 z 15 Feb 2016 MUCAPE 1937 J/kg Mean Q 11. 2 G/kg RAOB exhibits rapid moistening near ground, steepening lapse rates/EMLs and substantial destabilization 47

GFS 12 -h Fcst LCH Sounding Valid 12 z 15 Feb 2016 MUCAPE 790 J/kg (error ~1200 J/kg) GFS 12 -h forecast sounding has general character of RAOB but too cool/dry just above surface resulting in too little CAPE 48

GFSp 12 -h Fcst LCH Sounding Valid 12 z 15 Feb 2016 MUCAPE 657 J/kg (error ~1300 J/kg) GFSp 12 -h forecast sounding is similar to GFS forecast 49

RAP Analysis & GFS 12 -h Forecast Valid 12 z 15 Feb 2016 RAP MUCAPE Analysis GFS MUCAPE 12 -h Forecast Green Shading Denotes > 1000 J/kg GFS Forecast too Slow to Destabilize across SE TX & LA 50

RAP Analysis & GFSp 12 -h Forecast Valid 12 z 15 Feb 2016 RAP MUCAPE Analysis GFSp MUCAPE 12 -h Forecast Green Shading Denotes > 1000 J/kg GFSp Forecast too Slow to Destabilize Across SE TX & LA 51

GFS and & GFSp 12 -h Forecasts Valid 12 z 15 Feb 2016 GFS MUCAPE 12 -h Forecast GFSp MUCAPE 12 -h Forecast Green Shading Denotes > 1000 J/kg Little Practical Difference Between GFS and GFSp Both Substantially Under-forecast CAPE 52

18 z LIX Observed Sounding Valid 18 z 15 Feb 2016 MUCAPE 1928 J/kg Mean Q 11. 2 G/kg RAOB exhibits moist PBL and steep lapse rate layers supportive of severe weather potential 53

GFS 18 -h Fcst LIX Sounding Valid 18 z 15 Feb 2016 MUCAPE 810 J/kg (error ~1100 J/kg) GFS 18 -h forecast sounding is too warm/dry near ground which substantially reduced the CAPE forecast 54

GFSp 18 -h Fcst LIX Sounding Valid 18 z 15 Feb 2016 MUCAPE 693 J/kg (error ~1200 J/kg) GFSp 18 -h forecast sounding also exhibits erroneous warm/dry layer above ground similar to GFS forecast 55

RAP Analysis & GFS 18 -h Forecast Valid 18 z 15 Feb 2016 RAP MUCAPE Analysis GFS MUCAPE 18 -h Forecast Green Shading Denotes > 1000 J/kg Purple Shading Denotes >2000 J/kg GFS Forecast too Slow to Destabilize across SE TX, LA, MS & FL 56

RAP Analysis & GFSp 18 -h Forecast Valid 18 z 15 Feb 2016 RAP MUCAPE Analysis GFSp MUCAPE 18 -h Forecast Green Shading Denotes > 1000 J/kg Purple Shading Denotes >2000 J/kg GFSp Similar to GFS Forecast – No Improvement 57

SPC GFS Evaluation Summary • For the aspects examined in the SPC evaluation, the GFSp seems to be as good as or better than the operational GFS overall. • The low 2 -m dewpoint and instability bias of the operational GFS has been improved in the GFSp during the warm season over the plains, owing to land surface parameter changes. • This improvement is a step in the right direction; however, a notable low frequency bias still exists for large instability values (≥ 2000 J/kg). • Other issues noted in forecast sounding structure include substantial errors in the analysis (f 00) and smoothing out inversions & elevated mixed layers relevant for severe weather forecasting. • SPC appreciates the willingness of the EMC Global Branch to work with us on these issues through establishing more regular dialogue. • SPC Recommendation: Approve for implementation 58