WAFC turbulence and Cb hazard verification Recent results

WAFC turbulence and Cb hazard verification Recent results and future plans Dr Philip G Gill WAFSOPSG 7/14, 30 th April 2013 © Crown copyright Met Office

Contents This presentation covers the following areas • Introduction • Turbulence verification • Cb verification • Future plans • Questions and answers © Crown copyright Met Office

Introduction • Turbulence and Cb are a major cause of aviation incidents • Turbulence and Cb hazard forecasting are part of the World Area Forecast Centre (WAFC) service provided by WAFC London and WAFC Washington • Forecasts currently produced both by forecasters – Significant Weather Chart (SIGWX) and from gridded model output • Objective verification was set up in 2009 • Forecasts are global but verification is currently limited Photos © P Gill © Crown copyright Met Office

Objective turbulence verification © Crown copyright Met Office

Verification • Objective verification of gridded clear air turbulence (CAT) forecasts • Verification by severity of turbulence (eg. moderate or greater) • Verify against automated aircraft observations from the Global Aircraft Data Set (GADS) • Restricted to cruise level (above 28, 000 ft) to limit unwanted effects on observations of manoeuvring at lower levels © Crown copyright Met Office

Global Aircraft Data Set • Fleet of Boeing 747 -400 aircraft • Global coverage, but flights mainly over northern hemisphere • Automated aircraft observations available every 4 seconds Good coverage of N Atlantic, US and Europe © Crown copyright Met Office Poor coverage of E Asia/Pacific region and southern hemisphere – need more observations 10 -19 January 2009

Turbulence observations • Pilot Reports (PIREPS) are issued by pilots when turbulence is encountered – useful additional information but are subjective, aircraft specific, temporal and spatial errors, no routine null reports • AMDAR are automated reports from aircraft but are not regular and can differ between aircraft • Derived Equivalent Vertical Gust (DEVG) - Indicator of turbulence derived from automated in-situ flight data: vertical acceleration, aircraft mass, altitude and airspeed • Eddy Dissipation Rate (EDR) also a measurement of turbulence derived from automated in-situ data – ICAO standard At present access to EDR data limited to US – DEVG only option for wider coverage © Crown copyright Met Office

Verification methodology Turbulent event Turbulence forecast field Aircraft track within time window © Crown copyright Met Office Ellrod TI 1

Forecast assessment • Turbulent/non turbulent event defined on 10 min aircraft track ~120 km approx grid size • Forecast turbulent event – CAT potential >= Threshold • Observed (moderate or greater) turbulent event - DEVG>=4. 5 m/s • Construct 2 x 2 contingency tables for each threshold • Sum entries in contingency tables over the verification period Turbulence observed No turbulence observed Turbulence forecast Hit False alarm No turbulence forecast Miss Correct rejection 2 x 2 contingency table © Crown copyright Met Office

Forecast skill • Contingency tables can be used for various probability thresholds to produce a reliability table • From the reliability table various scores can be calculated • The skill of the forecast at discriminating between events and non-events can be measured using the Receiver Operating Characteristic (ROC) by plotting hit rate against false alarm rate • The curve from a probabilistic forecast and single point from a deterministic forecast can be compared. Perfect forecast Line of no skill Area under curve measures skill

WAFC CAT verification Robert Coulson © Crown copyright Met Office

Latest results – mean CAT Acceptable hit rates? Area of interest could be measured by partial area under ROC curve Acceptable false alarm rates?

Latest results

Latitudinal variation 50 N to 80 N 20 N to 50 N 20 S to 20 N 40 S to 20 S

Summary of Cb verification • Blending process continues to produce both consistent and skilful Cb forecasts. • Some variation in skill by latitude • Rolling 12 -month results produced quarterly since June 2012.

Future verification plans © Crown copyright Met Office

Improving turbulence verification • Switch from DEVG to EDR • ICAO standard • Forecasting and observing EDR • Source more EDR observations from other airlines to increase the area of coverage • Increase coverage to fill gaps in current GADS database

Improving Cb verification • Extend verification against lightning reports by outside of the ATDnet domain • Source additional lightning reports outside [80 N, 40 S, 100 W, 80 E] • Use geostationary satellite data in addition to lightning data • MSG [60 N, 60 S] • Overshooting convection product (work underway) • New satellites over America (GOES-R) and East Asia (Himawari-8) • Verify Cb cloud top height and Cb extent

MSG - the European Geostationary Imager • Scans Africa, Europe, Middle East and Atlantic every 15 minutes • 3/1 km spatial resolution at 0° N, 0° E • 12 channels (different wavelengths) • Meteosat-10 operational, Meteosat-8, -9 backup

Severe convection product • Data from Meteosat Second Generation • Calculated from the visible and infrared channels. • Generated every 15 mins. • Current product over Europe identifies area where severe convection is occurring (day-time only) • New overshooting convection product will identify intense convective areas with overshooting cloud tops. • Could be merged with lightning data from ATDnet • Could be merged later with Meteosat third generation lightning data

Cloud top height product • Data from Meteosat Second Generation • Calculated from the infrared channels at wavelengths of 10. 8, 12. 0 and 13. 4 microns. • Generated every 15 mins • Covers Europe, Atlantic, Africa • Extend to global coverage following GOES-R and Himawari-8 satellite launches (~ 2015) • Mask with Cb product

Verification – all forecasts • Verify probabilistic forecasts using suitable metrics • Forecast skill – ROC (partial area under curve? ) • Reliability diagram • Value – relative economic value measure • Publish verification results – plots and contingency tables • Breakdown into WMO regions • Forecast ranges T+12, T+24, T+36 • Rolling 12 -month scores • Geostationary satellite data limited to 60 N, 60 S. Polar orbiter data could supplement this if necessary.

Timescales • 2013/14 • Preliminary work on satellite data specifications • Routine CAT and Cb verification published by WMO region. • 2014/15 • Include satellite data in Cb verification • Develop Cb cloud top height verification • Move to using EDR in turbulence verification • 2015/16 • Expand coverage of Cb verification using new satellite data over East Asia (Himawari-8) • 2016/17 • Expand coverage of Cb verification using new satellite data over America (GOES-R)