http weather eos ubc cawxfcst Evaluation of Mesoscale

http: //weather. eos. ubc. ca/wxfcst Evaluation of Mesoscale Numerical Weather Prediction Models Roland Stull Professor, UBC Earth & Ocean Sciences Dept. Director: Geophysical Disaster Computational Fluid Dynamics Center rstull @ eos. ubc. ca 604 -822 -5901 NW-AIRQUEST Annual Meeting Portland, OR 6 -7 Oct 2003 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Topics • UBC. Who we are, & what models we run. • Mesoscale model characteristics & limitations • Sensitivity test approaches (& ensembles) • Verification metrics, validation, requirements • Credibility, accuracy, Pacific Data Void 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Who we are, and what models we run. • University of British Columbia (UBC) • Geophysical Disaster Computational Fluid Dynamics Center • Team: Director (Prof. Stull), 1 Sr. Scientist (Terry Clark), 6 Ph. D students, 1 MS student, 4 undergrad assistants, 4 computer scientists, & 1 volunteer. • Hardware: 256 processor IBM linux cluster, 24 processor Beowulf cluster, 8 processor Itanium, 8 processor SGI Origin 2000, 4 processor Origin 300, numerous servers & workstations • Mission: pure & applied NWP research for coastal, mtn terrain • Real-time, daily NWP forecasts since 1976. 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Geophysical Disaster Computational Fluid Dynamics Centre Director: Prof. Roland Stull, CCM Using large computers to forecast weather-related disasters • snowfall -> avalanches • forest firestorms • rain -> floods • cyclones & windstorms • air-pollution episodes Okanagan Park fire - 20 Aug 03 photo by Steve Devries 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst 256 processors 4 Oct 03 NW-AIRQUEST Real-time Forecasts Produced Daily, and tailored to the needs of industry.

http: //weather. eos. ubc. ca/wxfcst The Geo. Disaster CFD Centre also gathers real-time weather observations. We maintain a Real-Time, On-Line Database of Surface Weather Observations, called Emergency Weather Net - BC We also develop new sensors, including the Rocketsonde Buoy System 2302 weather stations reporting 4 Oct 03 NW-AIRQUEST

The Models http: //weather. eos. ubc. ca/wxfcst MC 2 Pro: Fast; allows higher resolution over larger domains Con: Less accurate in steep topography; doesn’t have 2 -way nesting; on-again/off -again support from EC MM 5: Pro: NCAR community support; 2 -way nesting; more accurate; large worldwide experience Con: slower; can cover medium domains; will be phased out by NCAR WRF: Pro: NCAR/NCEP; designed for distributed/shared mem. platforms (fast); will replace MM 5 Con: in beta-test; unknown characteristics NMS: Pro: two-way nesting; more accurate Con: very slow; smallest domains; no user support WFIS (Clark model): Pro: ultra-fine grid spacing (20 m); optimized for steep terrain Con: not real time 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Your entry point: http: //weather. eos. ubc. ca/wxfcst 4 Oct 03 NW-AIRQUEST

108 km MC 2 NMS 4 Oct 03 http: //weather. eos. ubc. ca/wxfcst MM 5 WRF NW-AIRQUEST

MC 2 36 km 4 Oct 03 http: //weather. eos. ubc. ca/wxfcst 12 km NW-AIRQUEST

800%http: //weather. eos. ubc. ca/wxfcst Zoom of MC 2 2 km pdf 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Traditional trajectory methods: 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Mesoscale model characteristics & limitations • Forecast skill in Pac NW decreases to 60% in 24 h in summer • Reason for poor skill is Pacific Data Void !!!! (no amount of ensembles nor finer grid resolution, and no tweeking of sigma-y, sigma-z, emission rates, physics params. , or PBL depth can compensate) • No effective limit to smallness of horiz. grid size; NWP models are becoming more like LES models 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Clark’s WFIS Model Simulation of forest fire in Rocky Mtns. showing T (color) & w (contours), at 0. 25 km horiz. grid spacing 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Sensitivity test approaches (& ensembles) • Because the Pacific Data Void as the weakest link in the AQ/ Met. forecast chain, the most appropriate sensitivity tests are: Observing System Simulation Experiments (OSSEs). • All other sensitivity tests (advection; sub-grid; turb closure; param schemes, etc. ) are irrelevant for the Pac NW. • MM 5 runs of 7 winter cyclones & 5 summer cyclones 2001 -2002. • Experimented with effects of buoy locations • Found N. Amer. paid 20 - 35% penalty in fcst accuracy due to the Pacific Data Void. • Optimum RBS: 6 buoys in cross (see Fig ->) 6 km altitude, 12 Z each day, only Fall-Spr. • Penalty reduced to 5 - 15% with optimum RBS Spagnol & Stull, 2003 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Multi-model, multigrid size Ensembles (with each model Kalman filtered before ensemble averaging) 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Verification metrics & validation, requirements Merritt, BC For the operational wx fcsts at UBC, we do: mean error, mean absolute error, RMSE, error variance, correlation, slope. Will soon do equitable threat score for precip. 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Credibility, accuracy, Pacific Data Void • No credibility at any time, anywhere in Pac NW. • To address the Pacific Data Void problem is THe Observing System Research and Predictability Experiment (THORpex). It is endorsed by WMO, World Weather Research Program (WWRP), US Weather Research Program (USWRP), Canadian Weather Research Program (CWRP). • Recommendation: Regional AQ managers via NWAIRQUEST should solidly support THORpex initiatives. (Good leverage/synergy, because many countries and national gov’ts are funding. ) 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Winter Storm Recon. Program. 3 yrs dropsondes from aircraft. Begs question: If the favored sounding locations are always in the same part of NE Pacific, why not deploy an array of fixed sounding systems, rather than use expensive manned aircraft? 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Rocketsonde Development at UBC. 4 Oct 03 NW-AIRQUEST

http: //weather. eos. ubc. ca/wxfcst Evaluation of Mesoscale Numerical Weather Prediction Models Roland Stull Professor, UBC Earth & Ocean Sciences Dept. Director: Geophysical Disaster Computational Fluid Dynamics Center rstull @ eos. ubc. ca 604 -822 -5901 Summary: • UBC GDCFDC. Run MC 2, MM 5, WRF, NMS, WFIS. • Mesoscale models becoming like LES models. • Use OSSEs & ensembles (but Pac. Data Void cancels) • All verification metrics possible. Building Rocketsonde. • No Credibility, poor accuracy, due to Pacific Data Void Recommendation: Plan strategically. Support THORpex. 4 Oct 03 NW-AIRQUEST