Ensemble4 DWX update focus on calibration and verification

Ensemble-4 DWX update: focus on calibration and verification Tom Hopson, Josh Hacker, Jason Knievel, Yubao Liu, Gregory Roux, Wanli Wu National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

Reminder of status in April 2009 • Running WRF V 3. 0. 1. 1 • Had just finished running E-4 DWX for ATC during Dec 2008 and Jan 2009, then returned configuration to DPG • Configured and ran E-4 DWX for supporting UT Division of Air Quality • Improved MYJ and YSU PBL height diagnosis and PBL mixing • Added new graphics and improve post-processing flexibility (installation for GMOD and plotting historical case archive) and computing parallelisms • Fully automated calibration just being tested • Calibrated variables: 2 -m T National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

Progress since April 2009 (1 of 2) • Improved scripting to manage the computational resources • Increased stations from 26 to 34 (regional) • Calibration hindcast increased from roughly 600 to over 1000 points – Increased hindcast size allows calibration of 2 additional lead-times (36 hr, 42 hr) • Included new skill measure for ensemble skill-spread relationship • Corrected over-dispersion of calibration process • Calibrated variables: 2 -m T, 2 -m Td (new) National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

Progress since April 2009 (2 of 2) • Examined results from CAD study at ATC (earlier talk) • Using performance of ensemble members to optimize DPG’s singlemodel configuration • Submitted proposal for brief paper in BAMS but was rejected. Have not decide whether to revise and resubmit • Continuing to work on scientific paper for MWR • Downtime of HPC (recent): May 13 -20, Agami 4 failure June 16 -18, SGI node July 11 -19, power outage caused Agami 4 failure July 29 - August 4, Agami 4 failure National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

Calibration and verification • Ensemble calibration to correct predicted distribution • Calibration is needed for users capable of decision making with probabilistic guidance. Will be needed foreseeable future • Verification of different ensemble characteristics is easily completed when performing calibration National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

obs “bias” Forecast PDF “spread” or “dispersion” calibration Probability Calibration: review and recap Temperature [K] obs Forecast PDF Temperature [K] • Calibration (and verification) is now fully-automated • Utilizes “persistence” if available • 34 sites in and near DPG • Full calibration for all sites ~ 1 X per week for each weather variable • Using lookup tables ~ every hour (8 hrs, was 1 hr) National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

3 -hr dewpoint time series Station DPG S 01 Before Calibration After Calibration National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

42 -hr dewpoint time series Station DPG S 01 Before Calibration After Calibration National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

PDFs: raw vs. calibrated Blue is “raw” ensemble Black is calibrated ensemble Red is the observed value Notice: significant change in both “bias” and dispersion of final PDF obs (also notice PDF asymmetries) NCAR/RAL - National Security Applications Program ATEC-4 DWX IPR, Aug 11 -12 2009 10

0 0 Counts 10 20 30 30 Troubled rank histograms 1 2 3 4 5 6 7 8 9 10 Ensemble # NCAR/RAL - National Security Applications Program ATEC-4 DWX IPR, Aug 11 -12 2009 11

3 -hr dewpoint rank histograms Station DPG S 01 National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

42 -hr dewpoint rank histograms Station DPG S 01 National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

RMSE of ensemble members Station DPG S 01 42 hr Lead-time 3 hr Lead-time National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

Significant calibration regressors Station DPG S 01 42 hr Lead-time 3 hr Lead-time National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.

Future plans § Refine lookup table process to decrease computation time (required as calibration data size continues to increase and new weather variables are added) § Implement for a) wind speed, b) wind direction, c) precipitation, d) surface pressure § Diagnose most informative model set to use operationally § Develop scheme for model points without surface observations over whole model domain National Security Applications Program Research Applications Laboratory ATEC-4 DWX IPR, Aug 11 -12 2009 © 2009 UCAR. All rights reserved.