What Is A Good Forecast Issues of Model

What Is A Good Forecast? Issues of Model Building and Forecast Uncertainty David C. Garen, Ph. D. Hydrologist USDA Natural Resources Conservation Service National Water and Climate Center Portland, Oregon

Forecast Model Characteristics • Robust, consistently performing models • Can be trusted, don’t have to second-guess or adjust much • Easily explainable • Physically meaningful, statistically valid, and operationally useful

Considerations in Building Forecast Models • Screen out useless variables beforehand • Fill in occasional missing values • Include “outliers” if they are real • Use automated, real-time stations whenever possible • SNOTEL era is now long enough to use as standard calibration period

Considerations in Building Forecast Models • Use nonlinear procedure if (1) there is a curvilinear relationship or (2) if the error distribution is skewed • Use station optimization as a guide, not the final word • Month-to-month consistency in variable usage • Consistency in principal component usage • Each variable must be physically meaningful

PC vs. Z-Score Regression • With Z-score, still need enough years to get good estimate of mean and sd • Can use Viper to estimate occasional missing values rather than leave variable out • PC does “grouping” implicitly, Z-score explicitly • PC and Z-score often give similar forecast results, but coefficients differ • Z-score assumes you have good representation of each main signal

What is a Forecast? • It is not a single number • It is best thought of as a (conditional) probability distribution • Must have realistic expectations of forecast accuracy (some basins are more predictable than others) • Uncertainty and risk, as quantified by the forecast distribution, should guide decision making

Sources of Forecast Error • Model error (functional form, simplifications) • Variables used (may not represent all relevant hydrological processes) • Spatial variability (may not be adequately represented by data network) • Future weather • Data error • Watershed change (vegetation, climate)

Forecast Shifts and Squeezes Historical Distribution

Actual Flow Can Land Anywhere on Forecast Distribution

Theoretical and Actual Flow Frequency for Forecast Confidence Intervals (Upper Klamath Lake, 1 Apr Forecasts of Apr-Sep Inflow, 1961 -2004) Theoretical 70% - 30% exc. 18 90% - 10% exc. 35 95% - 5% exc. 40 Actual 17 38 40

Implications of Forecast Uncertainty • User (not us) must choose risk level for operational forecast value, considering costs associated with receiving more or less water than each value • Management must be flexible to allow for receiving more or less water than the chosen operational value • The probability of receiving any specific flow value is zero • Climate change and variability adds a whole new dimension to forecast uncertainty