STATISTICAL WEATHER FORECASTING TECHNIQUES OBJECTIVE FORECASTING METHODS 1

STATISTICAL WEATHER FORECASTING TECHNIQUES (OBJECTIVE FORECASTING METHODS) 1 APKC : C 5. 2 (O F T)

The Atmosphere is a Chaotic, Dynamic System Predictability is primarily limited by errors in the analysis Sensitive to Initial Conditions: nearby solutions diverge Analogy Two adjacent drops in a waterfall end up very far apart. Describable State: system specified by set of variables that evolve in “phase space” Deterministic: system appears random but process is governed by rules Solution Attractor: Limited region in phase space where solutions occur Aperiodic: Solutions never repeat exactly, but may appear similar To account for this effect, we can make an ensemble of predictions (each forecast being a likely outcome) to encompass the truth. 2

Encompassing Forecast Uncertainty An analysis produced to run a model is somewhere in a cloud of likely states. Any point in the cloud is equally likely to be the truth. 12 h forecast 12 h verification T T 48 h forecast 24 h forecast 36 h forecast Nonlinearities drive apart the forecast trajectory and true trajectory (i. e. , Chaos Theory) 24 h verification T The true state of the atmosphere exists as a single point in phase space that we never know exactly. 36 h verification T 48 h verification T P H A A point in phase space completely describes an S S PA E instantaneous state of the atmosphere. (pres, temp, C E etc. at all points at one time. ) 3

Encompassing Forecast Uncertainty An ensemble of likely analyses leads to an ensemble of likely forecasts T An aly sis Re gio n Ensemble Forecasting: -- Encompasses truth P H S AS PA E C T -- Reveals uncertainty 4 -- Yields probabilistic information 48 h forecast Region

Why statistical reinterpretation of dynamical NWP output is useful for practical weather forecasting 1. NWP models simplify and homogenize sfc conditions. Statistical relationships can be developed btwn NWP output and desired forecast quantities. 2. NWP model forecasts are subject to error. To the extent that these errors are systematic, statistical forecasts based on NWP info can correct forecast biases. 3. NWP models are deterministic. Using NWP info in conjunction with statistical methods allows quantification of the uncertainty associated with different forecast situations. 5

STATISTICAL (OBJECTIVE) WEATHER PREDICTION Two types of approaches adopted by Statistical methods in Weather Prediction: Classical Approach. Without information from NWP models (Classical Statistical Methods). Viable and useful at a very short lead times, or very long times. Dynamic (NWP) Approach. Incorporates NWP information in conjunction with Statistical Methods for Weather Prediction. 6 BPKC : CLM (STATS) 6

STATISTICAL (OBJECTIVE) WEATHER PREDICTION Statistical forecast models are generally used to enhance results of Dynamical (NWP) forecasts and are essential as ‘Guidance’ products. This combined Statistical and Dynamical forecasting approach is important, especially for providing quantitative or location-specific forecasts, which are not represented by NWP models alone. 7 BPKC : CLM (STATS) 7

STATISTICAL (OBJECTIVE) WEATHER PREDICTION These types of Statistical forecasts are ‘Objective’ in nature, as a given set of inputs always produce same output. However, another important aspect of statistical weather forecasting is in the ‘Subjective’ formulation of forecasts, particularly when the forecast quantities are probabilities or set of probabilities. 8 BPKC : CLM (STATS) 8

STATISTICAL (OBJECTIVE) WEATHER PREDICTION Subjective probability assessment forms the basis of many operationally important forecasts and is a technique for broad usage to enhance the information content of operational forecasts. Based on Regression Analysis (e. g. , Simple & Multiple Regression, etc. ) and corresponding Goodness of Fit methods (e. g. , Least-square method, ANOVA, etc. ). 9 BPKC : CLM (STATS) 9

STATISTICAL (OBJECTIVE) WEATHER PREDICTION Classical Statistical Analysis (without NWP): Regression Analysis. ANOVA. Screening Regression. REEP and Logistic Regression. Objective Forecasting with NWP: Perfect Prognostic Method (PPM). Model Output Statistics (MOS). BPKC : CLM (STATS) 10 10

STATISTICAL WEATHER FORECASTING Has been practiced for thousands of year Collection of data, processing, use of result to forecast Knowledge of physical processes not required Based on historical patterns in climate Understanding of physical processes along with proper use of statistics will help BPKC : CLM (STATS) 11

STATISTICAL WEATHER FORECASTING Dynamical models simulate between ocean, land atm Based on laws of physics Explain transfer of heat and energy Dynamical models have limitations interaction Dynamical – Statistical (SD) models: incorporates numerically forecast data into statistical prediction BPKC : CLM (STATS) 12

STATISTICAL WEATHER FORECASTING Input into SD Models Predictand: variable to be forecasted /estimated Predictor: variable USED to forecast /estimate Arrangement of predictand predictors in data matrix for events BPKC : CLM (STATS) 13

TYPES OF PREDICTANDS Continuous: Can take any value in meaningful range. Ex Temp Categorical: can take on only a (small) specific set of values. e. g. , cloud amount in four categories clear, scattered, broken, and overcast). BPKC : CLM (STATS) 14

TYPES OF PREDICTANDS Transformed: value has been changed from the original observation. Transformation may be required To define the predictand according to operational requirements To transform a continuous variable into a categorical variable To alter the distribution of the predictand To achieve a specific non-linear fit BPKC : CLM (STATS) 15

PREDICTORS SELECTION Physical Relationship to the Predictands Availability Predictability Complete Not too Large Reasonably Independent Variables BPKC : CLM (STATS) 16

SOURCES OF PREDICTORS Observational Data Climatological Variables Model Output Basic Variables Derived Variable BPKC : CLM (STATS) 17

PROCESSING OF PREDICTORS Selection and processing of predictors is one of the most important steps of statistical technique development Should be transformed so that their relationship with the predictand is linear BPKC : CLM (STATS) 18

PROCESSING OF PREDICTORS Types of predictor processing • Interpolation. • Spatial and / or temporal differencing. • Spatial and / or temporal averaging. • Non-linear transformations. • Creating binaries. • Predictor combinations. • Empirical orthogonal functions. BPKC : CLM (STATS) 19

FORMULATION METHODS FOR STATISTICAL APPLICATION IN WEATHER FORECASTING BPKC : CLM (STATS) 20

FORMULATION METHODS FOR STATISTICAL APPLICATION IN WEATHER FORECASTING It is the source of dataset used in development that distinguishes the three formulations. Of the three, only PPM and MOS use NWP model output, while the Classical Method uses only observations. BPKC : CLM (STATS) 21

CLASSICAL METHODS Similar to subjective methods, used to predict weather, based on current observations or recent weather conditions. This approach does not use numerical model forecasts and relies purely on observed data. Before Dynamical Models, statistical systems were limited to Classical Approach. To develop equations with Classical Method, observations of the initial and resultant weather conditions are needed. BPKC : CLM (STATS) 22

CLASSICAL METHODS The resultant weather conditions are the ‘Predictands’, while the initial conditions are the ‘Predictors’. For Example, forecasting Max Temperature for tomorrow, input would consist only of observational data available at the time that the forecast was to be made. This situation can be expressed as: - BPKC : CLM (STATS) 23

CLASSICAL METHODS Here is estimate (forecast) of ‘Predictand’ (dependent variable) ‘Y’ at time ‘t’ and ‘X 0’ is a vector of observational data (independent variables) at initial time ‘ 0’. Observations are not necessarily made at initial time, but must be available at that time. Accuracy of any given forecast based on this approach is strongly dependent on significant changes occur in atmosphere between times of predictor observations and time of validity of BPKC : CLM (STATS) 24 resultant forecast.

CLASSICAL METHODS Greater the forecast projection, greater will be the chance of significant changes. Hence, this approach is good for short-range forecasts, but its skill falls sharply beyond a few hours. Practically no skill in medium range. Work best when the weather tends to be persistent (low variability). This approach is also used for very long-range seasonal forecasts, where there is little skill in numerical model predictions. BPKC : CLM (STATS) 25

PERFECT PROGNOSIS METHOD (PPM) BPKC : CLM (STATS) 26

PERFECT PROGNOSIS METHOD The need to accurately predict surface weather elements, led to the development of Perfect Prognosis Method (PPM) (Klien et al. , 1959). This is an objective method, in which, a concurrent relation is developed between the parameter to be predicted and the observed circulation around the location of interest, using several years of data. PPM technique is based on the assumption that numerical model forecasts are “Perfect”. BPKC : CLM (STATS) 27

1 st Statistical approach for dealing with forecasts from NWP �Perfect Prog (Klein et al. 1959) ◦ Takes NWP model forecasts for future atmosphere assuming them to be perfect ◦ Perfect prog regression equations are similar to classical regression equations except they do not incorporate any time lag. Example: equations specifying tomorrows predictands are developed using tomorrow’s predictor values. If the NWP forecasts for tomorrow’s predictors really are perfect, the perfect-prog regression equations should provide very good forecasts.

PERFECT PROGNOSIS METHOD

PERFECT PROGNOSIS METHOD Despite numerical models not being perfect, this approach gives an estimate of what to expect, if numerical models are correct It does not require numerical model data for development, but uses numerical output when equations are applied operationally Hence, it is important to make sure that variables used as ‘Predictors’ in development of Perfect Prognosis Equations will be available from NWP models, so that equations 30 can be applied operationally BPKC : CLM (STATS)

PERFECT PROGNOSIS METHOD Extensive sample of upper air observations and surface weather reports is collected Multiple Linear Regression equations are then derived In applying PPM equations, Dynamical Model output for a specific forecast projection is substituted for the developmental observations to give a forecast for the appropriate valid time BPKC : CLM (STATS) 31

PERFECT PROGNOSIS METHOD These specification equations can be used to generate guidance from any dynamical model as long as the model produces the required predictors and the original time relationships among predictors and predictand are preserved The PPM approach can be expressed numerically as: - BPKC : CLM (STATS) 32

PERFECT PROGNOSIS METHOD Here is estimate of ‘Predictand’ ‘Y’ at time ‘ 0’ and ‘X 0’ is a vector of observations of variables that can be predicted by numerical models Even though is an estimate, it is not a forecast, it is only a specification In application, is inserted into equation (2) to provide a forecast as: - BPKC : CLM (STATS) 33

MODEL OUTPUT STATISTICS (MOS) BPKC : CLM (STATS) 34

MODEL OUTPUT STATISTICS Like PPM, the MOS approach is also an objective type of forecasting technique (Glahn & Lowry, 1972). In contrast to Classical Approach and Perfect Prognosis Approach (which uses numerical model output only in operational application of the equations), the Model Output Statistics (MOS) approach uses numerical model forecasts for both the development and the operational application of the equations BPKC : CLM (STATS) 35

2 nd Statistical approach �Model Output Statistics (MOS) ◦ Preferred because it can include directly in the regression eqns the influences of specific characteristics of different NWP models at different projections into the future. Example: Predictand is tomorrow’s 1000 -800 mb thickness as forecast today by a certain NWP model. ◦ To get MOS forecast eqns you need a developmental data set with historical records of predictand, and records of the forecasts by NWP model. ◦ Separate MOS forecast equations must by made for different forecast projections.

MODEL OUTPUT STATISTICS

MODEL OUTPUT STATISTICS MOS Approach requires an archive of dynamical model forecasts : usually 2 -3 years of data are needed MOS Approach relates a weather observation (Predictand) to variables forecast by a numerical model (Predictors) The Predictors from the numerical model are usually forecasts that are valid at about the same time as the Predictand BPKC : CLM (STATS) 38

MODEL OUTPUT STATISTICS Some statistical method (usually Multiple Linear Regression) is then used to determine relationships among various observed weather elements and numerical model output variables at projections near (before, at, or amer) the specific valid time of the Predictand. To make operational forecasts, MOS equations are usually applied to the same dynamical model that provided the developmental sample. BPKC : CLM (STATS) 39

MODEL OUTPUT STATISTICS Mathematically MOS can be expressed as: - where is estimate of Predictand ‘Y’ at time ‘t’ and is a vector of forecasts from numerical models. The numerical model predictions need not be limited at time ‘t’; however, the projection times of the different variables will usually be grouped around ‘t’. BPKC : CLM (STATS) 40

MODEL OUTPUT STATISTICS The equations (4) account for some of the bias and systematic errors found in the dynamical model. Local topographic and environmental conditions of a location are also automatically accounted for in the forecast system. MOS technique also recognises the predictability of the model variables by selecting those variables that provide useful forecast information. BPKC : CLM (STATS) 41

MODEL OUTPUT STATISTICS Finally, as the dynamical model predictions deteriorate with increasing time, this approach produces forecasts that tend towards the mean of the predictand in the developmental sample. Drawback of this technique is that a sufficient sample of Model Output is required in order to derive a stable relation. Hence, it cannot be applied immediately when a new model is made operational. Also, if model undergoes a major change, MOS BPKC : CLM (STATS) 42 relations will have to be developed again.

COMPARISON BETWEEN CLASSICAL, PPM AND MOS TECHNIQUES BPKC : CLM (STATS)

COMPARISON Classical Approach is most useful for very short range (0 -12 hour), or if NWP model forecast data is not available. It is relatively simple to use, since it requires only that data, which are usually available without special archiving procedures. PPM and MOS approaches require NWP model forecast data in operational application of the equations. BPKC : CLM (STATS) 44

COMPARISON However, since PPM equations are derived from observed data only, there is usually a longer sample of data available from which to derive the equations. In contrast, MOS equations need to be developed from a stable sample of numerical model forecast data. Because most NWP models are constantly evolving, developmental sample for MOS equations is likely to be relatively short. BPKC : CLM (STATS) 45

COMPARISON When a model is changed significantly, forecast produced by MOS has less skill and accuracy and equations will have to be redeveloped. In contrast, PPM forecasts usually do not deteriorate with change in model. Generally an improved numerical model will lead to improved forecasts. If model is not modified and sufficient sample of stable numerical model output is available, BPKC : CLM (STATS) 46 MOS approach will produce the best results.

COMPARISON Classical Development Weather element of equations observed at T 0. PPM MOS Weather element observed at T 0. Predictors observed (Anal. ) at T 0 -d. T. (Anal. ) at T 0. Application in Operational Forecast mode Comments Predictor values observed now (T 0) to give forecast valid at T 0+d. T less than or equal to 6 hours preferable unless persistence works well. Time lag built into equations. Weather element observed at T 0. Predictors Forecast values valid at T 0 from PPM issued at T 0 -d. T. Predictor values valid for T 0+d. T from PPM issued now to give forecast valid at T 0+d. T can take any value, for which Forecast predictors sre available. Same application as PPM, but separate equations used for each d. T. BPKC : CLM (STATS) 47

COMPARISON Classical Specific Differences Relationship weaken rapidly as predictorweather element time lag increases. Modelindependent. Does not use model output. Large development sample possible. Must have access to observed or analysed variables. PPM MOS Relationships strong because only observed data concurrent in time are used. Modelindependent. Does not account for model bias – model errors decrease accuracy. Large development sample possible. Must have access to observed or analysed variables. Relationships weaken with increasing projection time due to increasing model error variance. Model-dependent. Partially accounts for model bias. Generally small development samples, depends on frequency of model changes. Must have access to model output variables that may not be observed. BPKC : CLM (STATS) 48

SUMMARY Pure Classical Statistical Weather Forecasts for periods of a few days are generally less useful, as Dynamical NWP models allow more accurate forecasts. Difference between Dynamical methods and Classical Approach is that predictor variables are taken and used from output of NWP models. Additionally, many more of these variables are available through NWP models. BPKC : CLM (STATS) 49

SUMMARY Mainly two types of methods: Perfect Prognosis Method (PPM). Model Output Statistics (MOS). In PPM process, NWP output for a specific forecast projection is substituted for developmental observations to give a forecast for the appropriate valid time. MOS approach uses NWP model forecasts for both, the development and the operational, applications of the equations. BPKC : CLM (STATS) 50

SUMMARY : COMPARISON Classical Approach : most useful for very short range (0 -12 hour), or if NWP model forecast data is not available. Dynamical Approach : require NWP model forecasts in developing equations. PPM Approach : equations are derived from observed data (longer data sample required). MOS Approach : equations need to be developed from a stable sample of NWP forecast data (samples are relatively shorter). BPKC : CLM (STATS) 51

Advantages and Disadvantages of Perfect-Prog and MOS:

Perfect Prog �Advantages: � Large developmental sample (fit using historical climate data) � Equations developed without NWP info, so changes to NWP models don’t require changes in regression equations � Improving NWP models will improve forecasts � Same equations can be used with any NWP models �Disadvantages: ◦ Potential predictors must be well forecast by the NWP model

MOS �Advantages: ◦ ◦ Model-calculated, but un-observed quantities can be predictors Systematic errors in the NWP model are accounted for Different MOS equations required for different projection times Method of choice when practical �Disadvantages: ◦ Requires archived records (several years) of forecast from NWP model to develop, and models regularly undergo changes. ◦ Different MOS equations required for different NWP models

ANY QUESTION ? BPKC : CLM (STATS) 55

MOS (Additional Information)

Model Output Statistics (MOS) REVISION � MOS relates observations of the weather element to be predicted (PREDICTANDS) to appropriate variables (PREDICTORS) via a statistical method Predictors can include: Ø NWP model output interpolated to observing site Ø Prior observations Ø Geoclimatic data – terrain, normals, lat/long, etc. Current statistical method: Multiple Linear Regression (forward selection)

MOS Development Strategy CAREFULLY define your predictand Stratify data as appropriate Pool data if needed (Single Station / Regional) Select predictors for equations AVOID OVERFITTING!

Predictand Strategies Predictands always come from meteorological data and a variety of sources: • • Point observations (ASOS, AWOS, Co-op sites) Satellite data (e. g. , SCP data) Lightning data (NLDN) Radar data (WSR-88 D) It is very important to quality control predictands before performing a regression analysis.

Predictand Strategies (Quasi-)Continuous Predictands: best for variables with a relatively smooth distribution • • Temperature, dew point, wind (u and v components, wind speed) Quasi-continuous because temperature available usually only to the nearest degree C, wind direction to the nearest 10 degrees, wind speed to the nearest m/s. Categorical Predictands: observations are reported as categories • Sky Cover (CLR, FEW, SCT, BKN, OVC)

Example of MOS Predictands �Temperature �Dry bulb temperature (every 3 h) �Dew point (every 3 h) �Daytime maximum temperature [0700 – 1900 LST] (every 24 h) �Nighttime minimum temperature [1900 – 0800 LST] (every 24 h) �Wind �U- and V- wind components (every 3 h) �Wind speed (every 3 h) �Sky Cover �Clear, few, scattered, broken, overcast [binary/MECE] (every 3 h)

Example of MOS Predictands �Po. P/QPF �Po. P: accumulation of 0. 01” of liquid-equivalent precipitation in a {6/12/24} h period [binary] �QPF: accumulation of {0. 10”/0. 25”/0. 50”/1. 00”/2. 00”*} CONDITIONAL on accumulation of 0. 01” [binary/conditional] � 6 h and 12 h guidance every 6 h; 24 h guidance every 12 h � 2. 00” category not available for 6 h guidance �Thunderstorms � 1+ lightning strike in gridbox [binary] �Severe � 1+ severe weather report in gridbox [binary]

Example of MOS Predictands �Ceiling Height �CH < 200 m, 200 -400 m, 500 -900 m, 1000 -1900 m, 2000 -3000 m, 3100 -6500 m, 6600 -12000 m, > 12000 m [binary/MECE] �Visibility < ½ km, < 1 km, < 2 kms, < 3 kms, < 5 kms, < 6 kms [binary] �Obstruction to Vision �Observed fog (fog w/ vis < 1 km), mist (fog w/ vis > 1 -2 km), haze (includes smoke and dust), blowing phenomena, or none [binary]

Example of MOS Predictands �Precipitation Type �Pure snow (S); freezing rain/drizzle, ice pellets, or anything mixed with these (Z); pure rain/drizzle or rain mixed with snow (R) �Conditional on precipitation occurring �Precipitation Characteristics (Po. PC) �Observed drizzle, steady precip, or showery precip �Conditional on precipitation occurring �Precipitation Occurrence (Po. PO) �Observed precipitation on the hour – does NOT have to accumulate

Stratification Goal: To achieve maximum homogeneity in our developmental datasets, while keeping their size large enough for a stable regression MOS equations are developed season wise

Pooling Data Generally, this means REGIONALIZATION: collecting nearby stations with similar climatology Regionalization allows for guidance to be produced at sites with poor, unreliable, or non-existent observation systems • All MOS equations are regional except temperature and wind

MOS Development Strategy �MOS equations are multivariate of the form: �Y = c 0 + c 1*X 1 + c 2*X 2 + … + c. N*XN �C’s are constants, X’s are predictors �N is the number of predictors in the equation and is specified when the equations are developed.

MOS Development Strategy �Forward Selection ensures that the “best” or most STATISTICALLY IMPORTANT predictors are chosen first. • First predictor selected accounts for greatest reduction of variance (RV) • Subsequent predictors chosen give greatest RV in conjunction with predictors already selected • STOP selection when max # of terms reached, or when no remaining predictor will reduce variance by a pre-determined amount

Post-Processing MOS Guidance • Meteorological consistencies – SOME checks ◦ T > Td; min T < max T; dir = 0 if wind speed = 0 ◦ Truncation (no probabilities < 0, > 1) ◦ Monotonicity enforced (for elements like QPF) ◦ BUT temporal coherence is only partially checked • Generation of “best categories”

Unconditional Probabilities from Conditional � If event B is conditioned upon A occurring: � Prob(B|A)=Prob(B)/Prob(A) � Prob(B) = Prob(A) × Prob(B|A) � Example: � Let A = event of > 10 mm. , and B = event of >. 25 mm. , then if: � Prob (A) =. 70, and � Prob (B|A) =. 35, then � Prob (B) =. 70 ×. 35 =. 245 B A U

Truncating Probabilities � 0 < Prob (A) < 1. 0 � Applied to QPF and thunderstorm probabilities � If Prob(A) < 0, Probadj (A)=0 � If Prob(A) > 1, Probadj (A)=1.

Normalizing MECE Probabilities � Sum of probabilities for exclusive and exhaustive categories must equal 1. 0 � If Prob (A) < 0, then sum of Prob (B) and Prob (C) = D, and is > 1. 0. � Set: Probadj (A) = 0, � Probadj (B) = Prob (B) / D, � Probadj (C) = Prob (C) / D

Monotonic Categorical Probabilities � If event B is a subset of event A, then: � Prob (B) should be < Prob (A). � Example: B is > 25 mm; A is > 10 mm � Then, if Prob (B) > Prob (A) � set Probadj (B) = Prob (A). � Now, if event C is a subset of event B, e. g. , C is > 0. 50 in, and if Prob (C) > Prob (B), � set Probadj (C) = Prob (B)

Temporal Coherence of Probabilities � Event A is > 10 mm occurring from 12 Z-18 Z � Event B is > 10 mm occurring from 18 Z-00 Z � A B is > 10 mm occurring from 12 Z-00 Z � Then P(A B) = P(A) + P(B) – P(A B) � Thus, P(A B) should be: � < P(A) + P(B) and � > maximum of P(A), P(B) A C B

MOS Best Category Selection An example with QPF… PROBABILITY (%) 80 TO MOS GUIDANCE MESSAGES 60 FORECAST THRESHOLD 40 EXCEEDED? 20 0 5 10 15 20 25 >25 PRECIPITATION AMOUNT EQUAL TO OR EXCEEDING

MOS GRID