Consolidation of Predictions of Seasonal Climate by Several

Consolidation of Predictions of Seasonal Climate by Several Atmospheric General Circulation Models at IRI Anthony Barnston, Lisa Goddard, Simon Mason and Andrew Robertson International Research Institute for Climate and Society (IRI)

IRI DYNAMICAL CLIMATE FORECAST SYSTEM 2 -tiered OCEAN ATMOSPHERE PERSISTED GLOBAL SST ANOMALY GLOBAL ATMOSPHERIC MODELS ECPC(Scripps) 10 24 24 ECHAM 4. 5(MPI) Persisted SST Ensembles 3 Mo. lead FORECAST SST TROP. PACIFIC: THREE scenarios (multi-models, dynamical and statistical) TROP. ATL, INDIAN (ONE statistical) EXTRATROPICAL (damped persistence) CCM 3. 6(NCAR) 12 NCEP(MRF 9) NSIPP(NASA) COLA 2 GFDL Forecast SST 24 30 Ensembles 3/6 Mo. lead 24 12 30 30 POST PROCESSING MULTIMODEL ENSEMBLING -Bayesian -Caninical variate

IRI DYNAMICAL CLIMATE FORECAST SYSTEM 2 -tiered OCEAN PERSISTED GLOBAL SST ANOMALY ATMOSPHERE GLOBAL ATMOSPHERIC MODELS ECPC(Scripps) ECHAM 4. 5(MPI) FORECAST SST TROP. PACIFIC: THREE scenarios: 1) CFS prediction 2) LDEO prediction 3) Constructed Analog prediction TROP. ATL, and INDIAN oceans CCA, or slowly damped persistence EXTRATROPICAL damped persistence CCM 3. 6(NCAR) NCEP(MRF 9) NSIPP(NASA) COLA 2 GFDL

Six GCM Precip. Forecasts, JAS 2000

RPSS Skill of Individual Models: JAS 1950 -97

Goals To combine the probability forecasts of several models, with relative weights based on the past performance of the individual models To assign appropriate forecast probability distribution: e. g. damp overconfident forecasts toward climatology

Probabilities and Uncertainty Climatological Probabilities GCM Probabilities k = tercile number t = forecast time m = no. ens members Above Normal 1/3 6/24 Near-Normal 1/3 8/24 Below Normal 1/3 10/24 Tercile boundaries are identified for the models’ own climatology, by aggregating all years and ensemble members. This corrects overall bias.

Bayesian Model Combination Combine climatology forecast (“prior”) and an AGCM forecast, with its evidence of historical skill, to produce weighted (“posterior”) forecast probabilities, by maximizing the historical likelihood score.

Aim to maximize the likelihood score k=tercile category t=year number The multi-year product of the probabilities that were hindcast for the category that was observed. (Could maximize other scores, such as RPSS) Prescribed, observed SST used to force AGCMs. Such simulations used in absence of ones using truly forecasted SST for at least half of AGCMs.

1. Calibration of each model, individually, against climatology Optimize likelihood score k=tercile category (1, 2, or 3) t=year number j=model number (1 to 7) w=weight for climo (c) or for model j PMMkt= weighted linear comb of Pjkt over all j, normalized by Σ(wj) 2. Calibration of the weighted model combination against climatol Optimize likelihood score where w. MM uses wj proportional to results of the first step above

Algorithm used to maximize the designated score: Feasible Sequential Quadratic Programming (FSQP) “Nonmonotone line search for minimax problems” C C C C C C M: TOTAL NUMBER OF CONSTRAINTS. ME : NUMBER OF EQUALITY CONSTRAINTS. MMAX : ROW DIMENSION OF A. MMAX MUST BE AT LEAST ONE AND GREATER THAN M. N: NUMBER OF VARIABLES. NMAX : ROW DIMENSION OF C. NMAX MUST BE GREATER OR EQUAL TO N. MNN : MUST BE EQUAL TO M + N. C(NMAX, NMAX): OBJECTIVE FUNCTION MATRIX WHICH SHOULD BE SYMMETRIC AND POSITIVE DEFINITE. IF IWAR(1) = 0, C IS SUPPOSED TO BE THE CHOLESKEY-FACTOR OF ANOTHER MATRIX, I. E. C IS UPPER TRIANGULAR. D(NMAX) : CONTAINS THE CONSTANT VECTOR OF THE OBJECTIVE FUNCTION. A(MMAX, NMAX): CONTAINS THE DATA MATRIX OF THE LINEAR CONSTRAINTS. B(MMAX) : CONTAINS THE CONSTANT DATA OF THE LINEAR CONSTRAINTS. XL(N), XU(N): CONTAIN THE LOWER AND UPPER BOUNDS FOR THE VARIABLES. X(N) : ON RETURN, X CONTAINS THE OPTIMAL SOLUTION VECTOR. U(MNN) : ON RETURN, U CONTAINS THE LAGRANGE MULTIPLIERS. THE FIRST M POSITIONS ARE RESERVED FOR THE MULTIPLIERS OF THE M LINEAR CONSTRAINTS AND THE SUBSEQUENT ONES FOR THE MULTIPLIERS OF THE LOWER AND UPPER BOUNDS. ON SUCCESSFUL TERMINATION, ALL VALUES OF U WITH RESPECT TO INEQUALITIES AND BOUNDS SHOULD BE GREATER OR EQUAL TO ZERO.

Circumventing the effects of sampling variability • Sampling variability appears to be an issue: noisy weight distribution with large number of zero weights and some unity weights • Bootstrap the optimization, omitting contiguous 6 -year blocks of the 48 -yr time series – – – yields 43 samples of 42 years shows the sampling variability of the likelihood over subsets of years We average the weights across the samples

Example Six GCMs’ Jul-Aug-Sep precipitation simulations Training period: 1950– 97 Ensembles of between 9 and 24 members

Model Weights – initially, by individual model

Climatological Weights – Multi-model

Model Weights – after second (damping) step

Model Weights – step 2, and Averaged over Subsamples

For more spatially smooth results, the weighting of each grid point is averaged with that of its 8 neighbors, using binomial weighting. X X X X X

Climatological Weights

Combination Forecasts of July-Sept Precipitation After first stage only After second (damping) stage After sampling subperiods After spatial smoothing

Reliability JAS Precip. , 30 S-30 N Below-Normal Observed relative Freq. Above-Normal Bayesian Pooled Individual AGCM Forecast probability (3 -model) from Goddard et al. 2003

RPSS Precip. from Roberson et al. (2004): Mon. Wea. Rev. , 132, 2732 -2744

RPSS 2 -m Temp. from Roberson et al. (2004): Mon. Wea. Rev. , 132, 2732 -2744

Conclusions - Bayesian • The “climatological” (equal-odds) forecast provides a useful prior for combining multiple ensemble forecasts • Sampling problems become severe when attempting to combine many models from a short training period (“noisy weights”) • A two-stage process combines the models together according to a pre-assessment of each against climatology • Smoothing of the weights across data subsamples and spatially appears beneficial

IRI’s forecasts use also a second consolidation scheme, whose result is averaged with the result of the Bayesian scheme. 1. Bayesian scheme 2. Canonical Variate scheme

Canonical Variate Analysis (CVA) A number of statistical techniques involve calculating linear combinations (weighted sums) of variables. The weights are defined to achieve specific objectives: • PCA – weighted sums maximize variance • CCA – weighted sums maximize correlation • CVA – weighted sums maximize discrimination

Canonical Variate Analysis

Canonical Variate Analysis The canonical variates are defined to maximize the ratio of the between-category (separation between the crosses) to the within-category (separation of dots from like-colored crosses) variance.

Conclusion IRI presently using a 2 -tiered prediction system. It is interested in using fully coupled systems also, and is looking into incorporating those. Within its 2 -tiered system it uses 4 SST prediction scenarios, and combines the predictions of 7 AGCMs. The merging of 7 predictions into a single one uses two multi-model ensemble systems: Bayesian and canonical variate. These give somewhat differing solutions, and are presently given equal weight.