Evaluation of AMPS Forecasts Using SelfOrganizing Maps SOMs

Evaluation of AMPS Forecasts Using Self-Organizing Maps (SOMs) John J. Cassano Cooperative Institute for Research in Environmental Science and Program in Atmospheric and Oceanic Sciences University of Colorado at Boulder Beardmore Glacier, Jan 2004

Outline • What are SOMs? • Application of SOMs for model evaluation studies • Application of SOM Analysis to AMPS data • Conclusions / Future Work

What are SOMs? • SOM - Self-Organizing Map • SOM technique uses an unsupervised learning algorithm (neural net) • Clusters data into a user selected number of nodes • SOM algorithm attempts to find nodes that are representative of the data in the training set – More nodes in areas of observation space with many data points – Fewer nodes in areas of observation space with few data points • SOMs are in use across a wide range of disciplines

Application of SOMs for Model Evaluation Studies • Synoptic pattern classification • Frequency of occurrence of synoptic patterns • Determine model errors for different synoptic patterns

Application of SOM Analysis to AMPS Data • Train SOM with AMPS SLP data – Result is a synoptic pattern classification • Evaluate frequency of occurrence of synoptic patterns predicted by AMPS as a function of forecast duration – Map 0 h, 24 h, and 48 h forecasts to SOM • Mis-mapping of AMPS forecasts • Model validation statistics for specific synoptic patterns (ongoing work) – Calculate model error statistics at points of interest (Willie Field) for different synoptic patterns – Are certain synoptic patterns prone to bias (e. g. error in predicted wind speed or direction)?

AMPS Data for SOM Analysis • SLP over Ross Sea sector of AMPS 30 km model domain • Summer only (NDJ) • 00 Z AMPS simulations from Jan 2001 through Feb 2003 – 186 model simulations • Evaluate 0, 24, and 48 h AMPS forecasts

AMPS SOM Analysis Domain

Synoptic Pattern Classification

Frequency of Occurrence

Misprediction of Synoptic Patterns • Consider all of the time periods for which the model analyses map to a particular node – For these time periods determine which nodes the model predictions map to • From this analysis we can determine biases in the model predictions of specific synoptic patterns relative to the model analyses – Percent of cases that map to the correct node – Mis-mapping of model predictions between nodes

AMPS 24 h Forecasts 86. 7% 64. 7% 73. 2% 1 1 2 1 81. 1% 1 71. 1% 2 4 100% 1

AMPS 48 h Forecasts 86. 7% 58. 8% 1 4 2 1 67. 6% 63. 4% 2 65. 8% 1 5 100% 2

Model Errors for Synoptic Patterns • Compare model predictions to in-situ atmospheric measurements • Calculate model validation statistics for all time periods that map to each node • Look for model errors that vary from node to node • This is ongoing work using AMPS data – This technique has been applied to ARCMIP data

An ARCMIP example SHEBA Surface Pressure (DJF)

Conclusions / Future Work • The use of SOMs provides an alternate method of evaluating model performance – Identify synoptic patterns which are over or underpredicted – Determine model tendency for mis-prediction of certain synoptic types – Provide information on model errors related to specific synoptic patterns • Complete SOM analysis for entire AMPS archive (Jan 2001 - present) • Calculate model biases as a function of forecast time and synoptic patterns