Numerical space weather prediction can meteorologists forecast the

Numerical space weather prediction: can meteorologists forecast the way ahead? Dr Mike Keil, Dr Richard Swinbank and Dr Andrew Bushell ESWW, November 2005 © Crown copyright 2005 Page 1

Introduction • What National Met Services (NMS) do and how does this fit in with Space Weather? • How did they get there? • What can be learned for Numerical Space Weather Prediction? • What does the future hold? © Crown copyright 2005 Page 2

National Met Services What do they do? © Crown copyright 2005 Page 3

How’s it done? Numerical Weather Prediction data assimilation Observations Model T+… T+48 T+24 T+12 T+6 T+5 T+4 T+3 T+2 T+1 Analysis © Crown copyright 2005 Forecast Page 4

Development of NWP: Vilhelm Bjerknes (1862 -1951) had a vision! L. F. Richardson’s first forecast sometime between 1916 and 1918. 1950 Charney ran the first forecast on a computer It took longer to subjectively quantify the ICs than run the forecasts! So far, no mention of Data Assimilation… Clearly a need for an objective way of specifying the initial conditions and analysis © Crown copyright 2005 Page 5

Development of DA: • 1949 Panofski had been creating objective analysis using interpolation techniques • 1954 Gilchrist and Cressman had two ideas: • numerical forecasts as a source of background info • automatic quality control of data • 1955 Bergothorsson and Doos – analyse observation increments • 1961 Thompson – use DA to propagate info into data voids © Crown copyright 2005 Page 6

NWP in the present day: • Development of NWP models and increased computer performance has led to more sophisticated assimilation schemes State Corrected forecast Initial forecast Observations T+0 © Crown copyright 2005 T+6 Time Page 7

The virtuous cycle observations science assimilation modelling © Crown copyright 2005 Page 8

1953 Storm © Crown copyright 2005 Page 9

The virtuous cycle observations science assimilation modelling © Crown copyright 2005 Page 10

Lessons from Numerical Weather Prediction § Data Assimilation combines information from observations with a background state. § The background state could come from a number of sources: subjective analysis, climatological averages, empirical models § To exploit the full potential of data assimilation, the background state should be produced using a physically-based numerical model. § This should be the approach to follow for SW assimilation © Crown copyright 2005 Page 11

Lessons from Numerical Weather Prediction § A physically-based numerical model is not just required for data assimilation. § A physically-based model is an essential part in fully establishing the virtuous cycle. § Empirical models can serve a useful purpose; however their potential for development is restricted. § Physically-based models provide a route for long-term space weather scientific growth © Crown copyright 2005 Page 12

Lessons – data issues NMS have experience in handling and processing vast amounts of data § Satellite data is the most obvious crossover area § Co-ordination is handled by WMO § Global Observing System § info / education / transition § Most NMS assimilate data from around 25 operational satellites §What about experimental satellites? §WMO set the “rules” § GTS infrastructure © Crown copyright 2005 Page 13

Lessons – common data sources GPS RO observations is a good example Mid-90 s humidity and temperature profiles from GPS Realistic assimilation first carried out at the Met Office Operational use next year Techniques can be applied to assimilate TEC COSMIC: Constellation Observing System for Meteorology, Ionosphere and Climate 6 space craft – provide TEC, allow operational monitoring Data available in near real time for scientific research © Crown copyright 2005 Page 14

Lessons – other questions along the way There are issues relevant to SW that have already been tackled by the met community: § Bias correction of data § Assimilation of derived products or raw values? § Pain before the gain – increasing complexity § Potential for development § Timeliness of data § Ensembles © Crown copyright 2005 Page 15

The future: operational met models Most operational met models are pushing beyond the stratosphere § Why? § Met Office global model will have a lid at 63 km § Research model with a 86 km lid § Other centres go higher – eg CMAM 210 km § Sensible to have a joined-up approach to common issues © Crown copyright 2005 Page 16

The future: scientific collaboration The Met Office are interested in Space Weather science! Potential areas of research: § Coupling between weather and space weather models § Lower boundary forcings? § Upwards/downwards control? § Fully coupled models (whole atmosphere approach)? § Applying data assimilation expertise to space weather assimilation § Radio occultation assimilation experience § Funding © Crown copyright 2005 Page 17

The future: numerical space weather prediction Within a decade (? ) there will be a requirement for operational numerical space weather prediction § Why? Primarily military with commercial applications § How? § Following the framework used in operational NWP § Learning from met experience in key areas § Utilising the facilities of NMS eg supercomputers, observation supply, 24/7 capabilities, down-stream dissemination to end users § This way of working already exists in operational oceanography at the Met Office © Crown copyright 2005 Page 18

Conclusions § The development over many years of NWP presents a framework for Numerical Space Weather Prediction § Fully establish the “virtuous cycle” for SW § Some pain can be avoided by learning from the met community! § Science can be pushed forward through collaboration § Operational Space Weather within a decade? § National Met Services offer crucial facilities § Successful partnerships of this kind already exist § Thanks for listening! © Crown copyright 2005 Page 19

Questions © Crown copyright 2005 Page 20

The framework of modern DA: Panofski Bergothorsson and Doos Gilchrist and Cressman data assimilation Model Observations Thompson Forecast Analysis © Crown copyright 2005 T+6 T+5 T+4 T+3 T+2 T+1 Bjerknes / Richardson / Charney Page 21

DA: hierarchy § Most assimilation schemes operate sequentially. § As long as the evolution of errors is close to linear, an extended Kalman filter is the optimum statistical assimilation method. § Hierarchy of different approximations to the Kalman filter: § § § Direct insertion Nudging Statistical interpolation 3 D-variational (old Met Office system) 4 D-variational (current Met Office global system) Ensemble Kalman Filter § Choose appropriate level of complexity / cost. © Crown copyright 2005 Page 22

DA: cost function § Analysis is found by minimising a cost function quantifying misfit between model fields x and both obs yo and background xb Where y=H(x) is a prediction of yo § In 3 D-VAR, the analysis is calculated using observations at one particular time § In 4 D-VAR, the analysis uses observations at their correct validity time § Met Office system written in incremental form © Crown copyright 2005 Page 23

DA: 4 D Var § 4 D-VAR uses observations over a given time window. § Allows use of observations at correct time, and exploit information in a time sequence. § Requires use of a (simplified, linear) model and its adjoint. (Not clear what level of simplification is appropriate for ionosphere). © Crown copyright 2005 Page 24

DA: summary § 3 DVAR (with a 6 h assimilation cycle for global model) currently used for the stratospheric version of the UM. 4 D VAR in the global model was implemented during 2004. § Main advantage of 4 DVAR is use of observations at correct time / use of time sequence of obs. Requires adjoint of forecast model. (3 DVAR only requires adjoint of observation operator. ) § Kalman Filter updates error covariance as well as state – much more expensive; requires drastic simplification. © Crown copyright 2005 Page 25