Ensemble Prediction Systems and Probabilistic Forecasting Richard Rick
Ensemble Prediction Systems and Probabilistic Forecasting Richard (Rick) Jones SWFDP Training Workshop on Severe Weather Forecasting Bujumbura, Burundi, Nov 11 -16 , 2013 Slides courtesy: UKMO Paul Davies and Chris Tubbs 11/27/2020 1
”The Blame Game” or ”The Passing of The Buck” Atmosphere The atmosphere is ”chaotic” Scientists Errorneous observations misled the NWP Computer models The NWP misled me Forecaster The forecaster misled me Customer/Public 11/27/2020 2
Understanding chaos 11/27/2020 3
Numerical Weather Prediction Forecast distribution The Atmosphere Analysis Error Data Assimilation Forecast Diagnostic Tools and Products Model System 11/27/2020 4
Quantifying uncertainty with ensembles Deterministic Forecast uncertainty Initial Condition Uncertainty X CHAOS Analysis Climatology time 11/27/2020 5
Exchanging the “accurate” forecast with a more “honest” one ψ Dangerous threshold Today’s NWP forecast Today’s EPS forecast -12 0 +12 +24 +36 +48 April 2013 Anders Persson +60 +72 +84 h 6
Correction for systematic errors ψ -12 27/11/2020 0 +12 +24 +36 +48 Anders Persson +60 +72 +84 h 7 7
The final ensemble forecast – with verification ψ ● ●● 70% ● ●● ● obs ● ● ● -12 27/11/2020 0 +12 50% +24 +36 +48 Anders Persson +60 ● +72 ● +84 h 8 8
Probability NWP today We might form an opinion by looking at the last NWPs from the same model, so called “lagged” forecasts Investigations show that “jumpiness” correlates badly to the accuracy of the last forecast NWP yesterday Anders Persson NWP the day before yesterday Ψ 9
Medium range forecasting…with deterministic and EPS information Most common case Latest three NWP Most common case with good agreement between EPS spread and NWP “jumpiness” Latest EPS 27/11/2020 Anders Persson 10
Rather common case Rather poor agreement between larger EPS spread and small NWP “jumpiness”. The analysis system has obviously managed to avoid possible problems because the NWP is not very “jumpy” Should the forecasters be more certain than the EPS indicates? 27/11/2020 Anders Persson 11
Not uncommon case Rather poor agreement between small EPS spread and large NWP “jumpiness”. The perturbations have not been quite able to cover the analysis uncertainties Should the forecasters be more uncertain than the EPS indicates? 27/11/2020 Anders Persson 12
Rare case Poor agreement between the main directions of the EPS and the NWP Best choice: create a “super ensemble” See TIGGE This puts the forecasters in a very difficult situation and there is not enough experience or investigations about this situation 27/11/2020 Anders Persson 13
The Effect of Chaos • We can usually forecast the general pattern of the weather up to about 3 days ahead. • Chaos then becomes a major factor Tiny errors in our analysis of the current state of the atmosphere lead to large errors in the forecast – these are both equally valid 4 -day forecasts. • Fine details (eg rainfall) have shorter predictability 11/27/2020 14
Ensembles • In an ensemble forecast we run the model many times from slightly different initial conditions • This provides a range of likely forecast solutions which allows forecasters to: – assess possible outcomes; – estimate risks – gauge confidence. 11/27/2020 15
Reminder on scales and predictability 11/27/2020 16
Temporal Resolution 1000 km Space Scale Tropical Cyclone 100 km MCS 10 km 1 km Front Thunderstorm Hail shaft Lifetime 10 mins 1 hr 12 hrs 3 days Predictability? 30 mins 3 hrs 36 hrs 9 days 11/27/2020 17
Short-range Ensembles ECMWF EPS has transformed the way we do Medium-Range Forecasting • Uncertainty also in short-range: – Rapid Cyclogenesis often poorly forecast deterministically – Uncertainty of sub-synoptic systems (eg thunderstorms) – Many customers most interested in short-range • Assess ability to estimate uncertainty in local weather – QPF – Cloud Ceiling, Fog – Winds etc 11/27/2020 18
Initial conditions perturbations • Perturbations centred around 4 D-Var analysis • Transforms calculated using same set of observations as used in 4 D-Var (including all satellite obs) within +/- 3 hours of data time • Ensemble uses 12 hour cycle (data assimilation uses 6 hour cycle) 11/27/2020 19
Initial conditions perturbations Differences with ECWMF Singular Vectors: - It focuses on errors growing during the assimilation period, not growing period: - Suitable for Short-range! - Calculated using the same resolution than the forecast - ETKF includes moist processes - Running in conjunction with stochastic physics to propagate effect 11/27/2020 20
Model error: parameterisations Random parameters Parameter §QUMP (Murphy et al. , 2004) §Initial stoch. Phys. Scheme for the UM (Arribas, 2004) Scheme min/std/Max Entrainment rate CONVECTION 2/3/5 Cape timescale CONVECTION 30 / 120 RH critical LRG. S. CLOUD 0. 6 / 0. 8 / 0. 9 Cloud to rain (land) LRG. S. CLOUD 1 E-4/8 E-4/1 E-3 Cloud to rain (sea) LRG. S. CLOUD 5 E-5/2 E-4/5 E-4 Ice fall LRG. S. CLOUD 17 / 25. 2 / 33 Flux profile param. BOUNDARY L. 5 / 10 / 20 Neutral mixing length BOUNDARY L. 0. 05 / 0. 15 / 0. 5 Gravity wave const. GRAVITY W. D. 1 E-4/7. 5 E-4 Froude number GRAVITY W. D. 2/2/4 11/27/2020 21
Using probabilities • Recipients of forecasts & warnings are sensitive to different levels of risk: reflecting cost of mitigation vs expected loss • An intelligent response to forecasts & warnings depends on risk analysis, requiring knowledge of impact probability • Use of ensembles to estimate probability at longer lead times is well established in meteorology 11/27/2020 22
0% 25% 10 20 50% 75% 100% 30 25%=12 -13 forecasts Ψ= 10 -22 40 25%=12 -13 forecasts Ψ= 23 -30 25%=12 -13 forecasts Ψ= 31 -37 50 Ψ 25%=12 -13 forecasts Ψ= 38 -50 Median 50 -50% | | ||| 10 || | ||| | | || || ||| 20 |||| || | | | ||| | | 30 WMO SWFDP Macau 10 April 2013 Anders Persson 40 || 50 Ψ 23
Probability maps 11/27/2020 24
11/27/2020 25
EC Total ppn prob > 20 mm 12 z Tue – 12 z Wed 11/27/2020 26
EPSgrammes 11/27/2020 27
max 90% median (50%) Total cloud cover Deterministic 75% 6 hourly precipitation 25% 10 m wind speed EPS control 10% 2 m temperature min 11/27/2020 28
Bujumbura 06 Nov 12 UTC 11/27/2020 30
11/27/2020 31
The Extreme Forecast Index (EFI) 11/27/2020 32
Extreme forecast index (EFI) • EFI measures the distance between the EPS cumulative distribution and the model climate distribution • Takes values from – 1 (all members break climate minimum records) and +1 (all beyond model climate records) • The main idea is to have an index that can be conveniently mapped – removing the effect from different climatologies – to use as an “alarm bell” 11/27/2020 33
EFI • Experience suggests that EFI magnitudes of 0. 5 - 0. 8 (irrespective of sign) can be generally regarded as signifying that "unusual" weather is likely whilst magnitudes above 0. 8 usually signify that "very unusual" or extreme weather is likely. Although larger EFI values indicate that an extreme event is more likely, the values do not represent probabilities as such 11/27/2020 34
Advantages with probability density functions Means and asymmetric variances are easily spotted 100% Clim. EPS mean Climate distribution 0% 11/27/2020 EPS distribution Temperature 35
Advantages with probability density functions Means and asymmetric variances are easily spotted 100% Climate distribution EPS distribution 0% Clim. mean 11/27/2020 EPS mean Temperature 36
EFI ~ 70% The EFI generally does not take the probability into account EFI ~ 70% 11/27/2020 37
EFI ~ 50% For temperature the EFI can take values < 0 EFI ~ -50% 11/27/2020 38
EFI 2 m Temp 11/27/2020 39
EFI 10 m gusts 11/27/2020 40
EFI wind speed 11/27/2020 41
EFI precip 11/27/2020 42
Other products 11/27/2020 43
Tropical strike probability 11/27/2020 44
Tropical strike 11/27/2020 45
Ensemble mean 11/27/2020 46
Forecast Probability 11/27/2020 47
UKMO African LAM 11/27/2020 48
UKMO African LAM 11/27/2020 49
Ensemble forecasts • Ensemble forecasts help us 1. To judge the (un)certainty of the weather situation 2. To acquire probability estimates of anomalous events (extreme or high impact) 3. To get the most accurate and least “jumpy” deterministic forecast value 11/27/2020 50
Questions & Answers 11/27/2020 52
- Slides: 50