Ensemble Prediction Systems and Probabilistic Forecasting Richard Rick
Ensemble Prediction Systems and Probabilistic Forecasting Richard (Rick) Jones SWFDP Training Workshop on Severe Weather Forecasting Bujumbura, Burundi, Nov 11 -16 , 2013 Slides courtesy: UKMO Paul Davies and Chris Tubbs 3/2/2021 1
”The Blame Game” or ”The Passing of The Buck” Atmosphere The atmosphere is ”chaotic” Scientists Errorneous observations misled the NWP Computer models The NWP misled me Forecaster The forecaster misled me Customer/Public 3/2/2021 2
Understanding chaos 3/2/2021 3
Numerical Weather Prediction Forecast distribution The Atmosphere Analysis Error Data Assimilation Forecast Diagnostic Tools and Products Model System 3/2/2021 4
Quantifying uncertainty with ensembles Deterministic Forecast uncertainty Initial Condition Uncertainty X CHAOS Analysis Climatology time 3/2/2021 5
Exchanging the “accurate” forecast with a more “honest” one ψ Dangerous threshold Today’s NWP forecast Today’s EPS forecast -12 0 +12 +24 +36 +48 April 2013 Anders Persson +60 +72 +84 h 6
Correction for systematic errors ψ -12 02/03/2021 0 +12 +24 +36 +48 Anders Persson +60 +72 +84 h 7 7
The final ensemble forecast – with verification ψ ● ●● 70% ● ●● ● obs ● ● ● -12 02/03/2021 0 +12 50% +24 +36 +48 Anders Persson +60 ● +72 ● +84 h 8 8
Probability NWP today We might form an opinion by looking at the last NWPs from the same model, so called “lagged” forecasts Investigations show that “jumpiness” correlates badly to the accuracy of the last forecast NWP yesterday Anders Persson NWP the day before yesterday Ψ 9
Medium range forecasting…with deterministic and EPS information Most common case Latest three NWP Most common case with good agreement between EPS spread and NWP “jumpiness” Latest EPS 02/03/2021 Anders Persson 10
Rather common case Rather poor agreement between larger EPS spread and small NWP “jumpiness”. The analysis system has obviously managed to avoid possible problems because the NWP is not very “jumpy” Should the forecasters be more certain than the EPS indicates? 02/03/2021 Anders Persson 11
Not uncommon case Rather poor agreement between small EPS spread and large NWP “jumpiness”. The perturbations have not been quite able to cover the analysis uncertainties Should the forecasters be more uncertain than the EPS indicates? 02/03/2021 Anders Persson 12
Rare case Poor agreement between the main directions of the EPS and the NWP Best choice: create a “super ensemble” See TIGGE This puts the forecasters in a very difficult situation and there is not enough experience or investigations about this situation 02/03/2021 Anders Persson 13
The Effect of Chaos • We can usually forecast the general pattern of the weather up to about 3 days ahead. • Chaos then becomes a major factor Tiny errors in our analysis of the current state of the atmosphere lead to large errors in the forecast – these are both equally valid 4 -day forecasts. • Fine details (eg rainfall) have shorter predictability 3/2/2021 14
Ensembles • In an ensemble forecast we run the model many times from slightly different initial conditions • This provides a range of likely forecast solutions which allows forecasters to: – assess possible outcomes; – estimate risks – gauge confidence. 3/2/2021 15
Reminder on scales and predictability 3/2/2021 16
Temporal Resolution 1000 km Space Scale Tropical Cyclone 100 km MCS 10 km 1 km Front Thunderstorm Hail shaft Lifetime 10 mins 1 hr 12 hrs 3 days Predictability? 30 mins 3 hrs 36 hrs 9 days 3/2/2021 17
Short-range Ensembles ECMWF EPS has transformed the way we do Medium-Range Forecasting • Uncertainty also in short-range: – Rapid Cyclogenesis often poorly forecast deterministically – Uncertainty of sub-synoptic systems (eg thunderstorms) – Many customers most interested in short-range • Assess ability to estimate uncertainty in local weather – QPF – Cloud Ceiling, Fog – Winds etc 3/2/2021 18
Initial conditions perturbations • Perturbations centred around 4 D-Var analysis • Transforms calculated using same set of observations as used in 4 D-Var (including all satellite obs) within +/- 3 hours of data time • Ensemble uses 12 hour cycle (data assimilation uses 6 hour cycle) 3/2/2021 19
Initial conditions perturbations Differences with ECWMF Singular Vectors: - It focuses on errors growing during the assimilation period, not growing period: - Suitable for Short-range! - Calculated using the same resolution than the forecast - ETKF includes moist processes - Running in conjunction with stochastic physics to propagate effect 3/2/2021 20
Model error: parameterisations Random parameters Parameter §QUMP (Murphy et al. , 2004) §Initial stoch. Phys. Scheme for the UM (Arribas, 2004) Scheme min/std/Max Entrainment rate CONVECTION 2/3/5 Cape timescale CONVECTION 30 / 120 RH critical LRG. S. CLOUD 0. 6 / 0. 8 / 0. 9 Cloud to rain (land) LRG. S. CLOUD 1 E-4/8 E-4/1 E-3 Cloud to rain (sea) LRG. S. CLOUD 5 E-5/2 E-4/5 E-4 Ice fall LRG. S. CLOUD 17 / 25. 2 / 33 Flux profile param. BOUNDARY L. 5 / 10 / 20 Neutral mixing length BOUNDARY L. 0. 05 / 0. 15 / 0. 5 Gravity wave const. GRAVITY W. D. 1 E-4/7. 5 E-4 Froude number GRAVITY W. D. 2/2/4 3/2/2021 21
Using probabilities • Recipients of forecasts & warnings are sensitive to different levels of risk: reflecting cost of mitigation vs expected loss • An intelligent response to forecasts & warnings depends on risk analysis, requiring knowledge of impact probability • Use of ensembles to estimate probability at longer lead times is well established in meteorology 3/2/2021 22
0% 25% 10 20 50% 75% 100% 30 25%=12 -13 forecasts Ψ= 10 -22 40 25%=12 -13 forecasts Ψ= 23 -30 25%=12 -13 forecasts Ψ= 31 -37 50 Ψ 25%=12 -13 forecasts Ψ= 38 -50 Median 50 -50% | | ||| 10 || | ||| | | || || ||| 20 |||| || | | | ||| | | 30 WMO SWFDP Macau 10 April 2013 Anders Persson 40 || 50 Ψ 23
Probability maps 3/2/2021 24
3/2/2021 25
EC Total ppn prob > 20 mm 12 z Tue – 12 z Wed 3/2/2021 26
EPSgrammes 3/2/2021 27
max 90% median (50%) Total cloud cover Deterministic 75% 6 hourly precipitation 25% 10 m wind speed EPS control 10% 2 m temperature min 3/2/2021 28
Bujumbura 06 Nov 12 UTC 3/2/2021 30
3/2/2021 31
The Extreme Forecast Index (EFI) 3/2/2021 32
Extreme forecast index (EFI) • EFI measures the distance between the EPS cumulative distribution and the model climate distribution • Takes values from – 1 (all members break climate minimum records) and +1 (all beyond model climate records) • The main idea is to have an index that can be conveniently mapped – removing the effect from different climatologies – to use as an “alarm bell” 3/2/2021 33
EFI • Experience suggests that EFI magnitudes of 0. 5 - 0. 8 (irrespective of sign) can be generally regarded as signifying that "unusual" weather is likely whilst magnitudes above 0. 8 usually signify that "very unusual" or extreme weather is likely. Although larger EFI values indicate that an extreme event is more likely, the values do not represent probabilities as such 3/2/2021 34
Advantages with probability density functions Means and asymmetric variances are easily spotted 100% Clim. EPS mean Climate distribution 0% 3/2/2021 EPS distribution Temperature 35
Advantages with probability density functions Means and asymmetric variances are easily spotted 100% Climate distribution EPS distribution 0% Clim. mean 3/2/2021 EPS mean Temperature 36
EFI ~ 70% The EFI generally does not take the probability into account EFI ~ 70% 3/2/2021 37
EFI ~ 50% For temperature the EFI can take values < 0 EFI ~ -50% 3/2/2021 38
EFI 2 m Temp 3/2/2021 39
EFI 10 m gusts 3/2/2021 40
EFI wind speed 3/2/2021 41
EFI precip 3/2/2021 42
Other products 3/2/2021 43
Tropical strike probability 3/2/2021 44
Tropical strike 3/2/2021 45
Ensemble mean 3/2/2021 46
Forecast Probability 3/2/2021 47
UKMO African LAM 3/2/2021 48
UKMO African LAM 3/2/2021 49
Ensemble forecasts • Ensemble forecasts help us 1. To judge the (un)certainty of the weather situation 2. To acquire probability estimates of anomalous events (extreme or high impact) 3. To get the most accurate and least “jumpy” deterministic forecast value 3/2/2021 50
Questions & Answers 3/2/2021 52
- Slides: 50