Systematic timing errors in kmscale NWP precipitation forecasts

Systematic timing errors in km-scale NWP precipitation forecasts Marion Mittermaier Model diagnostics and novel verification methods, Weather Science

Outline 1. Introduction 2. Method and rationale 3. Offsets as described by correlations 4. Impact of offsets on frequency bias and skill scores 5. Conclusions

Introduction

Introduction Spatial offsets can have a temporal origin 1. Double penalty effect Ø Errors are counted as false alarms and misses. Ø Detail penalised, closeness not rewarded 2. Unskilful scales Ø Grid-scale detail should not be believed Ø Lorenz (1969) argued that the ability to resolve smaller scales would result in forecast errors growing more rapidly -> more noise © Crown copyright Met Office

Methodology

Data and method • Consider 18 months of 36 h forecasts from the 1. 5 km Met Office Unified Model (UKV); compared to 2 km rainfall accumulations from the UK radar network. • Filter the noisy unskilful spatial scales out of the field through smoothing/upscaling to 12 km. • Correlate the hourly forecast accumulation to the adjacent hourly radar accumulations, up to 3 h hours either side Which offset has the largest correlation? • Compute the frequency bias and skill scores for a range of thresholds at all offset times Where is the bias and the score optimised? © Crown copyright Met Office

Caveats • Correlation measures the pattern association between two fields. It is insensitive to the intensity or bias and therefore, to some extent to the imperfections in the observations. • The categorical stats on the other hand are sensitive to the observations characteristics, which are known to be influential in this case. • Upscaling to 8 times the native grid resolution (from 1. 5 km to 12 km) may still not be enough for the traditional categorical statistics.

Two-dimensional distribution of maximum correlations as a function of offset • Hourly forecast correlations are separated by initialisation time • Showing the distribution of hourly maximum correlations aggregated over months as a function of initialisation and time offset. • • Evidence of offsets at early lead times for more convective months (both summer and winter) Less evidence of offsets during autumn (dominated by frontal rain) January 2012 July May 2012 SLOW FAST

Evolution of correlation coefficients separated by forecast run • June 2011 -Dec 2012 • Hourly forecast correlations are separated by initialisation time. • Average correlations as a function of lead time and offset. • Skewed towards model being too slow. • Later lead times often tending towards flat. Forecast error/loss of predictability dominating. SLOW FAST

Frequency bias as a function of lead time and offsets • Focus on 12 -18 Z for each model initialisation for afternoon convection. • Hourly forecast frequency bias for 4 mm/h separated by initialisation time. • Generally an over. NAE here slow forecast bias; offsets appear to have larger biases. 4 km • 0 -offset does not necessarily have best bias. © Crown copyright Met Office

ETS and SEDS as a function of lead time and offsets +ve = offset better • Focus on 12 -18 Z for each model initialisation for afternoon convection. • Hourly ETS or SEDS for 4 mm/h relative to 0 -offset, separated by initialisation time. • 0 -offset is not necessarily the most skilful based on the SEDS or ETS; over-forecast bias does not necessarily translate to maximised scores. • There is no clear signal for any particular offset to be more skilful overall. © Crown copyright Met Office

Concluding remarks

Preliminary findings • Start time of the run is influential irrespective of time of year with large variations between different initialisations. Influence of data being assimilated? • Seasonal variations have a strong influence, especially autumn stands out as being different. • Months dominated by more convective rain appear to show a delay in onset of precipitation for the afternoon hours. • Drop in correlation coefficient beyond the first few hours can be marked. Average correlation coefficients show some skewness towards the model being too slow. • Very occasionally better predictability in terms of the pattern is maintained beyond the first 6 hours. • Skill scores show more response to timing errors for larger accumulation thresholds. • Model tends to over-forecast larger accumulations.

Thanks for listening!