Bias Adjusted Precipitation Scores Fedor Mesinger NOAAEnvironmental Modeling
Bias Adjusted Precipitation Scores Fedor Mesinger NOAA/Environmental Modeling Center and Earth System Science Interdisciplinary Center (ESSIC), Univ. Maryland, College Park, MD VX-Intercompare Meeting Boulder, 20 February 2007
Most popular “traditional statistics”: ETS, Bias Problem: what does the ETS tell us ?
“The higher the value, the better the model skill is for the particular threshold” (a recent MWR paper)
Example: Three models, ETS, Bias, 12 months, “Western Nest” Is the green model loosing to red because of a bias penalty?
What can one do ?
J 12. 6 17 th Prob. Stat. Atmos. Sci. ; 20 th WAF/16 th NWP (Seattle AMS, Jan. ‘ 04) BIAS NORMALIZED PRECIPITATION SCORES Fedor Mesinger 1 and Keith Brill 2 1 NCEP/EMC and UCAR, Camp Springs, MD 2 NCEP/HPC, Camp Springs, MD
Two methods of the adjustment for bias (“Normalized” not the best idea) 1. d. Hd. F method: Assume incremental 2. change in hits per incremental change in 3. bias is proportional to the “unhit” area, O-H 4. Objective: obtain ETS adjusted to unit bias, 5. to show the model’s accuracy in placing precipitation 6. (The idea of the adjustment to unit bias to arrive at placement accuracy: 7. Shuman 1980, NOAA/NWS Office Note) 2. Odds Ratio method: different objective
Forecast, Hits, and Observed (F, H, O) area, or number of model grid boxes:
d. Hd. F method, assumption: can be solved; a function H (F) obtained that satisfies the three requirements:
• Number of hits H -> 0 for F -> 0; • The function H(F) satisfies the known value of H for the model’s F, the pair denoted by Fb, Hb, and, • H(F) -> O as F increases
Bias adjusted eq. threats West NMM GFS Eta
A downside: if Hb is close to Fb, or to O, it can happen that d. H/d. F > 1 for F -> 0 Physically unrealistic ! Reasonableness requirement:
“d. Hd. M” method: Assume as F is increased by d. F, ratio of the infinitesimal increase in H, d. H, and that in false alarms d. M=d. F-d. H, is proportional to the yet unhit area:
One obtains ( Lambertw, or Product. Log in Mathematica, is the inverse function of )
H (F) now satisfies the additional requirement: d. H/d. F never > 1
d. Hd. F method H=O H=F H(F) F b , Hb
d. Hd. M method H=O H=F H(F) F b , Hb
Results for the two “focus cases”, d. Hd. M method (Acknowledgements: John Halley Gotway, data; Dušan Jović, code and plots)
5/13 Case d. Hd. M wrf 2 caps wrf 4 ncar wrf 4 ncep
6/01 Case d. Hd. M wrf 2 caps wrf 4 ncar wrf 4 ncep
Impact, in relative terms, for the two cases is small, because the biases of the three models are so similar !
One more case, for good measure:
5/25 Case d. Hd. M wrf 2 caps wrf 4 ncar wrf 4 ncep
Comment: Scores would have generally been higher had the verification been done on grid squares greater than ~4 km This would have amounted to a poor-person’s version of “fuzzy” methods !
- Slides: 25