GEOGG 142 GMES Calibration validation of EO products

GEOGG 142 GMES Calibration & validation of EO products Dr. Mat Disney mdisney@geog. ucl. ac. uk Pearson Building room 113 020 7679 0592 www. geog. ucl. ac. uk/~mdisney

Outline · Calibration & validation · · Example: MODIS LAI and NPP products Meaning of parameters? ? Time, space, measurements? Scaling? Good place to start: CEOS Working Group on Cal/Val, Land Product Validation sub-group: http: //lpvs. gsfc. nasa. gov/ and see eg Biophysical & references therein: http: //lpvs. gsfc. nasa. gov/LAI_references. html and good practice guidelines: http: //lpvs. gsfc. nasa. gov/PDF/CEOS_LAI_PROTOCOL_Aug 2014_v 2. 0. 1. pdf 2

Calibration & validation? • Calibration: – Process of adjusting empirical relationship between empirical estimates of biophysical parameter estimated from 2 (or more) sources – e. g. ground-based and EO-derived LAI, or NDVI and f. APAR etc. – Local calibration to estimate fitting parameters (slope, intercept for a linear relationship) and uncertainty – Limitations? ? ? 3

Calibration & validation? • Calibration: – process of converting an instrument reading to a physically meaningful measurement – Particularly radiometric calibration i. e. from DN to radiance measurement – OR – Process of adjusting empirical relationship between estimates of biophysical parameter estimated from 2 (or more) sources e. g. ground and EO-derived 4

Calibration & validation? • Validation: – experiments designed to verify instrument measurements using independent measurements – Caveat: EO ‘validation’ often means testing one model-derived estimate against another • EO LAI, NPP etc. all require models • Field estimates of LAI also require models – i. e. NOT validation in true sense at all – See later: when is LAI not LAI • BUT: cal/val both essential to scientific remote sensing 5

Aside: focus on validation here but …. · Eg LAI CEOS WGCV: recommends CALIBRATION comparison between EO and ground-based · NEEDS: reference estimates traceable to in situ measurements · 3 sources available: 1. 2. 3. LAI measurements over individual Elementary Sampling Units (ESUs) Spatially extensive LAI reference maps based on data driven relationships calibrated using ESU LAI Spatially extensive LAI reference maps based on functional relationships calibrated using ESU LAI. · And what is an ESU? See later 6

Validation example: MODIS NPP · Productivity recap: Net Primary Productivity (NPP) · annual net carbon exchange · quantifies actual plant growth · Conversion to biomass (woody, foliar, root) – i. e. not just C 02 fixation (GPP) – NPP = GPP – Ra (plant respiration) • MODIS product example used here – MOD 17 GPP/NPP ATBD • ntsg. umt. edu/MOD 17 • http: //neo. sci. gsfc. nasa. gov/Search. html – Turner et al (2005) 7

Productivity recap • • GPP/NPP from MODIS Requirements? MOD 17 ATBD Running et al. (2004) Turner et al. (2005) Zhao et al. (2005) Heinsch et a. (2006) 8

MODIS GPP/NPP + QC? ? http: //secure. ntsg. umt. edu/projects/index. php/ID/ca 2901 a 0/fuseaction/projects. detail. htm 9

MOD 17 validation approach · Need to address time (days to years) and space (local to global) · Permanent network of ground validation sites · Quantify seasonal and interannual dynamics of ecosystem activity (cover time domain) · EO to quantify heterogeneity of biosphere · Quantify land cover, land cover change dynamics · Models to: · Quantify, understand unmeasured ecosystem · Provide predictive capability (in time AND space) 10

How on earth…. . ? ? • …can we “validate” an EO-derived estimate of something that depends on soil, climate, land cover etc. ? • Given that it requires various models to go from a satellite observation (radiance), to reflectance, to LAI/FAPAR, to PSN, to GPP to NPP • At 500 m-1 km pixels. Globally. • And how do you even “measure” NPP on the ground? ? 11

So, how might we validate? • Need to consider scale • Relate measurements at the small scale to 1 km pixels? ? • Flux tower approach • Eg BIGFOOT approach, FLUXNET etc. • Measurements and validation at many scales • Models to bridge time/space scales Fig from MOD 17 ATBD – (but how good are models…? ) 12

Ecosystem measurements: FLUXNET Fig from MOD 17 ATBD http: //daac. ornl. gov/FLUXNET/ 13

Ecosystem measurements: FLUXNET 2009 http: //daac. ornl. gov/FLUXNET/ http: //earthobservatory. nasa. gov/Features/Fluxnet/ 14

Ecosystem measurements: FLUXNET 2009 -11 http: //daac. ornl. gov/FLUXNET/ http: //www. fluxnet. ornl. gov/fluxnet/graphics. cfm http: //earthobservatory. nasa. gov/Features/Fluxnet/ 15

Ecosystem measurements: FLUXNET http: //daac. ornl. gov/FLUXNET/ 16

Ecosystem measurements: FLUXNET by biome Some distribution of biome types, but clearly biased in location Even considering only limited biomes http: //daac. ornl. gov/FLUXNET/ 17

Big. Foot approach to validating MODIS NPP · E. g. Turner et al. (2005), 6 sites spanning range of vegetation and climate · Crops, forest, tundra, grassland · 5 x 5 km site at each plot (25 MODIS pixels) · Flux tower & 100 (25 x 25 m) sample plots within each area, seasonally measured for LAI and above-ground (A)NPP (from harvested leaf and wood material) · Land cover from high res EO · Use measured data at sample plots to calculate NPP, GPP · Spatially distribute across site using (vegetation-calibrated) Biome. BGC model · Requires daily met data, land cover, LAI · Gives measured estimate from ground AND flux tower 18

Big. Foot v flux tower GPP Turner et al. (2005) 19

Big. Foot v MODIS GPP Not such good agreement as for flux tower (not surprisingly) Turner et al. (2005) 20

Comparison of MODIS NPP with flux data Differences due to Ra (autotrophic i. e. plant respiration)? PAR, VPD differences between those from DAO and actual? (VPD = deficit between the amount of moisture in the air and how much moisture the air can hold when it is saturated) Turner et al. (2005) 21

DAO PAR, VPD? Clearly some sites better agreement than others PAR generally good (relatively easy to measure) VPD less so e. g. SEVI (desert grassland site) VPD Other issues? Turner et al. (2005) 22

MODIS-estimated v Big. Foot FPAR How do you measure FPAR even on the ground? ? Requires models to interpret measurements of radiation Turner et al. (2005) 23

MODIS-estimated v Big. Foot LUE (light use efficiency) LUE inferred from flux data Again, hard to even measure this on the ground…. . Turner et al. (2005) 24

Zhao et al. (2005) Heinsch et al. (2006) 25

Process/SVAT (soil-veg-atm-transport) models Fig from MOD 17 ATBD 26

Process models: how do we test/validate? From Running et al. (2004) MOD 17 ATBD Biome-BGC model predicts the states and fluxes of water, carbon, and nitrogen in the system including vegetation, litter, soil, and the nearsurface atmosphere i. e. daily PSN 27

Process models: how do we test/validate? http: //www. ntsg. umt. edu/models/bgc/ Fig from MOD 17 ATBD 28

Data-Model Fusion [Using multiple streams of datasets with parameter optimization] C stock and flux measurement Inventory analyses Process-based information Climate data Remote sensing information CO 2 column from space Inverse modeling Process-based modeling Retrospective and forward ana 29 2000 Canadell et al.

Multi-level model/data validation • MOD 17 ATBD: Synergy of various carbon measurement programs Fig from MOD 17 ATBD 30

How do we decide on ground-based sampling strategy, scale? · CEOS WGCV LPV: Elementary Sampling Units (ESUs) · “ …. a contiguous spatial region over which the expected value of LAI can be estimated through in situ measurement … corresponds to finest spatial scale of LAI estimates used for reference LAI maps. ” · ESU size: “…. at least as large as one measurement footprint of the in situ instrument and typically includes a number of instrument measurements. ” · ESU size: “ …varies with surface condition, instrument field of view, illumination conditions (when transmission based measurements are used) and spatial sampling design. ” · ESU size: “…. should be sufficient to allow repeat visit with minimum uncertainty due to changes in illumination or geolocation. ” 31

ESUs: CEOS WGCV LAI validation protocol 32

Summary · Calibration · Needed to allow comparison of data & products from multiple sensors & algorithms over time · AND/OR to · Can be done on-board, or via sensor intercomparison etc. · Validation example: NPP · Far removed from EO measurement & spatially, temporally variable · Requires: observation networks over time and space and measurement of met. & biophysical data · Models to interpolate spatially from ground-based, site-scale measurements · Testing and intercomparison of models · Ideally: optimal combinations of models + data across scales (e. g. via data assimilation) 33

References: NPP • • • Running et al. (2004) A Continuous Satellite-Derived Measure of Global Terrestrial Primary Production, Bioscience 54(6), 547 -560. Ganguly et al. (2008 a, b) Generating vegetation leaf area index earth system data record from multiple Sensors, RSE, 112, 4318 -4332 (Part II) and 4333 -4343 (Part I) Turner et al. (2005) Site-level evaluation of satellite-based global terrestrial gross primary production and net primary production monitoring, Glob Change Biol, 11, 666 -684. Zhao et al. (2005) Improvements of the MODIS terrestrial net and gross primary production data sets, RSE, 95, 164 -176. Heinsch et al. (2006) Evaluation of Remote Sensing Based Terrestrial Productivity From MODIS Using Regional Tower Eddy Flux Network Observations, IEEE TGRS, 44(7), 1908 -1925. General validation • • http: //lpvs. gsfc. nasa. gov/LAI_references. html Morisette et al. (2002) A framework for the validation of MODIS Land products, RSE, 83, 77 -96. • Disney et al. (2004) IJRS, 25(23), 5297 -5317 34

Other cal/val links · NPP: http: //daac. ornl. gov/NPP/npp_home. html · Cal/val programs · CEOS-WFGCV (Committee on EO Working Group on Cal/Val) · http: //calvalportal. ceos. org/Cal. Val. Portal/welcome. do · · · http: //lpvs. gsfc. nasa. gov/ http: //landval. gsfc. nasa. gov/ SAFARI 2000: http: //daac. ornl. gov/S 2 K/safari. html VALERI: http: //w 3. avignon. inra. fr/valeri/ NCAVEO: http: //www. ncaveo. ac. uk/ JAXA: http: //www. eorc. jaxa. jp/ALOS/en/calval_index. htm · Etc etc 35

Example: MODIS core val sites http: //landval. gsfc. nasa. gov/coresite_gen. html Justice et al. (1998) http: //eospso. gsfc. nasa. gov/eos_observ/5_6_98/p 55. html Privette et al. (2002) and RSE 83, 1 -2, 1 -359 36