assessing uncertainty in bottomup modelling crop and grasslands

assessing uncertainty in bottom-up modelling crop- and grasslands M. Wattenbacha, Pia Gottschalka, Fred Hattermannb, Claus Rachimowb, Michael Flechsigb, Astley Hastingsa, Pete Smitha University of Aberdeen, School of Medicine and Life Science, Department of Plant and Soil Science, Cruickshank Building, St. Machar Drive, Aberdeen, AB 24 3 UU, UK, m. wattenbach@abdn. ac. uk b Potsdam Institute for Climate Impact Research, Potsdam, 14473, Telegrafenberg, Germany a Carbo. Europe meeting, Crete. 2006

outline • Definitions and background – Uncertainty – Sources of uncertainty • Some case studies • A concept for a framework approach a step towards comparability of model results • Conclusions • Things I learned recently don’t really understand but think might be important

Uncertainty • Uncertainty: the state of being unsure of something • In field science (ISO 1995 - the GUM ): “Uncertainty: parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand” – the term parameter may be, for example, a standard deviation (or a given multiple of it), or the half-width of an interval having a stated level of confidence. – two ways of the evaluation of uncertainty: • (A) is based on a series of measurements and their associated variance • second way which can also be expressed as standard deviation is based on expert knowledge (B) using all available sources

Uncertainty is not Error refers to the imperfection of a measurement due to systematic or random effects in the process of measurement. The random component is caused by variance and can be reduced by an increased number of measurements as we can reduce the systematic component if it occurs from a recognizable process. The uncertainty in the result of a measurement on the other hand arises from the remaining variance in the random component and the uncertainties connected to the correction for systematic effects (ISO 1995).

Definition of model Uncertainty measurement model scenario uncertainty - type D Scenario uncertainty baseline uncertainty - type C scientific judgement uncertainty – type B measured/statistical uncertainty - type A Ecosystem Model conceptual uncertainty - type E

example study One Model on site – DNDC at the Oensingen cropland site (Hastings et al. submitted and Wattenbach et al. poster session) Cross model comparison at one site: – Using five models at the same site to compare the model uncertainty – DNDC, FASSET, PASIM, EPIC, (CENTURY) (M. Wattenbach et al. - in preparation) Cross site comparison one model different sites – PASIM model on different grassland sites in Europe (P. Gottschalk et al. accepted AGEE)

DNDC at Oensingen cropland site Input parameters and variables Uncertainty site scale Fertilization (Nitrogen) +/- 10% each application Temperature +/- 1°C Precipitation +/- 5% Global radiation +/- 5% Clay content +/- 10% Initial soil carbon +/- 10% DNDC site NEE measured kg. C ha-1 best estimate run kg. C ha-1 Mean value of the Monte Carlo simulation kg. C ha-1 Oensingen 2004 5851 6735 6675

Cross site Pa. Sim model at European grassland sites

Cross site: Pa. Sim factor importance Oensingen in 2002 Carlow 2002 Oensingen In 2003 Carlow 2003

Results – cross model comparison 2002 2003

Contribution index normalized change of standard deviation 2002 in %

What we need (ISO ? ) • Standardized methods for uncertainty and sensitivity analysis for ecosystem models • Standardized datasets to allow inter-model comparison of uncertainty and sensitivity measures. • Standardized software interfaces for ecosystem models to allow access to databases for model experiments and results. • Databases for model evaluation results to allow scientists, stake-holders and policy maker’s easy access to information of model quality and uncertainty. • To implement the approach we propose a webbased client - server architecture

Framework design multi-run control input factors & sampling schemes model experiment dataset for comparison ecosystem model client result database evaluation client & visualization postprocessing & visualization model evaluation result database central modelling framwork server

conclusions • Ecosystem models produce very heterogeneous uncertainties • Results are only meaningful if they are accompanied by uncertainty ranges • Standardisation is necessary to reach inter -comparability • The presented framework approach might be a way to achieve this target

Things I learned recently don’t really understand but think might be important • Presentation of John Norton and Ken Reckhow about “Modelling and Monitoring Environmental Outcomes in Adaptive Management (AM)” IEMSS 2006 Principles: • Design management as continuing trial-and-error learning, in which some variation in system state is valuable because it yields information about the system’s behaviour: “learning by doing” • Compare results of alternative policies, through selected indicators, rather than attempting to optimise some cost function • Include resilience to disturbance as an objective

Things I learned recently, don’t really understand but think might be important • Lyapunov stability ( from Wikipedia) • In mathematics, the notion of Lyapunov stability occurs in the study of dynamical systems. – In simple terms, if all points that start out near a point x stay near x forever, then x is Lyapunov stable. More strongly, if all points that start out near x converge to x, then x is asymptotically stable. – The idea of Lyapunov stability can be extended to infinite-dimensional manifolds, where it is known as structural stability, which concerns the behaviour of different but "nearby" solutions to differential equations.

Thank you !

Necessity of consistency • Changing expectations of models – In the beginning models where made to explore the behaviour of systems for scientific reasons only – Today models are more and more used as predictive tools for risk analysis and as policy support systems • Consequences – We need to understand the predictive capacities and restrictions of our models – We need standardized quality checks to give meaningful uncertainty ranges

tools • Monte Carlo method – Monte Carlo methods: algorithms for solving various kinds of computational problems by using random numbers • Advantage: easy to use • Disadvantage: a lot of model runs – Latin Hypercube sampling: stratified sampling method, which can characterise the population equally well as simple random sampling with a smaller sample size • The tool we are using is Simlab (http: //sensitivity-analysis. jrc. cec. eu. int/) a software designed for Monte Carlo based uncertainty and sensitivity analysis – Advantage: easy to use because of the graphical user interface providing a great number of different distributions, sampling methods and parameter interactions – Disadvantage: difficult to integrate external models • Alternative tools and methods to Monte Carlo – University of Sheffield: Gaussian Emulation Machine for Sensitivity Analysis (GEM-SA) see http: //www. shef. ac. uk/st 1 mck/code. html – Advantage: easy to use, fast and efficient and high precision – Disadvantage: problems with thresholds and no time dependants integrated (will be done)

Sources of model uncertainty • Type A and B contributing to Type C - baseline uncertainty: – uncertainty resulting from accuracy and precision of measurements used to determine input factors – Input factors: parameter and variables – Accuracy and precision of the model to represent the processes it is supposed to simulate – Internal parameters: Their accuracy and precision is harder to evaluate as they are often derived parameters (e. g. statistical) based on different measurement methods. In addition they are mostly hard coded.

Sources of model uncertainty • Subset of type B - scenario uncertainty – resulting from the vagueness in scenarios of the future – The input factors are uncertain as they are dependant on unpredictable conditions – The base assumptions of our models may be uncertain because they are based on the current system conditions which may change in future