Modelling in Physical Geography Model validation Martin Mergili

Modelling in Physical Geography Model validation Martin Mergili, University of Vienna FT 2016 | Modelling in Physical Geography | Model validation Martin Mergili, University of Vienna

Model validation It is easy with GIS to impress people with nice colourful maps produced with computer models However, we have learned that model parameters are uncertain and also the process understanding is often limited These uncertainties reflect themselves in the model results Model results have to be evaluated against observations to test their validity Limitations Inventory Strategies FT 2016 | Modelling in Physical Geography | Model validation Instructions Martin Mergili, University of Vienna

Landslide inventories In the field of landslide modelling, landslide inventories are most useful for model validation Landslide inventories are freely available for some countries and regions Good landslide inventories are a lot of work to prepare Release and deposition areas should be represented by separate polygons The availability of a landslide inventory is a key requirement for landslide modelling efforts Limitations Inventory Strategies FT 2016 | Modelling in Physical Geography | Model validation Instructions Martin Mergili, University of Vienna

Model development and validation areas If the landslide inventory is used for model development (statistical models) it is always necessary to clearly separate between the model development area and the model validation area In such a case, many model runs with different development and validation areas may be performed in order to achieve robust results This separation is not necessary for physically-based models so that the entire study area may be used for model validation. Limitations Inventory Strategies FT 2016 | Modelling in Physical Geography | Model validation Instructions Martin Mergili, University of Vienna

Simple overlay Observed landslide area TN = True negative prediction TP = True positive prediction FP = False positive prediction FN = False negative prediction True positive rate r. TP = TP / (TP + FN) Modelled landslide area (FOS < 1) TN False positive rate r. FP = FP / (FP + TN) Inventory FN TP FN This only works with binary results Limitations FP FP Strategies FT 2016 | Modelling in Physical Geography | Model validation Instructions Martin Mergili, University of Vienna

ROC Plot 1 Observed landslide area r. TP n) it o 0. 0 O C CR U A AUCROC r. FP 0. 6 1 0. 0 3 0 = 5 0. n (ra 0. 0 p d 0. 3 0. 6 om Modelled landslide probability c di e r True positive rate 0– AUC in range ROC r. TP = TP / (TP + FN) 1 is indicator for False positive rate prediction r. FP = FP / (FP quality + TN) Limitations Inventory Strategies FT 2016 | Modelling in Physical Geography | Model validation Instructions Martin Mergili, University of Vienna

Instructions We will use an ROC Plot to evaluate our computed slope failure probability map against the landslide inventory for the Sant‘Anna test area. This ROC Plot will be created with the statistical software R. The corresponding R script will be provided. This script will be called directly from the Python script so that the validation procedure is directly included in the work flow of our model application. We will run our model application with varying ranges of the input parameters in order to explore how sensitive the prediction quality reacts. Limitations Inventory Strategies FT 2016 | Modelling in Physical Geography | Model validation Instructions Martin Mergili, University of Vienna

Instructions Python 2. 7 documentation: https: //docs. python. org/2. 7/ Arc. Py documentation: https: //desktop. arcgis. com/en/desktop/latest/analyze/arcpy/aquick-tour-of-arcpy. htm R documentation: https: //www. r-project. org/ Limitations Inventory Strategies FT 2016 | Modelling in Physical Geography | Model validation Instructions Martin Mergili, University of Vienna

Have fun! martin. mergili@univie. ac. at http: //www. mergili. at FT 2016 | Modelling in Physical Geography | Model validation Martin Mergili, University of Vienna