SWAT GRIDIFICATION SHORT SUMMARY ukasz Kokoszkiewicz CONTENTS Enviro

SWAT GRIDIFICATION SHORT SUMMARY Łukasz Kokoszkiewicz

CONTENTS Enviro. GRIDS project overview ü SWAT explanation SWAT gridification assumptions Parallelization results SWAT model parallelization ü LH-OAT uncertainty analysis parallelization ü SWAT-CUP sufi 2 parallelization ü Enviro. GRIDS VO Monitoring Tools

DHMO o o o Project timeframe: April 2009 – March 2013 Total budget of € 8. 1 mio. (€ 6. 2 mio. EC contribution) and 1367 man-months. The Enviro. GRIDS Project Team includes 27 partners from 15 countries, representing also several European (CERN, EEA) and United Nations organizations (UNEP, UNESCO). Among these partners, 22 belong partially or entirely to the Black Sea Catchment. Eight partners belong to an International Cooperation Partner Countries (Ukraine, Georgia and Russian Federation) and five to Associated Countries (Switzerland Turkey). Five partners can be defined as Small and Medium Enterprises (SME).

TECHNICAL OBJECTIVES Ø data gap analysis Ø modeling of large scale, high resolution distributed hydrologic processes Ø developing a grid-enabled Spatial Data Infrastructure (SDI) Ø spatially explicit regional scenarios of development Ø develop access to real time data from sensors and satellites Ø develop early warning and decision support tools at regional, national and local levels Ø Build capacities in the Black Sea region to enhance data interoperability, by promoting on OGC and open standards (ISO 19115/19119/19139; web services WMS, WFS, WCS)

SWAT (Soil and Water Assessment Tool) is a river basin, or watershed, scale model. It predicts the impact of land management practices on water, sediment and agricultural chemical yields in large complex watersheds with varying soils, land use and management conditions over long periods of time.

SWAT Mo de ls

Hydrological modeling

GRIDIFICATION ASSUMPTIONS Creating Enviro. GRIDS VO Easy to use client application called h. Browse framework was created and based on it Diane and Ganga monitoring tools were created Creating easy to use monitoring tools for the community SWAT Gridification Envirigrids VO exists for few months now 3 use cases, for the SWAT gridification, were found Research which use cases are possible to utilize on the GRID

SWAT MODEL GRIDIFICATION Sub-basin based model splitting followed by routing The main idea is to split model itself based on sub-basins. Pre and post processing of the model is required to complete splitting and merging steps. After splitting, each sub-model can be run separately (in parallel) on the grid. Subbasin based parallelization is very interesting for large models.

DANUBE RIVER 15 K HRU'S MODEL GRID RUNS Time [s] As was observed SWAT model sub-basin based grid executions show some promises but to be fully utilized it requires much bigger jobs. Ideal situation would be when single sub-basin execution time takes more than half an hour, in this kind of situations the grid overhead would be negligible. Simulation length [years]

LH-OAT SENSITIVITY ANALYSIS ALGHORITM PARALLELIZATION LH-OAT is an uncertainty analysis method. In uncertainty analysis we are sorting out parameters sensitive to selected real world physical process (eg. nitrogen transport). Then an effective model calibration can be applied. The mechanism behind LH-OAT method parallel run is very similar to SWAT-CUP parallelization. We simply run many SWAT runs with a different parameters. SWAT Model LH-OAT Splitter GRID SWAT Model, params set 1 SWAT Model, params set 2 SWAT Model, params set n swat Output LH-OAT Merger

SWAT-CUP SUFI 2 ALGORITHM PARALLELIZATION GRID New SWAT Inputs Parameters sw. Edit. exe swat MCMC SUFI-2 Para. Sol SWAT Outputs GLUE sw. Extract. exe Output SWAT-CUP algorithms SWAT-CUP is an application for calibration of SWAT models. Calibration of SWAT models requires running SWAT simulations hundreds of times with different parameters set The idea here is to run them separately or in small independent groups and resolve the best parameter configuration that gives simulation effects most corresponding to reality We have “ready to go” example script set, to run swat sufi 2 calibration efficiently from command line

ENVIROGRIDS VO Enviro. GRIDS VO is up and running since Autumn 2010 Resources from Romania were added shortly after the VO were started and are being used since ü ü CE - ce 01. mosigrid. utcluj. ro: 8443/cream-pbs-envir – 960 CPUs SE - se 01. mosigrid. utcluj. ro - 11. 739489 terabytes Registration ü ü ü https: //lcg-voms. cern. ch: 8443/vo/envirogrids. vo. eu-egee. org/vomrs Approval is automatic for a listed members of the EG project People that are not members should be “recommended” by a listed member VO configuration https: //voms. cern. ch: 8443/voms/envirogrids. vo. euegee. org/Configuration. do ü voms 113. cern. ch needs to be replaced by voms. cern. ch inside the VOMSES string ü CIC VO Identity card: https: //cic. egi. eu/index. php? section=vo&vo=envirogrids. vo. eu-egee. org ü

MONITORING h. Browse framework h. Browse. net It’s a common, easy to configure, client side, tool for creating job monitoring systems ü It can visualise any hierarchical data up to 4 levels deep in a neat, table based environment ü Google charts and highcharts can serve as renderer in creating visually attractive charts ü ü Based on h. Browse framewrok many monitoring tools was created including: ü ü http: //dashb-atlas-job. cern. ch/templates/client/index. html http: //dashb-atlas-job. cern. ch/templates/prodsys/index. html http: //gangamon. cern. ch/ganga/ http: //dianemon. cern. ch/diane/

MONITORING HBROWSE SYSTEM ARCHITECTURE The main system component is a Java. Script client which serves as a presentation layer. Different server layers can be applied. Server layer serves only as a data source. Client and server communicates using JSON data transfer format. JSON in this case is a perfect solution because of its syntax: ü in most cases it is the same as in Java. Script and Python CLIENT Java. Script Ajax j. Query JSON HTTP REQUEST SERVER DATABASE Active. MQ messages WEB SERVER

FINAL THOUGHTS SWAT Gridification state The state of the SWAT gridification is that we can run swat tasks on the grid ü With monitoring tools we can easily monitor our simulations/calibrations ü SWAT Gridifications final requirements ü Support of the linux executables (preferably both 32 -bit and 64 -bit) from SWAT developers is critically required

THANK YOU