Setting the Scene Jordi Farres HMAWG Meeting ESRIN

Setting the Scene Jordi Farres HMA-WG Meeting ESRIN, 23 Jan 2013 ESA UNCLASSIED - For Offical Use 23/01/2013 Page 1

Agenda 1. Introduction 2. A model of scientific exploitation – Earth Web 3. A model of the scientific user services 4. Conclusions ESA UNCLASSIED - For Offical Use 23/01/2013 Page 2

Introduction • Increasing data volumes and data footprint for scientific projects • Open data policy and related expectations • High pressure for better dissemination and processing services • Surge of Cloud Computing • Workshop Models for Scientific Exploitation of EO data: Innovation, harmonization, technology – large consensus on the need to foster collaboration and further exploit the concept of “collaborative ground segment – Participants underlined the huge benefits of synergic exploitation of EO data collected from different missions – There is a clear need for testing, harmonising and standardising the user-infrastructures already in place, in order to establish the terms for evolution and further collaboration. ESA UNCLASSIED - For Offical Use 23/01/2013 Page 3

Agenda 1. Introduction 2. A model of scientific exploitation – Earth Web 3. A model of the scientific user services 4. Conclusions ESA UNCLASSIED - For Offical Use 23/01/2013 Page 4

What is the preferred exploitation model? An Earth Box An Earth Web 1. Drop all data on large Dropbox 1. Expose data on an exploitation platform 2. Place all data in the cloud 2. Federate services from different EO providers 3. Users / service autonomously develop: Applications, Services and Research ESA UNCLASSIED - For Offical Use 23/01/2013 3. Users develop the value added services on a common framework Page 5

Intro: An Earth Web • A facility • for EO dissemination and processing services • geographically distributed • in a federated domain • largely scalable • with reliable Qo. S ESA UNCLASSIED - For Offical Use 23/01/2013 Page 6

A case study: G-POD Service ladder ESA ESRIN 6 - Portal – User and Info mgt √ 5 - Service Mgt Deployment √ 4 - Processing Services 3 - Data Mgt Discovery & Catalogue ESA UK-PAC ESAC* >50 3 √ √ Cloud 2 - Data Resources ASAR, RA, GOME, GOMOS, MIPAS, . . . (A)ATSR, MERIS SMOS SAR subset 1 - ICT Resources 2 Clusters 1 Cluster Iaa. S ESA UNCLASSIED - For Offical Use 23/01/2013 Page 7

A case study: G-POD Service ladder ESA ESRIN 6 - Portal – User and Info mgt ESA UK-PAC ESAC* Cloud PORTAL 5 - Service Mgt Deployment 4 - Processing Services 3 - Data Mgt Discovery & Catalogue CATALOGUE DATA COMPUTING 2 - Data Resources 1 - ICT Resources ESA UNCLASSIED - For Offical Use 23/01/2013 Page 8

A case study: G-POD Service ladder ESA ESRIN 6 - Portal – User and Info mgt ESA UK-PAC ESAC* Cloud PORTAL 5 - Service Mgt Deployment 4 - Processing Services PROCESSOR COMPUTING 3 - Data Mgt Discovery & Catalogue 2 - Data Resources 1 - ICT Resources ESA UNCLASSIED - For Offical Use 23/01/2013 Page 9 COMPUTING

An Earth Web Model – Service ladder Partner A Location 1 Location 2 Partner B Partner C 6 - Portal – User and Info mgt √ 5 - Service Mgt Deployment √ 4 - Processing Services 3 - Data Mgt Discovery & Catalogue √ EO √ Resources Provider 2 - Data Resources 1 - ICT Resources ESA UNCLASSIED - For Offical Use √ Cloud Provider 23/01/2013 √ Page 10 √ √ √ EO Community √ Mgt EO Service Provider √ √

An Earth Web Model – Interoperability Service ladder Technologies 6 - Portal – User and Info mgt Standards HMA-AWG SSO SAML Partial (Identity mgt) PF Virtual Machines Web Services IPF I/F OVF WPS-T NO 4 - Processing Services Web Services WPS Partial 3 - Data Mgt Discovery & Catalogue Web Services CSW YES https Metadata File Formats WCS YES OCCI, EC 2 NO 5 - Service Mgt Deployment 2 - Data Resources Web Services 1 - ICT Resources ESA UNCLASSIED - For Offical Use Iaa. S Grid brokers 23/01/2013 Page 11

An Earth Web Model – Challenges 1. Execute processing X on data Y over server Z (portability) 2. Quantify user authorization in relation processing resources 3. Profile users in relation to Qo. S 4. Ability to quote “budget” for processing X on data Y 5. Accounting à la cloud for ICT resources 6. Harmonized authorization and accounting 7. … ESA UNCLASSIED - For Offical Use 23/01/2013 Page 12

Agenda 1. Introduction 2. A model of scientific exploitation – Earth Web 3. A model of the scientific user services 4. Conclusions ESA UNCLASSIED - For Offical Use 23/01/2013 Page 13

Requirements 1/2 1. Capabilities - Support EO research activities 1. Access to large data sources 2. Availability of best processing/visualization tools 3. Ability to scale from local processing to large processing campaigns 4. Control of sharing capabilities for results (data), processing services (WPS), software, documentation 2. Power 1. Scalability to large ICT capabilities; e. g. hundreds of TBs and cores 2. CDN-like capabilities for large dissemination 3. Multi-sourcing ESA UNCLASSIED - For Offical Use 04/12/2012 Page 14

Requirements 2/2 3. Usability 1. Simple (seamless? ) deployment from WS to EP 4 x 2. Built-in collaboration capabilities 3. No entry-barriers - very simple start-up 4. Transparent accounting; e. g. single account for Iaa. S, data, sw licenses; pay-per-use model 4. Support Business Model 1. For Data Providers 2. For Software Vendors 3. For Cloud providers ESA UNCLASSIED - For Offical Use 04/12/2012 Page 15

Exploitation Model: Actors • Data Provider: ESA and other EO data providers serving commercial or public data over common ordering protocols. • PI: End-user of the platform. It can be a paying or a sponsored user. • Software Vendor Software vendor providing software download and licensing on cloud resources. License could be Open Source or not. Free or not. Software could be for usage on Iaa. S or also available also on Paa. S/Saa. S. • Cloud Provider for public Iaa. S, Paa. S and Saa. S services. ESA UNCLASSIED - For Offical Use 04/12/2012 Page 16

ESA UNCLASSIED - For Offical Use 04/12/2012 Page 17

ESA UNCLASSIED - For Offical Use 04/12/2012 Page 18

ESA UNCLASSIED - For Offical Use 04/12/2012 Page 19

ESA UNCLASSIED - For Offical Use 04/12/2012 Page 20

ESA UNCLASSIED - For Offical Use 04/12/2012 Page 21

ESA UNCLASSIED - For Offical Use 04/12/2012 Page 22

EO Data Exploitation ESA UNCLASSIED - For Offical Use 04/12/2012 Page 23

New EO Service development ESA UNCLASSIED - For Offical Use 04/12/2012 Page 24

New EO Product development ESA UNCLASSIED - For Offical Use 04/12/2012 Page 25

Agenda 1. Introduction 2. A model of scientific exploitation – Earth Web 3. A model of the scientific user services 4. Conclusions ESA UNCLASSIED - For Offical Use 23/01/2013 Page 26

Conclusions To Be Written ESA UNCLASSIED - For Offical Use 23/01/2013 Page 27