BIDS RAF 230719 CONCEPTEUR OPRATEUR INTGRATEUR DE SYSTMES

BIDS RAF – 23/07/19 CONCEPTEUR, OPÉRATEUR & INTÉGRATEUR DE SYSTÈMES CRITIQUES www. c-s. fr CS & SPACE /1

OBJECTIVE The Big Data from Space – Reference Architecture Framework BIDSRAF aims at prototyping scalable cloud working environments for development, deployment, and execution of EO Big Data Processing chains. To demonstrate the revelancy and the versatility of the frameworks, the 3 following computing use cases using Sentinel 2 data will be deployed: - MNT generation service using S 2 P stereo pipeline - WPS land-cover map service using IOTA 2 processing chain - Land cover segmentation service using a deep learning model CS & SPACE /2

WORK PROGRESS â State of the following actions – color codes: › › › New action to do In progress Finished CS & SPACE /3

USE CASE 1 : MNT GENERATION SERVICE USING S 2 P STEREO PIPELINE â Key technical functionalities : › Development of Lambda WPS V 1 › § Service Identification & Authentication: OK § Authorization: OK – Geo Data Module – Interface with the Lambda WPS service Data Access Client (Multi SDKs) § Using the EODAG library (Earth Observation Data Access Gateway) – Python SDK (Software Development Kit) for searching, aggregating and downloading remote sensed images using a unique API for any EO data sources – REST API based on Open. Search Geo and OGC Open. Search Extension â Expected results: › Deployment of MNT generation service (S 2 P) over Lambda WPS V 1 â Key new features : › › Kong reverse proxy + OIDC (Open. ID Connect) plugin Keycloak IAM (Identity and Access Management) CS & SPACE /4

CURRENT DEVELOPMENTS âKey technical functionalities : › Development of Lambda WPS V 2: High Availability § Gateway High-Availability: – « Gateway monitoring » service running on master servers (in activepassive mode) ‐ If gateway doesn't respond in time, it must be restarted § Lambda. WPS High-Availability: – To ensure that the availability of the service âUse case 2: › Deep Learning service aiming at producing crop map using innovative architecture approach CS & SPACE /5

REMINDER: EARTHSIGNATURE PROJECT è Based on: › › Copernicus Sentinel satellites: § high spatial resolution & high temporal resolution Hybridization of the network CS & SPACE /6

USE CASE 2 : DEEP LEARNING LAND COVER SEGMENTATION SERVICE â Development & Publication of service See demo of the Use case 1 â Execution of the process › Input Data § Time series of images on Toulouse + elevation data § Pre-trained neural network for crop land classification § Configuration files for NN building and data splitting Input Prediction FC … LSTM . . . LSTM Resnet 2 D January › December Output results § See Next slide CS & SPACE /7

USE CASE 2 : MAIN RESULTS â Automatic segmentation based on 5 classes â Results available on https: //90. 84. 19/geoserver/ › › user : admin passwd : v. LKU 0 oi. Nv. C 3 Hl. RFe CS & SPACE /8

USE CASE 2 : LAND COVER SEGMENTATION SERVICE USING A DEEP LEARNING MODEL âKey technical functionalities : › › Development of Lambda WPS V 3 § Development of Nvidia Docker (to access GPU accelerated cloud platform for deep learning frameworks) – The feature is under validation Deep Learning frameworks integration âExpected results: › Deployment of Deep Learning Land Cover segmentation service over Lambda WPS V 3 âKey new features : › Docker Nvidia (GPU) CS & SPACE /9

DRIVERS â In parallel, the following drivers will be developed: › › › EBRC § Cloud provider from Luxembourg § Waiting for the availability of the Object Storage AWS § The most important cloud provider Google AZURE And others, …. CS & SPACE / 10

MILESTONES â Summary of the main milestones: Milestone Description Associated Payment Key Availability of the 2 prototype frameworks: ‐ USE CASE 1 ‐ USE CASE 2 July 2019 Updated version of : September 2019 Software Package & CIDL September 2019 ‐ ‐ Use Cases Technical Note Requirements Baseline Design Definition File Design Justification File Demonstrations will be explained (given if not too long) during the progress meetings : ‐ Use case 1: end of May ‐ Use case 2: by mid July â Next Key Payment: 31/07/2019 CS & SPACE / 11

RISKS â The main risks likely to be encountered in performing the activity have been summarized below: Main risks Likelihood The heterogeneity of the Iaa. S (Infrastructure as a Service) layers Low Safe. Scale Infra an Ia. C (Infrastructure as Code) multi‐ cloud tool used to create virtual infrastructure on any cloud offering an Iaa. S Layer The lack of technical cohesion between the different Big Data and HPC technologies Low Safe. Scale Perform a Paa. S (Platform as a Service) tool to create on demand hybrid Big Data and HPC computing platforms on any cloud The complexity of building and maintaining a cloud computing infrastructure hinders the development of value‐added earth observation services Low Lambda WPS tool to develop, run, and manage WPS services without the complexity of building and maintaining the processing infrastructure CS & SPACE / 12