A Cloudy View on Computing workshop and CRe

A Cloudy View on Computing workshop and CRe. SIS Field Data Accessibility Jerome Mitchell 1, Jun Wang 1, Geoffrey Fox 1, Linda Hayden 2 Indiana University 1, Elizabeth City State University 2 Workshop Compute Resources • Future. Grid • Virtual machines + virtual networking to create sandboxed modules o Virtual “Grid” appliances: self-contained, pre-packaged execution environments o Group VPNs: simple management of virtual clusters by students and educators Details Who: Association of Computer/Information Sciences and Engineering Departments at Minority Institutions (ADMI) faculty/students Where: Elizabeth City State University (ECSU) When: June 7 - July 5 2011 What: A Teach-One-Teach-Many approach to cloud computing Cloud GIS Distribution Service Google Earth Example 2009 Antarctica Season Overview of 2009 Flight Paths Data Access for Single Frame CRe. SIS Field Data Accessibility Spatia. Lite Database Purpose Current CRe. SIS Data Organization • Introduce ADMI to the basics of the emerging Cloud Computing paradigm • Understand the computer systems constraints, tradeoffs, and techniques of setting up and using cloud • Understand how different algorithms can be implemented and executed on cloud frameworks • Evaluating the performance and identifying bottlenecks when mapping applications to the clouds • CRe. SIS’s data products website lists o direct download links for individual files • The data are organized by season o Seasons are broken into data segments • Data segments are arranged into frames o Associated data for each frame are stored in different file formats Ø CSV (flight path) Ø MAT (depth sounder data) Ø PDFs (image products) • File-based data system has no spatial data access support Schedule Used by Now I appreciate why Cloud Computing is important Parallelized by Map /Reduce Apache’s implementation T i m e End of 3 rd Week CGL’s implementation 2009 Antarctica Season Vector Data Online Data Distribution I i n e Matlab/GIS Hadoop Twister End of 5 th Week Single User GIS Cloud Service Google Earth Visual Crossover Analysis for Quality Control (development project) Field Data Access WMS Data Portal Algorithm • 2009 Antarctic flight path data o ~ 4 million entries - originally stored as 828 separate files and imported into one Spatia. Lite database file • Two main components: Cloud distribution service and special service for Polar. Grid field crew. • Data is supported among multiple spatial databases. Parallel Processing Programming Model Functional Programming Spatia. Lite Database Example Spatial Data Accessibility Project End of 1 st Week Now I understand Cloud Computing o Spatial extension to manages both vector and raster data and supports a rich set of GIS analysis functions through SQL. • The data can be directly accessed through GIS software and MATLAB Flight path data stored as YYYYMMDD_seg. ID_frame. ID. txt SQLite command to create the segs table: Field Data Service Geo. Server Spatial Database Spatia. Lite SQLite Database Virtual Storage Service Spatial Database Virtual Appliance Multiple Users (local network) KML Now I really understand Cloud Computing! References Polar. Grid Data Products: https: //www. cresis. ku. edu/data Spatia. Lite: http: //www. gaia-gis. it/spatialite/ Quantum GIS: http: //www. qgis. org/ CREATE TABLE segs ( UTCTime Number, Thickness Number, Elevation Number, Frame. ID VARCHAR(12), Surface Number, Bottom Number, Quality. Level Integer) SELECT Add. Geometry. Column ('segs', 'geometry', 4326, 'POINT', 2) *note: geometry: 2 -> xy, (longitude, latitude), 4326 -> WGS 84 coordinate system Spatia. Lite: MATLAB Direct Access Mksqlite package: a MEX-DLL to access SQLite databases from MATLAB http: //mksqlite. berlios. de/ Add this flag to build. m to enable SQLite to load Spatia. Lite as an extension: -DSQLITE_ENABLE_LOAD_EXTENSION=1 Testing in MATLAB: dbid = mksqlite(0, 'open', ‘test. sqlite' ) sql = ['SELECT load_extension(''', path_to_spatialite, ''')']; mksqlite(dbid, sql) % load extension mksqlite(dbid, 'SELECT sqlite_version()') mksqlite(dbid, 'SELECT spatialite_version()') mksqlite(dbid, 'SELECT X(geometry) as lon, Y(geometry) as lat from segs where Frame. ID=2009101601001'); mksqlite(dbid, 'close') Center for Remote Sensing of Ice Sheets Headquarters, University of Kansas This material is based upon work supported by the National Science Foundation under Grant No. ANT-0424589. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author (s) and do not necessarily reflect the views of the National