Ocean Observatories Initiative OOI Operations and Maintenance Functions

Ocean Observatories Initiative OOI Operations and Maintenance Functions related to Data Management 4/28/2014

OOI Data Management O&M Data Management ØEach layer imposes O&M activities ØData Management aspects will be discussed today Application Layer Infrastructure Layer OS Layer Hardware Management and Monitoring Tools ØLayers of O&M • Topmost layers are exposed to the users • Lower layers are used/managed by administrators and operators Network 2

Data Management in O&M ØKey responsibilities: • • • Keep the system running- fix problems Perform updates Don’t lose data Deliver data Monitor Performance o o Availability Reliability 3

Problem Management in O&M ØSame tool used by all OOI during O&M ØProblem Reporting • Use existing JIRA tool ØDefect Tracking & Resolution • Lifecycle state o o Triaging Correction Integration Deployment • Type specific attributes 4

Management and Monitoring Process ØToolset used to monitor and manage system health: • • Network performance System response time Storage status Security monitoring ØMaintained by CI Operations Team ØO&M Activities: • Technology Refresh of tools • Provide monitoring, analysis, and troubleshooting 5

Data Storage ØUncabled Instrument and Platform Data are first stored at Operations and Management Centers (OMCs) ØInstrument and Platform Data are ingested into OOINet via Agent Drivers • Cabled- directly- in near real-time • Uncabled- transferred from OMCs ØOOINet stores: • Data in the form received from instrument • Added QC flags • Human in the Loop QC results 6

Backup and Recovery ØScience and Engineering Data Files • Hierarchical flat files on RAID • Planned schedule and procedures for backups • Recovery tools and procedures to replace lost or damaged files ØOOINet metadatabase (Postgre. SQL) • Planned schedule and procedures for backups • Recovery tools and procedures to replace lost or damaged records 7

Data Transfer to Archive Ø Where: National Oceanographic Data Center (NODC) ØWhat: OOI Science Data Products and associated metadata ØHow: OOI-NODC telecons upcoming to determine: • • • Frequency Volume Metadata format Data format Transmission method (ftp, email, etc. ) 8

Algorithms, Calibration Tables, QC Lookup Tables ØAvailable through oceanobservatories. org ØAvailable through OOINet User Interface • Name and version numbers are maintained for algorithms (Data product, Calibration, QC) • Calibration Tables are associated with an instrument device (serial #)/instrument site/deployment • QC Lookup Tables are associated with a version of a QC Algorithm and specific site 9

Documents • Available through oceanobservatories. org • Examples – Cruise Data (e. g. , documents, photographs) – OOI Documents (e. g. , the Technical Data Package) – Calibration Tables 10

Asset Tracking ØProvided in OOINet ØFunctionality provided: • Track location of physical assets (including marine and land assets) • Assign assets to a deployment • Maintain history of asset location • Maintain history of asset repair and calibration 11

Property Management ØOL is the responsible organization ØOL flows down Property Management to the Marine IO institutions using their institutional systems. 12

Back up Slides ØThese slides are not directly related to data management 13

Outline ØGeneric Operations and Maintenance (O&M) ØCyber. Pop O&M • Support Layers • Management and Monitoring processes • Specific Management activities (Data, HW, OS, Network) ØOOINET O&M • Ongoing tasking • User Interface • Technology Refresh 14

Generic O&M ØProblem Management • Will be used by all aspects of OOI during O&M • Problem Reporting o Use existing JIRA tool • Defect Tracking o o Lifecycle state Type specific attributes ØDefect Resolution • Triaging • Correction • Integration 15

Network Management ØNetwork Layer is rarely “seen” by the user ØMaintained by System Administrators ØO&M Activities: • Technology Refresh • Monitoring data volumes 16

Hardware Management ØHardware Layer is rarely “seen” by the user ØMaintained by System Administrators ØO&M Activities: • Replacing failed components • Spares control o o o Plan for “on-site” sparing Monitoring status of hardware Obsolescence Management • Technology Refresh o 5 year cycle 17

Operating System (OS) Management ØOS Layer is rarely “seen” by the user ØMaintained by System Administrators ØO&M Activities: • Technology Refresh • Monitoring status of Operating System 18

Management and Monitoring Process ØToolset used to monitor and manage system health: • Network performance • System response time • Storage status ØMaintained by System Administrators ØO&M Activities: • Technology Refresh of tools • Provide monitoring, analysis, and troubleshooting 19

Application Layer ØComprised of OOINET (including User Interface) ØDirectly used by users ØMaintained by System Administrators 20

Application Layer O&M ØOOINET Core Services • Rolling out new deployments • Defect correction • Technology Refresh ØUser Interface • Eliciting user feedback • Providing Data Access and Retrieval • Technology refresh ØTechnology Refresh • Modernizing Core and UI components • Migrating off of deprecated support software 21

Marine IO O&M ØTypically used by operators and system administration personnel ØCalibration and configuration data saved within SAF • Used by OOINET O&M team to roll out new device drivers and/or fixes to existing drivers ØCruise Data transfer? ØAsset (and property) tracking through the OOINET Asset Tracking tool 22