Risk and Reliability Towards Targeted Reliability Oceans 2025

Risk and Reliability … Towards Targeted Reliability Oceans 2025 Theme 8 WP 8. 4 Gwyn Griffiths and Mario Brito National Oceanography Centre, Southampton 2008 Moorings Workshop Is there common ground between people working on moorings and those working on improving the reliability of Autonomous Underwater Vehicles?

Risk and Reliability … Nick Millard Fimbul Ice Sheet, Antarctica

Risk and Reliability … Inquiry following loss of Autosub 2 “The Board has recognised the competence and commitment of the NOC AUV team; they have a high level of understanding of the importance of reliability and have employed sound reliability principles to influence their design decisions. However, they have not employed any formal systems reliability analysis methods. The Board believe this to be a major shortfall. ”

Risk and Reliability … Recommendations following loss of Autosub 2 • NERC (or representatives) should define risk acceptance criteria • AUV development team should implement formal risk and reliability management systems • AUV development team should provide evidence of reliability achievement Full report available: Strut, J. (editor), 2006. Report of the inquiry into the loss of Autosub 2 under the Fimbulisen. NOCS Research and Consultancy Report: pdf at http: //eprints. soton. ac. uk/41098/

Risk and Reliability … Synopsis q Introduction to Targeted Reliability in a marine science context q Risk Management Process-AUV being used with Autosub q Informing the process Ø Detailed engineering and operational fault logs Ø Engineering and statistical follow-up Ø The use of Expert Judgement q Moorings Ø What might a Risk Management Process look like? Ø Example issues

Risk and Reliability … Risk Management Process-AUV Start e. g. P(loss)<20%

Risk and Reliability … Campaign requirements, Dr Jenkins (BAS) for Pine Island Glacier, Antarctica Feb. 2009 60 km open water missions to 200, 600, 1000 m depths, close to ice front. Minimum 3 x 60 km sub-ice-shelf missions to 600 m depth, in outer half of cavity. 3 x 120 km sub-ice-shelf missions to 1000 m depth, Desired to the "minimum headroom" limit of the cavity Sea ice may well be present in the area, beneath which Autosub 3 would need to travel to reach the ice front.

Risk and Reliability … How might we predict probability of loss? * Part A - Gather fault history, document human error and all incidents with the AUV Part B - Set out the key features and risks of the operating environment We postulate that combining Parts A and B cannot be done through scientific methods. Addressing Part A alone has been controversial. * Estimated as cause of ~60% of US military UAV faults/incidents by Tvaryanas et al. (2005)

Risk and Reliability … Part A: Example Autosub 3 fault log entries

Risk and Reliability … Autosub Engineering mitigation examples q Replace wet-mateable connectors with penetrators - prior experience of intermittent connection under pressure. Autosub 2 mission 313 Amundsen Sea

Risk and Reliability … Part B. Eliciting Expert Judgement 1. Set out the Issues 2. Select the Experts 3. Clearly Define the Issues 4. Train the Experts 5. Elicit the Judgements 6. Analyze and Aggregate the Results 7. Complete Analysis and Write-up Otway, H. and von Winterfeldt, D. , 1992. Expert judgement in risk analysis and management: Process, context, and pitfalls. Risk Analysis, 12(1): 83– 93.

Risk and Reliability … EEJ example for the fault history of Autosub 3 Ten AUV practitioners from Australia, Canada, USA from academia, research, commercial, military backgrounds. q Given the set of facts on all faults and incidents with Autosub 3 throughout its life to date we seek to predict the probability of loss of the vehicle in four operating environments: > Open water > Coastal > Sea ice present > Under an ice sheet 1. In the course of evaluating each fault log entry, the expert is asked to assess the following question: “What is the probability of loss of the vehicle in the given environment X given fault/incident Y? ”

Risk and Reliability … Your estimates for Autosub 3 Mission 384 Weights 1 -5

Risk and Reliability … Experts’ estimates for Autosub 3 Mission 384 Weights 1 -5

Risk and Reliability … Cumulative frequency statistics Open Water Coastal Sea Ice Shelf Ice Upper Q. . 026 . 037 . 17 . 40 Median . 018 . 020 . 088 . 17 Lower Q. . 0085 . 0083 . 045 . 072

Risk and Reliability … Autosub Statistical (procedural) example Operating procedure for under ice: Each mission has ‘open water’ 25 km segment before committing under ice. Only two missions beyond 250 km, one of which failed, hence large step. Kaplan-Meier method for estimating probability of survival with distance for all Autosub 3 missions to date. Prevalence of faults leading to ‘infant mortality’ using GG’s judgement.

Risk and Reliability … Estimated probability of loss Minimum mission set, no sea ice in front of glacier P(loss) = 9% Minimum mission set + 30 km of sea ice in front of glacier P(loss) = 16% Desired mission set with no sea ice in front of glacier P(loss) = 24% Desired mission set + 30 km of sea ice in front of glacier P(loss) = 30% Based on Autosub 3 history to end of March 2007. Will be updated after Terschelling June 2008 proving trials.

Risk and Reliability … Risk and Reliability: A new service to the NERC marine science community q Part of Oceans 2025 Technology Work Package on Risk and Reliability Ø Mario Brito - (awaiting) Ph. D in software reliability Ø Access to engineering specialists in the Underwater Systems Lab. Ø Early discussions on PAP mooring-related problems have already taken place. For no-fee consultations on risk and reliability issues for NERC marine science, contact either: Mario Brito (m. brito@noc. soton. ac. uk) or Gwyn Griffiths (gxg@noc. soton. ac. uk). http: //www. noc. soton. ac. uk/OED/gxg/risk. html

Risk and Reliability … Example - Flooded Glider, April 2008 Establish root cause of glider partially flooding with ~4 litres of water while on a tethered dockside post incident-free 3 -month deployment. Depth 0 to 7 m 1 min Pressure reduction 9. 5 to 6. 0 in Hg

Risk and Reliability … Systematic Fault Tree Analysis CTD Stern tube Leak Ammonite flooded ‘O’ rings Chance events Failure routes Establish actual cause or assign probabilities. Do NOT jump to conclusions! Pressure port Vacuum plug Pressure tubes separated Li battery vented Tie-rod cross threaded Pressure tubes not fully butted Tapered ring in wrong order

Risk and Reliability … The root cause: Assembly error Courtesy Peter Stevenson For no-fee consultations on risk and reliability issues, contact either: Mario Brito (mpb 2 o 07 @noc. soton. ac. uk) or Gwyn Griffiths (gxg@noc. soton. ac. uk).

Risk and Reliability … Glass buoyancy failure: WHOI VEX Mooring Array in ca. 5000 m Western Atlantic. Where can I find quantitative: • Failure rates? • Any difference in failure rates between brands e. g. Benthos, Vitrovex? • The major causes of failure? • What’s worse and why, time at depth or depth cycling?

Risk and Reliability … Acoustic release failure: WHOI VEX Mooring Array in ca. 5000 m Western Atlantic. Batch of EG&G releases had improperly machined release mechanisms. Below 2000 m, compression was such that the mechanism would never release. Example of a common mode, human error. Recovery using Isis ROV, April 2003.

Risk and Reliability … Work programme for 2008 q Analyze and write up Autosub 3 Expert Judgement Ø Work with Autosub 3 team on 2008 trials and risk management for 2009 Antarctic cruise q How do we incorporate quantitatively sea ice and vessel characteristics? Ø Paper for IPY Conference, St. Petersburg July 2008. q Work with Autosub 6000 team on Markov chain approach to stages of reliability and risk. q The reliability of deep ocean glass spheres. q Factors affecting the reliability of the PAP moorings Ø Related to Euro. SITES project and Oceans 2025 - propose to instrument a test mooring to establish in situ performance. q Discuss way forward for interaction with Rapid-Watch with NERC/Coordinator/Scientists

Risk and Reliability … Conclusions q From open literature searches, AUV community lags the UAV community in analysis of incident and fault data. We should be more proactive, e. g. use of wikis, blogs, list servers … q Recording fault and incident data, and sharing outcomes is important for the community as a whole q Controversy still surrounds attempts to model statistically AUV faults, and more so, and the use of expert judgement to estimate probability of loss from fault history. q We need to do more to engage with ocean engineers working on moorings, landers etc.