Anomaly Detection for Prognostic and Health Management System

Anomaly Detection for Prognostic and Health Management System Development Tom Brotherton TB 6/22/06 1

New Stealth Technology TB 6/22/06 2

Outline • What is Anomaly Detection – Different types of anomaly detectors • Radial Basis Function Neural Net Anomaly Detector – – The basics Comparison with other neural net approaches Feature ‘off-nominal’ distance measures Training • Implementations – Continuous = Gas turbine engine monitoring – Snap shot = Web server helicopter vibration condition indicators • RBF NN & Boxplots • Application to detection of helicopter bearing fault • Application to monitoring fish behavior for water quality monitoring TB 6/22/06 3

What is Anomaly Detection? • Anomaly Detection = The Detection of Any Off-Nominal Event Data – Known fault conditions – Novel event = New - never seen before data • New type of fault • New variation of ‘known’ nominal or fault data • What is ‘Nominal’ – Sets of parameters that behave as expected • Physics models • Statistical models TB 6/22/06 4

Approaches Physics • State Variable Models (derived from physics) Parametric • Hybrid Model: Combine Physics + Empirical - Estimate of physics Ac cu rac y& Co st • Ex: Gas Turbine Engine Deck: Component level physics model Empirical - Derived from collected data • JPL: BEAM (coherence = model of linear relationships) • Neural nets (non-linear relationships) • Fused empirical: BEAM + NN • Academic: Support Vector Applicability • Simple statistics TB 6/22/06 5

Empirical Modeling An anomaly Idea: Idea Theoretical boundary (multidimensional ‘tube’) that data should lie within: - Nominal data is inside the boundary - Anomaly data is outside Problem: How to estimate / approximate the boundary? Collected ‘Nominal’ Data Problem: What measurement(s) caused the anomaly? Problem: How far off-nominal is the anomaly / feature? TB 6/22/06 6

RBF Neural Net Anomaly Detection: The Idea Radial Basis Function (RBF) Neural Net Model NN = Model for Nominal Data • Dynamic data = Lots of NN basis units to model • • • ? ‘Distance’ from Nominal Model = Sample of nominal data = Sample of anomalous data Distance measure = Function of the signal set Individual signal distances from nominal = distance from “closest” basis unit – Yes • • Piecewise stationary approximation Detection can be for set of signals when no single signal is anomalous The model can be adaptively updated to include additional data / known fault classes Trajectories of features relative to basis unit = Prognosis TB 6/22/06 7

Why Use Radial Basis Function Neural Nets? • Radial Basis Function Neural Net – Nearest neighbor classifier – Distance metric : Measure “nominal” – Multi-layer perceptron (MLP) does not have these properties MLP NN RBF NN ? ? TB 6/22/06 8

Support Vector Machine Training data Support Vector Machine Model RBF Model • In some sense, much better model of ‘truth’ …. but - Automated selection of number of basis units • Lots! • Trade off between fidelity vs smoothness • Not practical for on-wing • How to compute individual signal distances • Loss of intuition TB 6/22/06 9

Feature Distance Calculation NN = Model for Nominal Data Mahalanobis Distance s 2 Mahalanobis Distance s 1 ? § Nearest Neighbor Distance TB 6/22/06 10

Alternative Distance Calculation NN = Model for Nominal Data Closest Basis Unit Truth - Truth: Single Feature X = ‘Bad’ -Report: Feature X = ‘OK’ & Feature Y = ‘Bad’ -Alternative Distance = Which Basis Unit gives the smallest number of individual off-nominal features -> Hamming Distance (from digital communications decoding) TB 6/22/06 11

‘RBF’ NN Architectures Input features Weights • • • Detector Output Is output for Nominal? =1 Yes > 1 - Likely < 1 - ? < 1 - No 0< < <1 Basis Units Gaussian elliptical basis function : = Gaussian Mixture Model Rayleigh basis function : Fuzzy membership basis function : Good for magnitude spectral data * Basis function is ‘matched’ to the data distribution For those who like things fuzzy TB 6/22/06 12

Training : Neural Net Architectures – How to select parameters - Small number of clusters Small number of basis units Low False Alarms Very general Missed detections Too General ? - Large number of clusters Good ‘tracking’ of data dynamics Large number of basis units More sensitive to outliers More false alarms Over Trained ? Don’t know a-priori what are the ‘best’ settings TB 6/22/06 13

M of N Detection Idea: M of N detection allows one sample high false alarm rate – Then integrate over time to remove Only 2 points = false alarm False alarms? Large scale factor Small scale factor 4 points persist over time = detection • Trade off single point detection capability vs false alarm rate Large Scale Factor / Small N - Short – high SNR anomalies Small Scale Factor / Large N - Long – persistent – low SNR anomalies Detection? False alarm? TB 6/22/06 15

Alternatives • This technique works well – Demonstrated by Pratt & Whitney for C-17 F 117 applications • Transient engine operations – Long time to train – lots of different types of transients – Model can become very complex • Engine control system • On-wing memory and timing constraints • Alternative – Combine equipment operating regime recognition with anomaly detector – Ex: Identify steady operation and then take a snapshot of the data • Simple statistics may suffice TB 6/22/06 16

Example Gas Turbine Operations Input Signal Vector Scale Signal Break the big problem in to a set of small problems Regime recognition Regime Recognition - Regimes: • • Neural Net Select Neural. Net Detection Transient Throttle up Transient Throttle down Steady state – B 14 open Steady state – B 14 closed Median Filter Trained NNs Off-Nominal Signal Distance Detection Flag TB 6/22/06 17

Anomaly Detection of Stationary Regime Detected Data • Web Server Implementation for Helicopter Vibration Data – Condition Indicators (CIs) = Features derived from on-board vibration measurements • Two types of problems: – Single CI for a component • Simple statistics solution = Boxplot – Intuitive = Army user’s like it • RBF neural net implementation as well – Multi-CIs for a component • RBF neural net implementation TB 6/22/06 18

On Board System FWDLAT FWDVRT FWDSP CPITVRT CPITLAT FWDXMSNVRT FWDXMSNLAT Advanced Rotor Smoothing / Engine Diagnostics HB 2 HB 3 HB 4 HB 5 HB 6 HB 7 Engines ENG 1 COMP ENG 1 NOSE ENG 1 AXIAL ENG 1 LAT ENG 2 COMP ENG 2 NOSE ENG 2 AXIAL ENG 2 LAT Transmissions XSHAFT 1 XSHAFT 2 AFTLAT AFTVRT AFTSP CBOXOCFA CBOXOCLAT APU Intermediate Gearbox Tail Gearbox AFTFANLAT AFTXMSNVRT AFTXMSNLAT Configuration • 36 Vibration Sensors • 2 Speed Sensors • 1553 connection to HUD Main Rotor Cockpit VMU Cockpit Control Head Parameter Data CVR-FDR Main D/S USB Memory Drive Absorbers Hanger Bearings USB Download IAC-1209 • 18 Sensors Installed – Vibration Ethernet Modern Signal • Automated Exceedance Monitoring using HUD data Processing Unit • Automated engine HIT, Max Power Check and exceedances +28 VDC Power (MSPU) • Complete aircraft vibration survey in under 30 seconds Accelerometer Tach Sensor Other Connections TB 6/22/06 19

Aircraft / Server Physical Connectivity SCARNG USB Memory Stick Data Download AIRCRAFT OEMs VMEP PARTNER Browser PC-GBS Remote PC-GBS Facility AARNG INTERNET Wireless link PC-GBS Remote PC-GBS Facility Deployed Unit PC-GBS Remote TB 6/22/06 20

Aircraft / Server Logical Connectivity Facility Portable Systems System -Army P-GBS Aircraft Maintenance -Electronic help desk - Automated data archive - Automated s/w upgrades Support Team - e-mail notification - Fleet level reports - Automated s/w upgrades Web Client - Army F-GBS Browse r MDS Server Help Training Base Electronic Manuals FAQs Network Security Help Desk Diagnostics Prognostics Anomaly Detection Automated Data Archive Fleet Statistics & Reports Data Archive A/C config files TB 6/22/06 21

Advanced Engineering on the Web The role of anomaly detection on the website is to detect and bring to engineering’s attention the MOST INTERESTING data = Something that has NOT been encountered before - More normal data not really of interest TB 6/22/06 22

Single Feature Anomaly Detection Boxplots = Simple statistics - single feature anomaly detector. No Gaussian assumption, just counting points. They seem to work very well! Default based on boxplot statistics User set TB 6/22/06 23

Threshold Setting TB 6/22/06 24

Anomaly Analysis Summary of all aircraft TB 6/22/06 25

The Raw Data TB 6/22/06 26

Gaussian Transformation Data • Problem: How to select a “matched” basis function – Gaussian assumption? Usually violated! • Statistical Model Fit – Transform data to be Gaussian • Transformation stored and is part of the model – Almost always only a single basis unit is required! • Works on single feature data • All processing “behind the scenes” done on transformed data Original Transformed TB 6/22/06 27

RBF Anomaly Detection TB 6/22/06 28

RBF Anomaly Detection TB 6/22/06 29

Case Study: Apache Swashplate Bearing Spectral Server Data • Anomalous data identified with RBF NN AD running on the Server – Aircraft was in Iraq – Automatic email alert sent to users • “Evidence” sent as well – Data reviewed by AED-Aeromechanics and IAC via i. MDS website • Large peak in spectral data at 1250 Hz for tail #460 • Sidebands spaced at intervals corresponding to bearing fault frequencies • Suspected bad swashplate bearing Main SP Spectra Other A/C Tail 460 Other A/C TB 6/22/06 30

Case Study Apache Swashplate Bearing • AED-Aeromechanics acquired raw vibe data Apr 04 and received swashplate May 04 before aircraft was turned-in for D model conversion • Swashplate disassembled by PIF per DMWR Aug 04 • Minor spalling, corrosion and broken cage discovered • Additional algorithms developed from raw data and implemented into VMEP for release Sep 04 Broken Cage Spalling/Corrosion TB 6/22/06 31

Follow Up • Specific algorithms to identify this fault now included with the on-board system • US Army now uses ‘on-condition’ information from the system to perform maintenance – True condition-based maintenance (CBM) TB 6/22/06 32

Other Applications Water Quality Bio-Monitor üIAC 1090 is a mobile, web-enabled automated biomonitoring system that utilizing the ventilatory and body movement patterns of the bluegill fish as a biosensor, much like a canary in a coal mine. üSixteen Bluegills are placed in individual flowthrough Plexiglas chambers. Each chamber is equipped with an individual water input and drainage system. By utilizing sixteen different Bluegills, the IAC 1090 samples more biosensors than any other system on the market resulting in lower false alarm rates. üAll fish generate a micro volt level electric field. Each individual fish is monitored by non-contact electrodes suspended above and below each fish in a Plexiglas chamber. üThe electrical signals generated by the fish’s normal movement is amplified, filtered and passed on via the internet to IAC’s Bio-Monitoring Expert (BME) software system for automated analysis. TB 6/22/06 33

Water Quality Bio-Monitor üBME is a neural network based expert system that provides for rapid, real time assessment of water toxicity based on the ventilatory behavior of fish. BME has shown excellent detection capabilities for toxic compounds with a low false alarm rate. False alarms, common in other similar systems, are typically generated by normal, non-toxic variations in the environment. üAutomated data collection and management tools, user interfaces, and real-time data interpretation employing advanced (artificial intelligence) models of fish ventilatory behavior make BME easy to use. üRemote (Internet) access to IAC 1090 is provided through an easy-to-use graphical user interface. BME’s modular design provides users with the ability to reconfigure the system for different biomonitoring applications and biosensors TB 6/22/06 34

Questions? Conference papers / case studies available at: www. iac-online. com TB 6/22/06 35