PathState Modeling for Time Series Anomaly Detection Matt
- Slides: 33
Path-State Modeling for Time Series Anomaly Detection Matt Mahoney
Outline • Review of time series anomaly detection – Gecko – Compression – Path modeling • Piecewise linear approximation of path • Fast testing using state • Experimental results on NASA valve data
Problem: How to Detect Anomalies in Time Series Data • Normal Marotta Fuel Valve Solenoid Current (Used on Space Shuttle) • Abnormal (poppet partially blocked)
Goal • Reduce human workload in specifying “normal” model • Editable rule based model (in SCL) • Real time testing (1 K-10 K samples per second)
Manual Method • Identify features (zero crossings, peaks…) • Specify correct behavior using SCL rules
Gecko (Stan Salvador) • Identify model states (parabolic segments) – Multiple training series are averaged by dynamic time warping • Classify points (x, d 2 x) using RIPPER • Construct linear state machine • Pass/fail test result
Compression Model Normal, uncompressed Abnormal, uncompressed Normal, compressed Abnormal, compressed Normal 1 or 2 Normal 2 Abnormal
TEK Compression Anomaly Scores
Goal Evaluation Manual Gecko Compression Reduce Workload No Yes Real Time Yes Possible Editable model Yes No
Problem with Gecko/RIPPER: State Machine May Underconstrain Model Training Segment 1: x = 0, dx = 0 Segment 2: 0 < x < 1, dx = 1 Test Segment 1: x = 0, dx = 0 Segment 2: 0 < x < 1, dx = 3 dx > 0. 5 State 1 State 2 Accept
Path Model dx Test Path (d 2 = 4) 3 2 Training Path (scaled to unit cube) 1 1 2 3 x
Path Model Example Training Normal Too steep Too low dx d 2 x x Anomaly Score
Example TEK Results Anomaly Score TEK 0 TEK 10 (Training) (Normal) TEK 11 TEK 12
Problems with Path Modeling • Testing is slow, O(n 2) – Compares n test points to n training points each • Model is complex (stores n points)
Proposed Solution • Piecewise linear approximation of path – Editable (k segments, k << n) – Faster testing, O(kn) • State machine model (nearest segment) – Fast testing, O(n) (same as Gecko) – Local minima problem (same as Gecko)
Piecewise Approximation Algorithm • Repeat n – k times – Remove vertex with lowest cost = dh 2 • Run time is O(n log n) using doubly linked heap h d
Test k: compare to all segments Nearest segment: 0 -19 x dx Anomaly Score TEK 0 training TEK 3 near normal TEK 12 stuck poppet TEK 16 late release
Paths (not segmented) TEK 16 TEK 12 x TEK 0 TEK 3 dx d 2 x
TEK 0 approximation with k = 20 segments
Test 2: compare only to current and next segment (fails) TEK 0 training TEK 3 OK TEK 12 local minima TEK 16 local minima
Test 4 segments (previous, current, next 2) succeeds Training OK Skips past minimum Transitions back
Test 4 fails with k = 50 Training OK Not complete Delayed completion
Test 5 (previous, current, next 2, and one random segment) succeeds
Path Fitting (optimal if no sharp bends) • Repeat n – k times – Remove lowest cost vertex (cost = dh 2) – Move adjacent vertices by h/4 toward removed vertex
Vertex Removal vs. Path Fitting • TEK 0 self anomaly scores – Path fitting better for k > 50 – Vertex removal better for k < 50 K 200 100 50 20 Vertex removal Maximum Total 0. 000008 0. 000656 0. 000057 0. 005802 0. 000345 0. 027968 0. 010298 0. 601229 Path fitting Maximum Total 0. 000005 0. 000350 0. 000019 0. 003903 0. 000542 0. 025327 0. 015872 0. 961845
Path Modeling vs. Gecko • Data: Voltage Test 1 at 14 V, 16 V, 18 V. . . to 32 V – 10 x 20 K points – 31 sets of 1 -3 training files • Gecko – Transition threshold = 3 – Error threshold = 10 or 20 – Results: pass at 10 (P), pass at 20 (P/F) or fail • Path Modeling – – Filter delay 2 x 50 samples per dimension k = 50 segments Test 5 (last, current, next 2, and random) Results: maximum and total anomaly score
Typical Results Test file V 37898 V 14 V 37898 V 16 V 37898 V 18 V 37898 V 20 V 37898 V 22 V 37898 V 24 V 37898 V 26 V 37898 V 28 V 37898 V 30 V 37898 V 32 T 21 T 21 T 21 + = Train R 00 s. txt + R 00 s. txt Maximum Total 0. 041018 58. 254755 0. 021778 43. 696323 0. 006596 26. 814669 0. 000913 0. 705107 0. 008819 48. 095410 0. 006635 23. 487464 0. 000361 0. 593473 0. 009032 48. 236476 0. 033475 194. 134671 0. 076193 448. 467580 Gecko P P/F P P
Gecko Summary (Stan) • Gecko – 1 training file: correct behavior • 10 self: 10 P (100% correct) • 90 others: 3 P/F, 87 F (97 -100% correct) – 2 -3 training files: some generalization • 26 self: 23 P, 3 F (14 V, 16 V) (88% correct) – 14 V is too different from the others • 22 “between”: 8 P, 6 P/F, 8 F (36 -63% correct) • 162 others: 1 P/F, 161 F (99 -100% correct)
Path Model Summary • Anomaly score proportional to training-test difference (correct) • Multiple training sets: no generalization (expected)
Run Time Performance • Tested on data set 1 (218 x 20 K points) – 50 training files = 106 samples – 168 test files = 3. 36 x 106 samples • 750 MHz Duron, tsad 4. cpp, g++ -O 2. 95. 2 – Read and filter 106 points: 23 sec – Approximate to k = 100 segments: 30 sec. – Test k: 162 sec (500 ns per point per segment)
Summary Meets all goals Output Training speed Test speed Parameters Local minima Generalization Path Model Yes Numeric O(n log n) O(n) Filter delay, number of segments Yes No Gecko Yes Pass/fail O(n 2) (DTW) O(n) Transition and error thresholds Yes Some
Future Work • Test path modeling with other data sets – UCR archive, http: //www. cs. ucr. edu/~eamonn/TSDMA/ – Power load profiles, http: //www. delelect. com/pdfs/Del-Res. txt • Test with multiple dimensions • Generalization?
Thank You Further Reading http: //cs. fit. edu/~mmahoney/nasa/
- Anomaly detection spark
- Anomaly detection for data quality
- Elasticsearch anomaly detection
- Flink anomaly detection
- System log analysis for anomaly detection
- Helen erickson biography
- Dimensional modeling vs relational modeling
- Robust real-time object detection
- Paul viola
- Robust real-time object detection
- Virtual memory management techniques
- Transaction flow testing
- Application of data flow testing
- Anomaly score
- Anomaly score
- Nfu page replacement algorithm
- Anomaly mcm
- Belady's anomaly example
- Anomaly score
- True anomaly calculator
- May-hegglin anomaly
- Hypobranchial eminence
- Belady's anomaly example
- Cisco anomaly detector
- Mount ararat anomaly
- Vascular ring anomaly
- Ararat anomaly google earth
- Anomaly: instruction "lea" is modifying the stack
- Polyrhinia
- Craneosquisis
- Descriptive meaning in semantics
- Anomaly management systems
- Birman cat neutrophil granulation anomaly