From Symptom to Cause Localizing Errors in Counterexample
- Slides: 47
From Symptom to Cause: Localizing Errors in Counterexample Traces Mayur Naik, Purdue University Thomas Ball, Microsoft Research Sriram K. Rajamani, Microsoft Research
Model Checking + Fully automatic: Does not require the user to provide annotations. + Transparent: Produces a source-level error trace (counterexample). − An error trace represents a symptom of the error as opposed to its cause. − State-of-the-art model checkers report only one error trace.
The Problem 1. How do we localize the cause of the error in an error trace? 2. How do we produce multiple error traces having distinct causes? Note: Problem is relevant to other error-detection techniques as well.
What is a “Cause”? w. We define a “cause” to be those parts of an error trace not contained in any correct trace. w. The program fragments containing the cause are rendered unreachable. w. The model checker is invoked again to produce additional error traces.
Example main() { Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }
Error #1: Lock acquired in succession main() { Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }
Correct Trace Computation main() { Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }
Error Cause Localization main() { Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }
Error Recovery Insert halt main() { Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; } Unreachable from entry of main in future runs of the model checker
Error #2: Lock held on exit main() { Acquire. Lock(); if (. . . ) Release. Lock(); else { halt; . . . } Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }
Correct Trace Computation main() { Acquire. Lock(); if (. . . ) Release. Lock(); else { halt; . . . } Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }
Error Cause Localization main() { Acquire. Lock(); if (. . . ) Release. Lock(); else { halt; . . . } Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }
Error Recovery Insert halt main() { Acquire. Lock(); if (. . . ) Release. Lock(); else { halt; . . . } Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; } Unreachable from entry of main in future runs of the model checker
Final (Error-Free) Program main() { Acquire. Lock(); if (. . . ) Release. Lock(); else { halt; . . . } return; }
Our Results w A technique that exploits correct traces for error cause localization. w Efficient algorithm for computing correct traces. w Experimental results in the context of the SLAM toolkit.
Transitions and Edges 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock();
Transitions and Edges 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); <(3, L), (5, U)>
Transitions and Edges 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); project(<(3, L), (5, U)>) = (3, 5)
High-Level Algorithm while true do switch Model. Check(G, ve) of // ve is of the form assert(e) case FAILURE(T): let C = Get. Correct. Transitions(G, ve) and K = project(T) project(C) in if K = Ø then break for each (vi, vj) in K do insert a halt statement between vi and vj case SUCCESS: break
Computing Correct Transitions 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); (1, U) (2, L) (3, L) (4, L) (5, U) ve ≡ assert(s==U) (5, L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> Transitions in Error Trace (T) Reachable state-space computed by model checker
Computing Correct Transitions 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); (1, U) (2, L) (3, L) (4, L) (5, U) ve ≡ assert(s==U) (5, L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> Transitions in Error Trace (T) Correct Transitions (C)
Computing Correct Transitions 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); (1, U) (2, L) (3, L) (4, L) (5, U) ve ≡ assert(s==U) (5, L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(3, L), (5, U)> Transitions in Error Trace (T) Correct Transitions (C)
Computing Correct Transitions 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); (1, U) (2, L) (3, L) (4, L) (5, U) ve ≡ assert(s==U) (5, L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(2, L), (3, L)> <(3, L), (5, U)> Transitions in Error Trace (T) Correct Transitions (C)
Computing Correct Transitions 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); (1, U) (2, L) (3, L) (4, L) (5, U) ve ≡ assert(s==U) (5, L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> Transitions in Error Trace (T) Correct Transitions (C)
Example 1: An omission error (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . .
Error Cause Localization (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . . K = project (T) project (C)
Error Cause Localization (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . . K = project (T) project (C)
Error Cause Localization (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . . K = project (T) project (C)
Error Cause Localization (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . . K = project (T) project (C) = { (2, 4), (4, 5) }
Experimental Results Name of driver mouse packet filter LOC 984 serial mouse port 7441 keyboard packet filter 1067 IEEE 1394 bus driver 5818 keyboard class driver 13161 i 8042 port 22168 packet-based DMA 24971 serial port 30905
Experimental Results Name of driver LOC Number of edges in error trace mouse packet filter 984 error cause 73 0 110 4 serial mouse port 7441 56 1 keyboard packet filter 1067 73 0 107 4 45 7 44 7 60 1 81 3 85 0 IEEE 1394 bus driver 5818 keyboard class driver 13161 158 0 i 8042 port 22168 127 1 124 5 packet-based DMA 24971 75 1 serial port 30905 248 3
Experimental Results Name of driver LOC Number of edges in error trace mouse packet filter 984 error cause Error cause localized? 73 0 No 110 4 Yes serial mouse port 7441 56 1 Yes keyboard packet filter 1067 73 0 No 107 4 Yes 45 7 Yes 44 7 Yes 60 1 Yes 81 3 Yes 85 0 No IEEE 1394 bus driver 5818 keyboard class driver 13161 158 0 No i 8042 port 22168 127 1 Yes 124 5 Yes packet-based DMA 24971 75 1 Yes serial port 30905 248 3 Yes
Example 2: A variable-value error (1, (S, S)) (2, (S, S)) (4, (S, S)) (5, (S, F)) (6, (S, F)) Error Trace (6, (S, S)) (6, (F, F)) main() { 1 int status = S; 2 if (*) status = foo(); (3, (S, S)) 3 else { 4 foo(); 5 status = S; } 6 assert(status==x); } Correct Traces Program state is of the form (status, x) enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; }
Example 2: A variable-value error (1, (S, S)) (2, (S, S)) (4, (S, S)) (5, (S, F)) (5, (S, S)) (6, (S, F)) Error Trace (6, (S, S)) (6, (F, F)) main() { 1 int status = S; 2 if (*) status = foo(); (3, (S, S)) 3 else { 4 foo(); 5 status = S; } 6 assert(status==x); } Correct Traces Program state is of the form (status, x) enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; }
Error Cause Localization (1, (S, S)) (2, (S, S)) (4, (S, S)) (5, (S, F)) (5, (S, S)) (6, (S, F)) Error Trace (6, (S, S)) (6, (F, F)) Correct Traces K = project (T) project (C) = Ø main() { 1 int status = S; 2 if (*) status = foo(); (3, (S, S)) 3 else { 4 foo(); 5 status = S; } 6 assert(status==x); } enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; }
High-Level Algorithm while true do switch Model. Check(G, ve) of // ve is of the form assert(e) case FAILURE(T): let C = Get. Correct. Transitions(G, ve) and K = project(T) project(C) in if K = Ø then break for each (vi, vj) in K do insert a halt statement between vi and vj case SUCCESS: break
High-Level Algorithm while true do switch Model. Check(G, ve) of // ve is of the form assert(e) case FAILURE(T): let C = Get. Correct. Transitions(G, ve) and K = project(T C) in if K = Ø then break for each (vi, vj) in K do insert a halt statement between vi and vj case SUCCESS: break
Example 2: A variable-value error (1, (S, S)) (2, (S, S)) (4, (S, S)) (5, (S, F)) (5, (S, S)) (6, (S, F)) Error Trace (6, (S, S)) (6, (F, F)) main() { 1 int status = S; 2 if (*) status = foo(); (3, (S, S)) 3 else { 4 foo(); 5 status = S; } 6 assert(status==x); } Correct Traces Program state is of the form (status, x) enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; }
Error Cause Localization (1, (S, S)) (2, (S, S)) (4, (S, S)) (5, (S, F)) (5, (S, S)) (6, (S, F)) Error Trace (6, (S, S)) (6, (F, F)) Correct Traces K = project (T C) = { (4, 5), (5, 6) } main() { 1 int status = S; 2 if (*) status = foo(); (3, (S, S)) 3 else { 4 foo(); 5 status = S; } 6 assert(status==x); } enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; }
Example 1: An omission error (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . .
Error Cause Localization (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . . K = project (T C)
Error Cause Localization (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . . K = project (T C)
Error Cause Localization (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . . K = project (T C)
Error Cause Localization (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . . K = project (T C) = { (2, 4), (4, 5), (5, 6) }
Limitations w Control-based approach fails to localize the cause when every edge in the error trace is contained in some correct trace. w Transition-based approach localizes the cause to a suffix of the error trace. w Model Imprecision: Infeasible paths can misguide error cause localization using either approach.
Related Work Multiple Counterexamples § Verisim [Bhargavan et al. , TSE ’ 02] Error Cause Localization § § § Explaining counterexamples [Jin et al. , TACAS ’ 02] Explaining type errors [Wand, POPL ’ 86; Johnson & Walz, POPL ’ 86; Beaven & Stansifer, LOPLAS ’ 93; Duggan & Bent, SCP ’ 96; Chitil, ICFP ’ 01; Tip & Dinesh, TOSEM ’ 01] Program Slicing [Weiser, TSE ’ 84] Algorithmic Debugging [Shapiro, Ph. D. thesis ’ 82] Delta Debugging [Zeller, FSE ’ 99] Anomaly Detection § Static: Meta-Level Compilation [Hallem et al. , PLDI ’ 02] § Dynamic: Daikon [Ernst et al. , TSE ’ 01], DIDUCE [Hangal & Lam, ICSE ’ 02]
Conclusions w We have presented a technique for localizing the causes of errors in counterexample traces. w A combination of the control-based and transitionbased approaches appears promising. w Our technique is quite general and should be applicable to error detection tools based on dataflow analysis as well.
- 6 troubleshooting steps
- Counterexample geometry
- Proof by counterexample
- Conjecture examples
- Counterexample calculator
- Direct proof
- Counterexample discrete math
- Conjecture and counterexample examples
- Kontradiksi adalah
- What is counterexample in inductive reasoning
- Pilot testing in performance management
- Halscancer symtom
- Pqrst symptom analysis
- Sekundr
- Types of somatic disorder
- Somatic symptom disorder
- Esasr
- Trauma symptom checklist for children™ screening form
- Cengage
- Obstruktiv hydrocefalus
- Mikael riska
- Somatic disorder
- Esas skala
- Palliative care problem severity score
- Gastroparesis cardinal symptom index calculator
- Factitious disorder
- Chapter 17 somatic symptom disorders
- Pregnant or period
- Skytteglasögon ansi/mil-spec
- Trauma symptom checklist for children™ screening form
- Nursing diagnosis of somatoform disorder
- Trauma symptoms inventory
- Underlying cause and immediate cause
- Greylag goose egg-retrieval behavior
- Ultimate and proximate causes of behaviour
- Imprinting biology example
- Critical errors
- Communication completeness
- Attribution theory psychology
- Swompt
- Food safety errors: case studies
- Errors affecting trial balance with examples
- Propagation
- Difference between formal and informal fallacy
- Indeterminate error
- Alpine flight training
- Experimental errors and uncertainty
- Language production