From Symptom to Cause Localizing Errors in Counterexample

Model Checking + Fully automatic: Does not require the user to provide annotations. +

The Problem 1. How do we localize the cause of the error in an

What is a “Cause”? w. We define a “cause” to be those parts of

Example main() { Acquire. Lock(); if (. . . ) Release. Lock(); else {.

Error #1: Lock acquired in succession main() { Acquire. Lock(); if (. . .

Correct Trace Computation main() { Acquire. Lock(); if (. . . ) Release. Lock();

Error Cause Localization main() { Acquire. Lock(); if (. . . ) Release. Lock();

Error Recovery Insert halt main() { Acquire. Lock(); if (. . . ) Release.

Error #2: Lock held on exit main() { Acquire. Lock(); if (. . .

Final (Error-Free) Program main() { Acquire. Lock(); if (. . . ) Release. Lock();

Our Results w A technique that exploits correct traces for error cause localization. w

Transitions and Edges 1 2 3 4 5 Acquire. Lock(); if (. . .

High-Level Algorithm while true do switch Model. Check(G, ve) of // ve is of

Computing Correct Transitions 1 2 3 4 5 Acquire. Lock(); if (. . .

Example 1: An omission error (1, U) (2, L) (3, L) 1 2 3

Error Cause Localization (1, U) (2, L) (3, L) 1 2 3 4 (4,

Experimental Results Name of driver mouse packet filter LOC 984 serial mouse port 7441

Experimental Results Name of driver LOC Number of edges in error trace mouse packet

Example 2: A variable-value error (1, (S, S)) (2, (S, S)) (4, (S, S))

Error Cause Localization (1, (S, S)) (2, (S, S)) (4, (S, S)) (5, (S,

Limitations w Control-based approach fails to localize the cause when every edge in the

Related Work Multiple Counterexamples § Verisim [Bhargavan et al. , TSE ’ 02] Error

Conclusions w We have presented a technique for localizing the causes of errors in

Slides: 47

Download presentation

From Symptom to Cause: Localizing Errors in Counterexample Traces Mayur Naik, Purdue University Thomas Ball, Microsoft Research Sriram K. Rajamani, Microsoft Research

Model Checking + Fully automatic: Does not require the user to provide annotations. + Transparent: Produces a source-level error trace (counterexample). − An error trace represents a symptom of the error as opposed to its cause. − State-of-the-art model checkers report only one error trace.

The Problem 1. How do we localize the cause of the error in an error trace? 2. How do we produce multiple error traces having distinct causes? Note: Problem is relevant to other error-detection techniques as well.

What is a “Cause”? w. We define a “cause” to be those parts of an error trace not contained in any correct trace. w. The program fragments containing the cause are rendered unreachable. w. The model checker is invoked again to produce additional error traces.

Example main() { Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }

Error #1: Lock acquired in succession main() { Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }

Correct Trace Computation main() { Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }

Error Cause Localization main() { Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }

Error Recovery Insert halt main() { Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; } Unreachable from entry of main in future runs of the model checker

Error #2: Lock held on exit main() { Acquire. Lock(); if (. . . ) Release. Lock(); else { halt; . . . } Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }

Correct Trace Computation main() { Acquire. Lock(); if (. . . ) Release. Lock(); else { halt; . . . } Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }

Error Cause Localization main() { Acquire. Lock(); if (. . . ) Release. Lock(); else { halt; . . . } Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; }

Error Recovery Insert halt main() { Acquire. Lock(); if (. . . ) Release. Lock(); else { halt; . . . } Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } return; } Unreachable from entry of main in future runs of the model checker

Final (Error-Free) Program main() { Acquire. Lock(); if (. . . ) Release. Lock(); else { halt; . . . } return; }

Our Results w A technique that exploits correct traces for error cause localization. w Efficient algorithm for computing correct traces. w Experimental results in the context of the SLAM toolkit.

Transitions and Edges 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock();

Transitions and Edges 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); <(3, L), (5, U)>

Transitions and Edges 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); project(<(3, L), (5, U)>) = (3, 5)

High-Level Algorithm while true do switch Model. Check(G, ve) of // ve is of the form assert(e) case FAILURE(T): let C = Get. Correct. Transitions(G, ve) and K = project(T) project(C) in if K = Ø then break for each (vi, vj) in K do insert a halt statement between vi and vj case SUCCESS: break

Computing Correct Transitions 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); (1, U) (2, L) (3, L) (4, L) (5, U) ve ≡ assert(s==U) (5, L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> Transitions in Error Trace (T) Reachable state-space computed by model checker

Computing Correct Transitions 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); (1, U) (2, L) (3, L) (4, L) (5, U) ve ≡ assert(s==U) (5, L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(3, L), (5, U)> Transitions in Error Trace (T) Correct Transitions (C)

Computing Correct Transitions 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); (1, U) (2, L) (3, L) (4, L) (5, U) ve ≡ assert(s==U) (5, L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(2, L), (3, L)> <(3, L), (5, U)> Transitions in Error Trace (T) Correct Transitions (C)

Computing Correct Transitions 1 2 3 4 5 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . } Acquire. Lock(); (1, U) (2, L) (3, L) (4, L) (5, U) ve ≡ assert(s==U) (5, L) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> Transitions in Error Trace (T) Correct Transitions (C)

Example 1: An omission error (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . .

Error Cause Localization (1, U) (2, L) (3, L) 1 2 3 4 (4, L) (5, U) (6, L) (6, U) <(1, U), (2, L)> <(2, L), (4, L)> <(4, L), (5, L)> <(5, L), (6, L)> <(1, U), (2, L)> <(2, L), (3, L)> <(3, L), (5, U)> <(5, U), (6, U)> Transitions in Error Trace (T) Correct Transitions (C) 5 6 Acquire. Lock(); if (. . . ) Release. Lock(); else {. . . }. . . Acquire. Lock(); . . . K = project (T) project (C)

Experimental Results Name of driver mouse packet filter LOC 984 serial mouse port 7441 keyboard packet filter 1067 IEEE 1394 bus driver 5818 keyboard class driver 13161 i 8042 port 22168 packet-based DMA 24971 serial port 30905

Experimental Results Name of driver LOC Number of edges in error trace mouse packet filter 984 error cause 73 0 110 4 serial mouse port 7441 56 1 keyboard packet filter 1067 73 0 107 4 45 7 44 7 60 1 81 3 85 0 IEEE 1394 bus driver 5818 keyboard class driver 13161 158 0 i 8042 port 22168 127 1 124 5 packet-based DMA 24971 75 1 serial port 30905 248 3

Experimental Results Name of driver LOC Number of edges in error trace mouse packet filter 984 error cause Error cause localized? 73 0 No 110 4 Yes serial mouse port 7441 56 1 Yes keyboard packet filter 1067 73 0 No 107 4 Yes 45 7 Yes 44 7 Yes 60 1 Yes 81 3 Yes 85 0 No IEEE 1394 bus driver 5818 keyboard class driver 13161 158 0 No i 8042 port 22168 127 1 Yes 124 5 Yes packet-based DMA 24971 75 1 Yes serial port 30905 248 3 Yes

Example 2: A variable-value error (1, (S, S)) (2, (S, S)) (4, (S, S)) (5, (S, F)) (6, (S, F)) Error Trace (6, (S, S)) (6, (F, F)) main() { 1 int status = S; 2 if (*) status = foo(); (3, (S, S)) 3 else { 4 foo(); 5 status = S; } 6 assert(status==x); } Correct Traces Program state is of the form (status, x) enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; }

Example 2: A variable-value error (1, (S, S)) (2, (S, S)) (4, (S, S)) (5, (S, F)) (5, (S, S)) (6, (S, F)) Error Trace (6, (S, S)) (6, (F, F)) main() { 1 int status = S; 2 if (*) status = foo(); (3, (S, S)) 3 else { 4 foo(); 5 status = S; } 6 assert(status==x); } Correct Traces Program state is of the form (status, x) enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; }

Error Cause Localization (1, (S, S)) (2, (S, S)) (4, (S, S)) (5, (S, F)) (5, (S, S)) (6, (S, F)) Error Trace (6, (S, S)) (6, (F, F)) Correct Traces K = project (T) project (C) = Ø main() { 1 int status = S; 2 if (*) status = foo(); (3, (S, S)) 3 else { 4 foo(); 5 status = S; } 6 assert(status==x); } enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; }

High-Level Algorithm while true do switch Model. Check(G, ve) of // ve is of the form assert(e) case FAILURE(T): let C = Get. Correct. Transitions(G, ve) and K = project(T C) in if K = Ø then break for each (vi, vj) in K do insert a halt statement between vi and vj case SUCCESS: break

Error Cause Localization (1, (S, S)) (2, (S, S)) (4, (S, S)) (5, (S, F)) (5, (S, S)) (6, (S, F)) Error Trace (6, (S, S)) (6, (F, F)) Correct Traces K = project (T C) = { (4, 5), (5, 6) } main() { 1 int status = S; 2 if (*) status = foo(); (3, (S, S)) 3 else { 4 foo(); 5 status = S; } 6 assert(status==x); } enum { S, F } x = S; int foo() { if (*) x = S; else x = F; return x; }

Limitations w Control-based approach fails to localize the cause when every edge in the error trace is contained in some correct trace. w Transition-based approach localizes the cause to a suffix of the error trace. w Model Imprecision: Infeasible paths can misguide error cause localization using either approach.

Related Work Multiple Counterexamples § Verisim [Bhargavan et al. , TSE ’ 02] Error Cause Localization § § § Explaining counterexamples [Jin et al. , TACAS ’ 02] Explaining type errors [Wand, POPL ’ 86; Johnson & Walz, POPL ’ 86; Beaven & Stansifer, LOPLAS ’ 93; Duggan & Bent, SCP ’ 96; Chitil, ICFP ’ 01; Tip & Dinesh, TOSEM ’ 01] Program Slicing [Weiser, TSE ’ 84] Algorithmic Debugging [Shapiro, Ph. D. thesis ’ 82] Delta Debugging [Zeller, FSE ’ 99] Anomaly Detection § Static: Meta-Level Compilation [Hallem et al. , PLDI ’ 02] § Dynamic: Daikon [Ernst et al. , TSE ’ 01], DIDUCE [Hangal & Lam, ICSE ’ 02]

Conclusions w We have presented a technique for localizing the causes of errors in counterexample traces. w A combination of the control-based and transitionbased approaches appears promising. w Our technique is quite general and should be applicable to error detection tools based on dataflow analysis as well.