Tracking Bad Apples Reporting the Origin of Null

Tracking Bad Apples: Reporting the Origin of Null & Undefined Value Errors Michael D. Bond Nicholas Nethercote Stephen W. Kent UT Austin National ICT Australia UT Austin Samuel Z. Guyer Kathryn S. Mc. Kinley Tufts University UT Austin

Example Code User code: float[][] data = {{1. 0 f, 2. 0 f}, {3. 0 f, 4. 0 f}}; Scatter. Plot plot = new Scatter. Plot(data, null); . . . plot. draw(. . . ); 2

Example Code User code: float[][] data = {{1. 0 f, 2. 0 f}, {3. 0 f, 4. 0 f}}; Scatter. Plot plot = new Scatter. Plot(data, null); . . . plot. draw(. . . ); Null. Pointer. Exception at Scatter. Plot. draw(): 315 at Test. do. Stuff(): 124 3

Example Code User code: Library code: float[][] data = {{1. 0 f, 2. 0 f}, {3. 0 f, 4. 0 f}}; Scatter. Plot { draw(. . . ) {. . . Scatter. Plot plot = xaxis. draw(); new Scatter. Plot(data, null); . . . } plot. draw(. . . ); Null. Pointer. Exception at Scatter. Plot. draw(): 315 at Test. do. Stuff(): 124 4

Unusable Value n Null is unusable value q n Scatter. Plot { draw(. . . ) { Use causes error . . . How/why did it become null? q xaxis. draw(); Null’s origin? . . . } } plot. draw(. . . ); Null. Pointer. Exception at Scatter. Plot. draw(): 315 at Test. do. Stuff(): 124 5

Origin of Unusable Value float[][] data = {{1. 0 f, 2. 0 f}, {3. 0 f, 4. 0 f}}; Scatter. Plot { draw(. . . ) {. . . Scatter. Plot plot = xaxis. draw(); new Scatter. Plot(data, null); . . . Origin: Test. init(): 37 } } plot. draw(. . . ); 6

Origin of Unusable Value Scatter. Plot { float[][] data = Scatter. Plot(data, xaxis) { {{1. 0 f, 2. 0 f}, {3. 0 f, 4. 0 f}}; this. data = data; this. xaxis = xaxis; Scatter. Plot plot = } new Scatter. Plot(data, null); . . . Origin: Test. init(): 37 draw(. . . ) {. . . plot. draw(. . . ); xaxis. draw(); . . . } } Origin: Test. init(): 37 7

Origin Tracking Scatter. Plot { float[][] data = Scatter. Plot(data, xaxis) { {{1. 0 f, 2. 0 f}, {3. 0 f, 4. 0 f}}; this. data = data; this. xaxis = xaxis; Scatter. Plot plot = } new Scatter. Plot(data, null); . . . Origin: Test. init(): 37 draw(. . . ) { plot. draw(. . . ); Track every unusable value’s origin for 4% overhead Key: store origin in place of unusable value . . . xaxis. draw(); . . . } } Origin: Test. init(): 37 8

Outline n n n Introduction Unusable values Instances of origin tracking q q n n Redefining program operations Evaluation q q n Null pointer exceptions (Java) Undefined value errors (C/C++) Performance Usefulness Related Work 9

Unusable Values n n Using value causes error Examples: q q n n Null values Undefined values Tough bugs: no info Why is value unusable? q Where did unusable value originate? 10

Unusable Values n n Using value causes error Examples: q q n n Null values Undefined values Opportunity: Store info in place of value Tough bugs: no info Why is value unusable? q Where did unusable value originate? 11

Origin Tracking Implementations n Null pointers (Java) q n Jikes RVM Undefined values (native code) q Valgrind’s Mem. Check 12

Origin Tracking Implementations n Null pointers (Java) q n Jikes RVM Research Archive Undefined values (native code) q Valgrind’s Mem. Check Valgrind Source Code Repository 13

Origin Tracking Implementations n Null pointers (Java) q n Jikes RVM Research Archive Undefined values (native code) q q Valgrind’s Mem. Check Valgrind Source Code Repository Identifies origin for 32 -bit values n n q 47 of 147 are 32 -bit For 32 -bit: 34 of 47 identified Adds negligible overhead (28 X 28 X) 14

Storing Origins in Null Values n Requirements q q q Need bits in null values multiple null values Program operations support null values Recover origin at exception 15

Storing Origins in Null Values n Requirements q q q Need bits in null values multiple null values Program operations support null values Recover origin at exception Null high 5 bits are zero (27 bits available) Reserve & protect address range: 0 x 0000– 0 x 07 ffffff 000002 Method ID 5 bits 14 bits Bytecode index 13 bits 16

Implementing Java Operations Java semantics Standard VM Origin tracking Assignment of null constant obj = null; obj = 0; obj = this_location; Object allocation obj = new Object(); foreach ref slot i obj[i] = 0; foreach ref slot i obj[i] = this_location; Null reference comparison if (obj == null) { if (obj == 0) { if ((obj & 0 xf 8000000) == 0) { if (obj 1 == obj 2) { if (((obj 1 & 0 xf 8000000) == 0) ? ((obj 2 & 0 xf 8000000) == 0) : (obj 1 == obj 2)) { General reference comparison 17

Implementing Java Operations Java semantics Standard VM Origin tracking Assignment of null constant obj = null; obj = 0; obj = this_location; Object allocation obj = new Object(); foreach ref slot i obj[i] = 0; foreach ref slot i obj[i] = this_location; Null reference comparison if (obj == null) { if (obj == 0) { if ((obj & 0 xf 8000000) == 0) { if (obj 1 == obj 2) { if (((obj 1 & 0 xf 8000000) == 0) ? ((obj 2 & 0 xf 8000000) == 0) : (obj 1 == obj 2)) { General reference comparison 18

Implementing Java Operations Java semantics Standard VM Origin tracking Assignment of null constant obj = null; obj = 0; obj = this_location; Object allocation obj = new Object(); foreach ref slot i obj[i] = 0; foreach ref slot i obj[i] = this_location; Null reference comparison if (obj == null) { if (obj == 0) { if ((obj & 0 xf 8000000) == 0) { if (obj 1 == obj 2) { if (((obj 1 & 0 xf 8000000) == 0) ? ((obj 2 & 0 xf 8000000) == 0) : (obj 1 == obj 2)) { General reference comparison 19

Implementing Java Operations Java semantics Standard VM Origin tracking Assignment of null constant obj = null; obj = 0; obj = this_location; Object allocation obj = new Object(); foreach ref slot i obj[i] = 0; foreach ref slot i obj[i] = this_location; Null reference comparison if (obj == null) { if (obj == 0) { if ((obj & 0 xf 8000000) == 0) { if (obj 1 == obj 2) { if (((obj 1 & 0 xf 8000000) == 0) ? ((obj 2 & 0 xf 8000000) == 0) : (obj 1 == obj 2)) { General reference comparison 20

Implementing Java Operations Java semantics Standard VM Origin tracking Assignment of null constant obj = null; obj = 0; obj = this_location; Object allocation obj = new Object(); foreach ref slot i obj[i] = 0; foreach ref slot i obj[i] = this_location; Null reference comparison if (obj == null) { if (obj == 0) { if ((obj & 0 xf 8000000) == 0) { if (obj 1 == obj 2) { if (((obj 1 & 0 xf 8000000) == 0) ? ((obj 2 & 0 xf 8000000) == 0) : (obj 1 == obj 2)) { General reference comparison 21

Implementing Java Operations Java semantics Standard VM Origin tracking Assignment of null constant obj = null; obj = 0; obj = this_location; Object allocation obj = new Object(); foreach ref slot i obj[i] = 0; foreach ref slot i obj[i] = this_location; Null reference comparison if (obj == null) { if (obj == 0) { if ((obj & 0 xf 8000000) == 0) { General reference comparison if (obj 1 == obj 2) { if (((obj 1 & 0 xf 8000000) == 0) ? ((obj 2 & 0 xf 8000000) == 0) : (obj 1 == obj 2)) { General assignment obj 1 = obj 2; 22

Outline n n n Introduction Unusable values Instances of origin tracking q q n n Redefining program operations Evaluation q q n Null pointer exceptions (Java) Undefined value errors (C/C++) Performance Usefulness Related Work 23

Methodology n Adaptive methodology q q n n Mix of application & compilation time Single iteration; 25 trials Da. Capo, SPEC JBB 2000, SPEC JVM 98 3. 6 GHz Pentium 4 w/Linux 24

Performance of Java Implementation 25

Finding and Fixing Bugs n 12 real NPEs from Source. Forge Origin: identified by origin tracking? Triviality: origin obvious by inspection? Usefulness: origin useful for fixing bug? 26

Null Pointer Exceptions Program Lines Exception description Origin? Trivial? Useful? Mckoi SQL DB 94, 681 Access closed connection Yes Nontrivial Definitely useful Free. Marker 64, 442 JUnit test crashes unexpectedly Yes Nontrivial Definitely useful JFree. Chart 223, 869 Plot without x-axis Yes Nontrivial Definitely useful JRefactory 231, 338 Invalid class name Yes Nontrivial Definitely useful Malformed XML document Yes Nontrivial Most likely useful Eclipse 2, 425, 709 Checkstyle 47, 871 Empty default case Yes Nontrivial Most likely useful JODE 44, 937 Exception decompiling class Yes Nontrivial Most likely useful Jython 144, 739 Use built-in class as variable Yes Nontrivial Potentially useful JFree. Chart 223, 869 Stacked XY plot with lines Yes Jython 144, 739 Problem accessing __doc__ Yes JRefactory 231, 338 Package and import on same line Yes Trivial Not useful Close Eclipse while deleting project Yes Trivial Not useful Eclipse 2, 425, 709 Somewhat nontrivial Marginally useful 27

Null Pointer Exceptions Program Lines Exception description Origin? Trivial? Useful? Mckoi SQL DB 94, 681 Access closed connection Yes Nontrivial Definitely useful Free. Marker 64, 442 JUnit test crashes unexpectedly Yes Nontrivial Definitely useful JFree. Chart 223, 869 Plot without x-axis Yes Nontrivial Definitely useful JRefactory 231, 338 Invalid class name Yes Nontrivial Definitely useful Malformed XML document Yes Nontrivial Most likely useful Eclipse 2, 425, 709 Checkstyle 47, 871 Empty default case Yes Nontrivial Most likely useful JODE 44, 937 Exception decompiling class Yes Nontrivial Most likely useful Jython 144, 739 Use built-in class as variable Yes Nontrivial Potentially useful JFree. Chart 223, 869 Stacked XY plot with lines Yes Jython 144, 739 Problem accessing __doc__ Yes JRefactory 231, 338 Package and import on same line Yes Trivial Not useful Close Eclipse while deleting project Yes Trivial Not useful Eclipse 2, 425, 709 Somewhat nontrivial Marginally useful 28

Null Pointer Exceptions Program Lines Exception description Origin? Trivial? Useful? Mckoi SQL DB 94, 681 Access closed connection Yes Nontrivial Definitely useful Free. Marker 64, 442 JUnit test crashes unexpectedly Yes Nontrivial Definitely useful JFree. Chart 223, 869 Plot without x-axis Yes Nontrivial Definitely useful JRefactory 231, 338 Invalid class name Yes Nontrivial Definitely useful Malformed XML document Yes Nontrivial Most likely useful Eclipse 2, 425, 709 Checkstyle 47, 871 Empty default case Yes Nontrivial Most likely useful JODE 44, 937 Exception decompiling class Yes Nontrivial Most likely useful Jython 144, 739 Use built-in class as variable Yes Nontrivial Potentially useful JFree. Chart 223, 869 Stacked XY plot with lines Yes Jython 144, 739 Problem accessing __doc__ Yes JRefactory 231, 338 Package and import on same line Yes Trivial Not useful Close Eclipse while deleting project Yes Trivial Not useful Eclipse 2, 425, 709 Somewhat nontrivial Marginally useful 29

Null Pointer Exceptions Program Lines Exception description Origin? Trivial? Useful? Mckoi SQL DB 94, 681 Access closed connection Yes Nontrivial Definitely useful Free. Marker 64, 442 JUnit test crashes unexpectedly Yes Nontrivial Definitely useful JFree. Chart 223, 869 Plot without x-axis Yes Nontrivial Definitely useful JRefactory 231, 338 Invalid class name Yes Nontrivial Definitely useful Malformed XML document Yes Nontrivial Most likely useful Eclipse 2, 425, 709 Checkstyle 47, 871 Empty default case Yes Nontrivial Most likely useful JODE 44, 937 Exception decompiling class Yes Nontrivial Most likely useful Jython 144, 739 Use built-in class as variable Yes Nontrivial Potentially useful JFree. Chart 223, 869 Stacked XY plot with lines Yes Jython 144, 739 Problem accessing __doc__ Yes JRefactory 231, 338 Package and import on same line Yes Trivial Not useful Close Eclipse while deleting project Yes Trivial Not useful Eclipse 2, 425, 709 Somewhat nontrivial Marginally useful 30

Null Pointer Exceptions Program Lines Exception description Origin? Trivial? Useful? Mckoi SQL DB 94, 681 Access closed connection Yes Nontrivial Definitely useful Free. Marker 64, 442 JUnit test crashes unexpectedly Yes Nontrivial Definitely useful JFree. Chart 223, 869 Plot without x-axis Yes Nontrivial Definitely useful JRefactory 231, 338 Invalid class name Yes Nontrivial Definitely useful Malformed XML document Yes Nontrivial Most likely useful Eclipse 2, 425, 709 Checkstyle 47, 871 Empty default case Yes Nontrivial Most likely useful JODE 44, 937 Exception decompiling class Yes Nontrivial Most likely useful Jython 144, 739 Use built-in class as variable Yes Nontrivial Potentially useful JFree. Chart 223, 869 Stacked XY plot with lines Yes Jython 144, 739 Problem accessing __doc__ Yes JRefactory 231, 338 Package and import on same line Yes Trivial Not useful Close Eclipse while deleting project Yes Trivial Not useful Eclipse 2, 425, 709 Somewhat nontrivial Marginally useful 31

Debugging Timeline age Early gu n a L i des gn sis S c tati ly ana g tin s e T De nt pl e oym Late 32

Debugging Timeline age Early gu n a L i des gn sis c tati S ly ana g tin s e T De nt pl e oym Late Prevent bugs n Memory bugs [Java & C#] n Null pointer exceptions [Chalin & James ‘ 06] q q q n Add “never null” types to Java Programmer effort Exceptions still possible Functional languages 33

Debugging Timeline age Early gu n a L i des gn sis S c tati ly ana g tin s e T De nt pl e oym Late Detect errors in any execution [Find. Bugs] [PMD] [ESC/Java] [JLint] [Metal] n n n Dataflow analysis & pattern matching Program complexity conservative (false positives) Some intentionally unsound (false negatives) 34

Debugging Timeline age Early gu n a L i des gn sis c tati ly ana S g tin s e T De nt pl e oym Late Catch errors in real executions n Assertions n Checking tools [Valgrind] [Purify] n Dynamic slicing [Agrawal & Horgan ’ 90] [Zhang et al. ‘ 07] Powerful but high overhead 35

Debugging Timeline age Early gu n a L i des gn sis c tati ly ana S g tin s e T De nt e oym pl Late Ideal environment for debugging? n Stack/dump reporting n Invariant-based bug detection [Liblit et al. ’ 05] q n Many executions Limited dynamic slicing [Taint. Check] [Chilimbi & Hauswirth ’ 04] [Origin tracking] q Single execution q Narrow focus 36

Summary n Unusable values: tough bugs q q q n n Error stack trace not enough Opportunity: store origin in place of unusable value Managed and native implementations Java null origins: fast, useful, silent Add it to your VM today! 37

Summary n Unusable values: tough bugs q q q n n Error stack trace not enough Opportunity: store origin in place of unusable value Managed and native implementations Java null origins: fast, useful, silent Add it to your VM today! Thank you! 38

Extra slides 39

Runtime Overhead (origin tracking) 40