Evaluating the Precision of Static Reference Analysis Using

Evaluating the Precision of Static Reference Analysis Using Profiling Maikel Pennings, Donglin Liang, Mary Jean Harrold Georgia Institute of Technology Supported by NSF CCR-9988294, CCR-0096321, EIA-0196145, Boeing Aerospace Corp, State of Georgia Yamacraw Misson

Static Reference Analysis Task 1: Identify instances using static names (naming scheme) 1. Use allocation site (level 0) • p and q point to a 2 2. Use allocation site + N top most call sites (level N) • p points to a 2 -5 • q points to a 2 -6 3. Use allocation site + the allocation site for the receiver of the most recent call site 1. A create. A() { 2. return new A(); 3. } 4. … 5. p = create. A(); 6. q = create. A();

Static Reference Analysis Task 2: Determine the points-to set for each reference field or reference variable • Flow-sensitive vs. flowinsensitive • Calling-context-sensitive vs. calling-context-insensitive • Object-sensitive vs. objectinsensitive 1. p = new A(); 2. p. m(); 3. p = new A();

Evaluation Framework JVM events Profiler dynamic information Subject program Static reference analysis reference information

Evaluation Framework JVM Subject program events Profiler Mapping instances to static names Static reference analysis dynamic information reference profile reference information

Comparison of the Information … p. m(); // C 1 for (I=0; I<2; I++) { p = new A(); // C 2 if (I==0) p. m 1(); // C 3 else p. m 2(); // C 4 }

Comparison of the Information C 1 … p. m(); // C 1 for (I=0; I<2; I++) { p = new A(); // C 2 if (I==0) p. m 1(); // C 3 else p. m 2(); // C 4 } C 2 C 3 C 4 i 1 X X i 2 X Reference profile N X X X Reference information N X X X Dynamic information X X Instances are identified using allocation site only

Comparison of the Information C 1 Study 1: Effectiveness Study 2: Precision C 2 C 3 C 4 i 1 X X i 2 X Reference profile N X X X Reference information N X X X Dynamic information X X Instances are identified using allocation site only

Subjects Program Locs #Class #Reached methods #Covered methods Java_cup 10155 35 269 206 (77%) Jess 19317 207 974 757 (78%) Sablecc 44521 295 1598 1376 (86%)

1: Evaluate Naming Schemes Compute a precision reference value PRV[i] for each instance i C 1 Dynamic information Reference profile i 1 i 2 C 2 X X C 3 X N X X C 4 X X PRV 0. 67

Distribution of PRVs for Instances Java_cup Jess Sablecc

Effectiveness at Allocation Sites Compute the average of the PRVs for instances allocated at each allocation site Let I(a) = {instances allocated at allocation site a} Average PRV for allocation site a PRV[i] S i in I(a)

Average PRVs for Allocation Sites Java_ cup Jess Sablecc (0. 0, 0. 2) [0. 2, 0. 4) [0. 4, 0. 6) [0. 6, 0. 8) [0. 8, 1. 0) 1. 0 Level 0 1 2 11 22 21 130 Level 1 1 2 9 24 21 130 Level 2 0 2 9 24 22 130 Level 3 0 2 9 24 22 130 Level 0 7 25 46 54 45 200 Level 1 2 23 38 54 56 204 Level 2 1 17 44 47 63 205 Level 3 0 15 44 49 62 207 Level 0 3 9 18 42 71 318 Level 1 1 5 14 44 74 323 Level 2 0 4 13 46 75 323 Level 3 0 2 13 39 81 326

Case Study Java_ cup (0. 0, 0. 2) [0. 2, 0. 4) [0. 4, 0. 6) [0. 6, 0. 8) [0. 8, 1. 0) 1. 0 Level 0 1 2 11 22 21 130 Level 1 1 2 9 24 21 130 Level 2 0 2 9 24 22 130 Level 3 0 2 9 24 22 130 Java_cup. lalr_item_set Java_cup. terminal_set Java. lang. Vector

Comparison of the Information C 1 Study 1: Effectiveness Study 2: Precision C 2 C 3 C 4 i 1 X X i 2 X Reference profile N X X X Reference information N X X X Dynamic information X X Instances are identified using allocation site only

Andersen’s Algorithm • Overview • Identifies instances using only allocation sites (level-0 naming scheme) • Context-insensitive and flow-insensitive. • Our implementation • Context-sensitive and model-based approach for calls to methods of library classes (e. g. , Vector, Map, Set) • Avoids analyzing library methods (more efficient) • Computes more precise information for programs that use library classes

2: Evaluate Andersen’s Algorithm Compute a precision reference value PRV[A] for each allocation site A Reference profile Reference information C 1 A A X C 2 X X C 3 X X C 4 X X PRV 0. 75

Distribution of PRVs

Case Study Total 74 allocation sites whose precision reference values are in (0, 0. 1) Jess. Context Jess. Funcall. Value Jess. Int. Array. Value Jess. Value. Vector Jess. Variable 3 1 6 1 51 6 6

Conclusion Study 1: Effectiveness of naming schemes • Using allocation sites may be effective for instances allocated at many allocation sites • Using allocation sites + N most recent call sites may increase the effectiveness • The naming schemes may be ineffective for instances used for recursive data structures. Study 2: Precision of Andersen’s algorithm • Andersen’s algorithm can be imprecise for many allocation sites, especially when instances are allocated to construct recursive data structures

Future work • Perform further empirical and case studies • More subjects • More test cases • Develop more effective naming schemes • Use insight to develop better reference analysis techniques

QUESTIONS?