University of Saarland Department of Computer Science Impact

  • Slides: 92
Download presentation
University of Saarland Department of Computer Science Impact Analysis Yury Chebiryak urriy@wjpserver. cs. uni-sb.

University of Saarland Department of Computer Science Impact Analysis Yury Chebiryak urriy@wjpserver. cs. uni-sb. de /96

Impact Analysis n n is often used to assess the effects of a change

Impact Analysis n n is often used to assess the effects of a change after change has been made is rarely used to predict the effect of change before it is instantiated 2

Techniques 3

Techniques 3

Dependency-based IA 4

Dependency-based IA 4

Whole Program Path-Based Dynamic Impact Analysis James Law Gregg Rothermel /96

Whole Program Path-Based Dynamic Impact Analysis James Law Gregg Rothermel /96

Path. Impact n n n Dynamic Call-based Works on binaries Relatively low cost Relatively

Path. Impact n n n Dynamic Call-based Works on binaries Relatively low cost Relatively low overhead 6

Call Graph 7

Call Graph 7

Call Graph B is changed 8

Call Graph B is changed 8

Call Graph Assumption: change in B has a potential impact on nodes reachable from

Call Graph Assumption: change in B has a potential impact on nodes reachable from B 9

Execution Traces (ET) Run M() ET: M 10

Execution Traces (ET) Run M() ET: M 10

Execution Traces Run B() ET: MB 11

Execution Traces Run B() ET: MB 11

Execution Traces Return ET: MBr 12

Execution Traces Return ET: MBr 12

Execution Traces Run A() ET: MBr. A 13

Execution Traces Run A() ET: MBr. A 13

Execution Traces Run C() ET: MBr. AC 14

Execution Traces Run C() ET: MBr. AC 14

Execution Traces Run D() ET: MBr. ACD 15

Execution Traces Run D() ET: MBr. ACD 15

Execution Traces Return ET: MBr. ACDr 16

Execution Traces Return ET: MBr. ACDr 16

Execution Traces Run E() ET: MBr. ACDr. E 17

Execution Traces Run E() ET: MBr. ACDr. E 17

Execution Traces Return ET: MBr. ACDr. Er 18

Execution Traces Return ET: MBr. ACDr. Er 18

Execution Traces Return ET: MBr. ACDr. Err 19

Execution Traces Return ET: MBr. ACDr. Err 19

Execution Traces Return ET: MBr. ACDr. Errr 20

Execution Traces Return ET: MBr. ACDr. Errr 20

Execution Traces Return ET: MBr. ACDr. Errrrx 21

Execution Traces Return ET: MBr. ACDr. Errrrx 21

Execution Traces II n Single MBr. ACDr. Errrrx n Multiple MBr. ACDr. Errrrx. MBGrrrrx.

Execution Traces II n Single MBr. ACDr. Errrrx n Multiple MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx Programs with loops => very long trace 22

SEQUITUR algorithm Data compression algorithm by Larus: n Online n Created grammar reproduces trace

SEQUITUR algorithm Data compression algorithm by Larus: n Online n Created grammar reproduces trace exactly n O(N) running time n O(N) size in the worst case n O(log N) size in the best case n e. g. 2 GB trace → 100 MB 23

SEQUITUR algorithm SEQUITUR( S ) input Execution Trace S output Grammar G Rule T

SEQUITUR algorithm SEQUITUR( S ) input Execution Trace S output Grammar G Rule T 1. for each token in S 2. append token to end of production for T 3. if duplicate digram appears 4. if other occurrence is a rule g in G 5. replace new digram with nonterminal of g. 6. else 7. form a new rule and replace duplicate 8. digrams with the new non-terminal. 9. if any rule in G is used only once 10. remove the rule by substituting the production. 11. return G 24

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx 25

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx 25

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → MBr. ACDr. Errrrx 26

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → MBr. ACDr. Errrrx 26

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → MBr. ACDr. E 1

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → MBr. ACDr. E 1 1 x 1 → rr 27

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → MBr. ACDr. E 11

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → MBr. ACDr. E 11 x 1 → rr 28

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → MBr. ACDr. E 11

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → MBr. ACDr. E 11 x. M 1 → rr 29

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → MB r. ACDr. E

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → MB r. ACDr. E 11 x MB 1 → rr 30

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → 2 r. ACDr. E

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → 2 r. ACDr. E 11 x 2 1 → rr 2 → MB 31

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → 2 r. ACDr. E

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T → 2 r. ACDr. E 11 x 2 1 → rr 2 → MB 32

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T 1 2 3 4 →

SEQUITUR: example ET: MBr. ACDr. Errrrx. MBGrrrrx. MBCFrrrrx T 1 2 3 4 → → → 2 r. ACDr. E 113 G 43 CF 4 rx rr MB x 2 1 r grammar => whole path DAG 33

Whole path DAG 34

Whole path DAG 34

Path. Impact 35

Path. Impact 35

Path. Impact 36

Path. Impact 36

Path. Impact(E): I {E} Returns=0 I = {} 37

Path. Impact(E): I {E} Returns=0 I = {} 37

Path. Impact(E): up(T, E) Returns=0 I = {E} 38

Path. Impact(E): up(T, E) Returns=0 I = {E} 38

Path. Impact(E): forward(1) Returns=0 I = {E} 39

Path. Impact(E): forward(1) Returns=0 I = {E} 39

Path. Impact(E): forward(r) Returns=0 I = {E} 40

Path. Impact(E): forward(r) Returns=0 I = {E} 40

Path. Impact(E): Returns++ Returns=0 I = {E} 41

Path. Impact(E): Returns++ Returns=0 I = {E} 41

Path. Impact(E): forward(r) Returns=1 I = {E} 42

Path. Impact(E): forward(r) Returns=1 I = {E} 42

Path. Impact(E): Returns++ Returns=1 I = {E} 43

Path. Impact(E): Returns++ Returns=1 I = {E} 43

Path. Impact(E): after forward(1) Returns=4 I = {E} 44

Path. Impact(E): after forward(1) Returns=4 I = {E} 44

Path. Impact(E): forward(3) -> x Returns=4 I = {E} 45

Path. Impact(E): forward(3) -> x Returns=4 I = {E} 45

Path. Impact(E): backward(r) Returns=4 Skip=0 I = {E} 46

Path. Impact(E): backward(r) Returns=4 Skip=0 I = {E} 46

Path. Impact(E): Skip++ Returns=4 Skip=0 I = {E} 47

Path. Impact(E): Skip++ Returns=4 Skip=0 I = {E} 47

Path. Impact(E): backward(D)->Skip-- Returns=4 Skip=1 I = {E} 48

Path. Impact(E): backward(D)->Skip-- Returns=4 Skip=1 I = {E} 48

Path. Impact(E): backward(C)-> I : = I U {C}, Returns-- Returns=4 Skip=0 I =

Path. Impact(E): backward(C)-> I : = I U {C}, Returns-- Returns=4 Skip=0 I = {E} 49

Path. Impact(E): backward(A)-> I : = I U {A}, Returns-- Returns=3 Skip=0 I =

Path. Impact(E): backward(A)-> I : = I U {A}, Returns-- Returns=3 Skip=0 I = {E, C} 50

Path. Impact(E): backward(r)->Skip++ Returns=2 Skip=0 I = {E, C, A} 51

Path. Impact(E): backward(r)->Skip++ Returns=2 Skip=0 I = {E, C, A} 51

Path. Impact(E): after backward(2) Returns=1 Skip=0 I = {E, C, A, M} 52

Path. Impact(E): after backward(2) Returns=1 Skip=0 I = {E, C, A, M} 52

Chianti: A Tool for Change Impact Analysis of Java Programs Ophelia Chesley Xiaoxia Ren

Chianti: A Tool for Change Impact Analysis of Java Programs Ophelia Chesley Xiaoxia Ren Barbara G. Ryder Fenil Shah Frank Tip /96

Chianti: high-level concept Inputs: 1) test suite T 2) original version of code 3)

Chianti: high-level concept Inputs: 1) test suite T 2) original version of code 3) edited version of code 54

Chianti: high-level concept 1) set of interdependent atomic changes A 55

Chianti: high-level concept 1) set of interdependent atomic changes A 55

Chianti: high-level concept 1) 2) set of interdependent atomic changes A call graph for

Chianti: high-level concept 1) 2) set of interdependent atomic changes A call graph for each test in T 56

Chianti: high-level concept 1) 2) 3) set of interdependent atomic changes A call graph

Chianti: high-level concept 1) 2) 3) set of interdependent atomic changes A call graph for each test in T T´ T – potentially affected by c A 57

Chianti: high-level concept 1) 2) 3) 4) set of interdependent atomic changes A call

Chianti: high-level concept 1) 2) 3) 4) set of interdependent atomic changes A call graph for each test in T T´ T – potentially affected by c A for ti T´ determine A´ A that may have affected ti 58

Chianti: high-level concept 1) 2) 3) 4) set of interdependent atomic changes A call

Chianti: high-level concept 1) 2) 3) 4) set of interdependent atomic changes A call graph for each test in T T´ T – potentially affected by c A for ti T´ determine A´ A that may have affected ti determine subset of methods affected by a set of changes isolate subset of changes that affect a given test 59

Chianti: architecture 60

Chianti: architecture 60

Chianti: architecture 61

Chianti: architecture 61

Chianti: architecture 62

Chianti: architecture 62

Example: original version class A{ public A() {} public void foo (){} } class

Example: original version class A{ public A() {} public void foo (){} } class B extends A{ public B(){} public void foo(){} } class C extends A{ public C(){} } 63

Example: changed version 64

Example: changed version 64

Example: tests class Tests { public static void test 1(){ A a = new

Example: tests class Tests { public static void test 1(){ A a = new A(); a. foo(); } public static void test 2(){ A a = new B(); a. foo(); } public static void test 3(){ A a = new C(); a. foo(); } } 65

Review: Atomic Changes Change AC DC AM DM CM LC AF DF Description Add

Review: Atomic Changes Change AC DC AM DM CM LC AF DF Description Add an empty class Delete an empty class Add an empty method Delete an empty method Change body of a method Change virtual method lookup Add a field Delete a field 66

Review: Atomic Changes II Change CFI CSFI AI DI CI ASI DSI CSI Description

Review: Atomic Changes II Change CFI CSFI AI DI CI ASI DSI CSI Description Change definition of an instance field initializer Change definition of a static field initializer Add an empty instance initializer Delete an empty instance initializer Change definition of an instance initializer Add an empty static initializer Delete an empty static initializer Change definition of a static initializer 67

Example: Step 1 68

Example: Step 1 68

Example: Step 1 69

Example: Step 1 69

Example: Step 1 70

Example: Step 1 70

Example: Step 1 71

Example: Step 1 71

Example: atomic changes 72

Example: atomic changes 72

Recall: high-level concept 1) 2) 3) 4) set of interdependent atomic changes A call

Recall: high-level concept 1) 2) 3) 4) set of interdependent atomic changes A call graph for each test in T T´ T – potentially affected by c A for ti T´ determine A´ A that may have affected ti 73

Example: affected tests A test is affected if its call graph contains: 1) node

Example: affected tests A test is affected if its call graph contains: 1) node that corresponds to CM or DM, or 2) edge that corresponds to LC 74

Example: affected tests 75

Example: affected tests 75

Example: test 1 void test 1(){ A a = new A(); a. foo(); }

Example: test 1 void test 1(){ A a = new A(); a. foo(); } 76

Example: test 2 void test 2(){ A a = new B(); a. foo(); }

Example: test 2 void test 2(){ A a = new B(); a. foo(); } 77

Example: test 3 void test 3(){ A a = new C(); a. foo(); }

Example: test 3 void test 3(){ A a = new C(); a. foo(); } 78

Recall: high-level concept 1) 2) 3) 4) set of interdependent atomic changes A call

Recall: high-level concept 1) 2) 3) 4) set of interdependent atomic changes A call graph for each test in T T´ T – potentially affected by c A for ti T´ determine A´ A that may have affected ti 79

Example: affecting changes The set of atomic changes that affect a given affected test

Example: affecting changes The set of atomic changes that affect a given affected test includes: 1) all atomic changes for nodes with AM and CM, 2) atomic changes in LC category, and 3) their transitively prerequisite atomic changes 80

Example: affecting changes 81

Example: affecting changes 81

Example: Affecting. Changes(test 2) 82

Example: Affecting. Changes(test 2) 82

Example: Affecting. Changes(test 2) 83

Example: Affecting. Changes(test 2) 83

Example Remember the atomic change: This atomic change does not affect any of the

Example Remember the atomic change: This atomic change does not affect any of the tests => additional tests are needed. 84

Snapshot 85

Snapshot 85

Chianti: Evaluation Number of atomic changes between each pair of Daikon versions. 86

Chianti: Evaluation Number of atomic changes between each pair of Daikon versions. 86

Chianti: Evaluation Categorization of the atomic changes. 87

Chianti: Evaluation Categorization of the atomic changes. 87

Chianti: Evaluation Percentage of affected tests for each of the Daikon versions. 88

Chianti: Evaluation Percentage of affected tests for each of the Daikon versions. 88

Chianti: Evaluation Average percentage of affecting changes, per affected test. 89

Chianti: Evaluation Average percentage of affecting changes, per affected test. 89

Summary Path. Impact n dynamic – uses dynamic traces n online n works on

Summary Path. Impact n dynamic – uses dynamic traces n online n works on binaries Chianti n uses dynamic call graphs n applicable for OOP (e. g. Java) n determines affecting changes for every test 90

References [1] Chesley, O. , Dolby, J. , Ren, X. , Ryder, B. ,

References [1] Chesley, O. , Dolby, J. , Ren, X. , Ryder, B. , Shah, F. , and Tip, F. Chianti: A prototype change impact analysis tool for Java. Tech. Rep. DCS-TR 533, Rutgers University Department of Computer Science, September 2003. [2] Chesley, O. , Ren, X. , Ryder, B. , Shah, F. , and Tip, F. Chianti: a tool for change impact analysis of java programs. In Proceedings of the 19 th annual ACM SIGPLAN Conference on Object-oriented programming, systems, languages, and applications, pp. 432 448. [3] Law, J. , Rothermel, G. Whole program Path-Based dynamic impact analysis. In Proc. of the 25 th International Conference on Software Engineering, May 03 -10, 2003, Portland, Oregon. [4] Ryder, B. , Tip, F. Change impact analysis for object-oriented programs. In Proceedings of the 2001 ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering, pp. 46 53, June 2001, Snowbird, Utah, United States. 91

Thanks for listening. Questions? 92

Thanks for listening. Questions? 92