Precise Condition Synthesis for Program Repair Y Xiong

Precise Condition Synthesis for Program Repair Y. Xiong at el. International Conference on Software Engineering 2017 Presentor : Jee-weon Jung

Index • Overview • Introduction • Motivating example • Approach • Evaluation 2

Overview • Object : produce PRECISE patches • Concentrate on condition synthesis • Study three approaches and finally propose ACS • Know what variables should exist in a ‘if’ condition use sorting method based on dependency relations between variables • Propose document analysis technique on API documents filter variables • Mine a set of frequently used predicates • Successfully repaired 18 defects on 4 projects of Defects 4 J • Best performance reported, precision was 78. 3% (others usually 40%) 3

Introduction (1) • Automatic program repair techniques generate patch that match specification • Test suite is mostly used as specification • Patch is considered ‘plausible’ if it passes all tests • Concept of precision is adtopted in plausible patches • Precision is VERY important • Low precision patches decrease efficiency • Precision of well known programs such as Gen. Prog is about 4% • Reason : precise patch is sparse in the space of repair systems 4

Introduction (2) • Rank patches to achieve high precision • Studied widely (Prophet, Historical. Fix, Direct. Fix, …) • This paper proposes more accurate ranking criteria for condition synthesis • Condition synthesis : insert / modify ‘if’ condition • One of most strongest techniques • Proposed method decompose condition synthesis • Variable selection : decide which variable to use • Predicate selection : decide what to do 5

Introduction (3) • Example : ‘if (a > 10)’ • First select variable ‘a’ select predicate ‘>10’ • Use proposed 3 techniques for ranking variables and predicates • Dependency-based ordering : more recent dependency showing variable is more likely to be used in condition • Document analysis : analyze Javadoc comments in the src code • Predicate mining : extract from existing projects sort by frequency 6

Motivating example (1) • Math 99 defect in Defects 4 J dataset • 2 test cases given • ‘a=1, b=50’ / ‘a=Integer. MIN_VALUE, b=1’ • Many plausible conditions exist most are not precise • Proposed ACS addes lines 2~4 • Synthesize the condition at line 2 7

Motivating example (2) • Dependency-based ordering • In the previous example: ‘lcm’ depends on ‘a’ and ‘b’ ‘lcm’ is more likely to be used • Use dependency in ordering • Document Analysis • Mnay java methods come with Javadoc comments assist condition synthesis using comments • Consider ‘@throws’ tag in the document • Analyze the subject of sentence • Consider only the mentioned variables when making guard condition 8

Motivating example (3) • Predicate Mining • Use variable type, name as context • Ex : ‘hour’ frequently used with >24, <12, … • Ex : ‘factorial’ frequently used with <21 (20! Is the largest factorial representable with 64 bit integer 9

Approach (1) - overview • Use 2 types of templates • Directly return the oracle • Identify the last executed statement s in the failed test insert one of if (c) return v / if (c) throw e before the statement • Modification of an existing condition • Locate potential faulty ‘if’ condition c apply either widening (c c || c) / narrowing (c c && c) • c : synthesized condition • Use SBFL & predicate switching for locating potential faulty condition 10

Approach (2) – extracting the oracle • Three types • Constant : directly copy • Specified via XXXException. class annotation : throw an instance • Function mapping the test input to the output : complicated. . First perform backward slicing from the oracle expression perform backward slicing from the test input arguments 11

Approach (3) – variable ranking • 3 steps : preparing candidate filtering by document analysis sort using dependency-based ordering • Preparing candidate variable • Consider 4 types : local variable / method parameter / ‘this’ pointer / expressions used in other ‘if’ conditions in the current method • Sorting by dependency • Create a dependency graph between variables • Node : variable / dege : dependency relation 12

Approach (4) – predicate ranking • Mining related conditions • Use variable type, name, and method name to decide context • Two word is considered similar when word decomposition by capital letter has common word • Variable name is considered meaningful when length >=2 • Synthesize a condition c with variable x in the method m • A conditional expression c 0 is considered to be in a similar context of c, if (1) it contains one variable x 0, (2) x 0 has the same type as x, (3) the name of x 0 is similar to x when the name of x is meaningful, or the name of the method surrounding c 0 is similar to m when the name of x is not meaningful. 13

Approach (5) – counting predicates • Using collected similar expressions, extracted used predicates • Choose among only predefined predicates only • Expanding search space often leads to incorrect patches • Syntactically differenct predicates may semantically be the same • Apply ‘pred’ function to extract multiset sort by frequencies heuristically select top 20 14

Evaluation (1) • Implementation • Based on the source ocde of Nopol • Use fault localization library Gzoltar • Apache Open. NLP is exploited • Dataset • Top five most starred Java projects on Git. Hub (2016/7/15) • Four projects from Defects 4 J 15

Evaluation (2) • Research questions • RQ 1: how do the three ranking techniques perform on ranking variables and predicates? • RQ 2: how does our approach perform on real world defects? • RQ 3: how does our approach compare with existing approaches? • RQ 4: to what extent does each component of our approach contribute to the overall performance? 16

Evaluation (3) • RQ 1: performance of the three techniques 17

Evaluation (4) • RQ 1: performance of the three techniques 18

Evaluation (5) • RQ 1: performance of the three techniques 19

Evaluation (6) • RQ 2: performance of ACS 20

Evaluation (7) • RQ 3: comparison with existing approaches 21

Evaluation (8) • RQ 4: detailed analysis of the components 22

Conclusion • Study refined ranking techniques for condition synthesis • Achieve high precision and reasonable recall on Defects 4 J 23