How Slim Will My System Be Estimating Refactored

How Slim Will My System Be? Estimating Refactored Code Size by Merging Clones Norihiro Yoshida 1, Takuya Ishizu 2, Bufurod EdwardsⅢ 2, Katsuro Inoue 2 1 Nagoya University 2 Osaka University ICPC 2018

Background Software Maintenance Company (Industry Collaborators) Other Companies l. Activities l. System operation l. Bug fixes l. Refactoring Source Code They try to evaluate the value of current system from the view point of maintenance effort and cost.

Slimming Problem Q. How Slim will it be after merging reducible code clones? l. A lot of code clone detection techniques have been developed. ? Current System Refactored System 3

Reduction Rate |S| Code Clone Call Code Clone |S’| Refactoring Call Common Function System S’ l 4

Slimming Methods Three Different Slimming Methods Method Complete Reduction Rate Higher Bound Applied Clones Heuristic Modest Selecting Merge Candidates Basic Lower Bound Only Function Clones All( Overlapping and Embedding ) l. Based on the various approaches, the reduction rate change largely. 5

Evaluation-OSS l. RQ 1: What are the reduction rates of Basic, Complete, and Heuristic Methods for popular OSS systems in written C/C++? 12. 0% 10. 2% 10. 0% 7. 3% 8. 0% 5. 7% 6. 0% 4. 5% 3. 1% 4. 0% 2. 0% 0. 0% 1. 2% 0. 8%0. 4% 1. 6% 0. 0% git 2. 0% 0. 8%0. 7% 1. 0% Clone rate Basic Complete Heuristic 0. 0% libcurl Skynet Linux Kernel Answer to RQ 1 l. Complete Method shows the highest reduction, and Basic Method shows the lowest. l. Heuristic Method is between them.

Evaluation-Commercial System l. RQ 2: Is the reduction rate is stable in a different environment? l. Target system l. A huge and actual commercial system written in COBOL l. An account managing system for a chain store Total Clone(% of Total) Complete(%) 1, 259, 184 Heuristic(%) 853, 271(68%) 194, 358(15. 4%) 102, 113(8. 1%) l. There a lot of overlapping fragments in cloned codes. l. Complete: Increasing the overhead for the chunked procedures. l. Heuristic: Reducing the merging opportunities. Answer to RQ 2 l: No, the reduction is quite different from the target’s domain.

Conclusion l. We have presented the slimming problem for estimation of refactored system size. l. Three methods for slimming. l. Application to OSS systems and a commercial system. l. A high code clone rate does not always guarantee high reduction rate. l. Future Work l. Further investigation are needed to clarify the high clone rate and their overlapping situation.

Feedback • This approach gives industry collaborators a solid theoretical basis for the cost estimation. • The reduction size does not perform actual refactoring. • A tool performing actual refactoring for the clones which are safely refactored automatically.

Research Question l. Reduction rates will give the maintainer very good indicator future maintenance overhead. l. RQ 1: What are the reduction rates of Basic, Complete, and Heuristic Methods for popular OSS systems in written C/C++? l. RQ 2: Is the reduction rate is stable in a different environment?

Aim l. Slimming methods do not guarantee the actual existence of S’. l. Due to the programming language constraints or readability of system. l. Reduction rates will give the maintainer very good indicator future maintenance overhead. l(Reduction rate is low) We would need to continue current maintenance efforts. l(Reduction rate is high) There will be a chance to reduce the system size and its maintenance cost. 11

Execution time System Clone Detection(s) Basic(s) Heuristic(s) Complete(s) git 87. 1 5. 7 0. 7 4. 5 libcurl 52. 8 5. 6 0. 6 3. 9 Skynet 18. 3 1. 1 0. 2 0. 9 1. 5 hour 484. 1 103. 0 407. 5 Linux

Basic-example |S| Function Clone A Call Statement to A Block Clone B Function Clone A Block Clone B System S Refactor only Function Clones Call Statement to A |S’| Block Clone B Shared Function A Slimmed System S’ 13

Complete-Straightforward Strategy Clone A Overlapping fragment Call B Clone A Clone B Call A Overlapping Call Afragment is Create shared functions executed twice by both A for each clone set, and place call statements for Call. BB each clone instance System S Overlapping fragments as clones Shared Function A Shared Function B System S’ 14

Complete-Overview(1)(2) Clone A Clone B System S Overlapping fragment (1)Clone Detection • We use CCFinder. X as before, and get the list of clone pairs. (2)Overlapping Detection • Each code clone instance is located onto the source code. • The overlapping fragments are identified. System S’ 15

Complete-Overview(3) Clone A’ Clone chunk Clone AB Clone B’ Clone AB (3)Clone Chunk Construction • Each code clone instance is divided into clone chunk. • Clone chunks internally contain no partial overlapping fragment with any other clones. System S 16

Complete-Overview(4)(5) Clone A’ Clone AB Clone B’ (4)Construction of Shared Functions • For each clone set, a set of shared functions are created based on the finest partition of each clone chunk. Clone AB System S (5)Removing Clones in Shared Functions Remove cloned overlapping fragments Create for overlapping fragment Shared Function A’ Call AB Shared Function B’ AB 17

Complete-Overview Clone A’ Clone chunk Clone AB Call B’ Clone AB Clone AB System S Call A’ Create shared function for each clone chunk created by overlapping fragment Remove cloned overlapping fragments Create for overlapping fragment Call A’ Call AB Call’ AB Shared Function A’ Call AB Shared Function B’ AB System S’ 18

Software Maintenance Business l. In industry, this business is very popular. l. A software maintenance company • The maintenance of the system was not developed by itself. are interested in the effort of existence of e clone to the maintenance activities.

Basic Slimming the system by merging only function clones l. Objective l. To show a lower bound of the code reduction. l. With a simple strategy easily performed by developers. External Code Clone Detector (CCFinder. X) Extraction of Function Clones Report r and |S’| Source Code S Merge Function Clones and Calculate r 20

Complete Slimming the system by merging all clones l. Objective l. To show a higher bound of the code reduction. l. Merging and removing greedily all clone instances. l. Note l. We do not care about the feasibility and usefulness of this merging in practice. 21

Heuristic Slimming the system by selecting and merging only one of overlapped clones l. Objective l. To show modest of the code reduction. l. The refactoring will be much easier than Complete Mehod. 22

Resources for the Evaluation LOC(Lines Of Code) l. Measuring the system’s size easily. l. But it would not be sufficient. l. Containing much unused code. l. Implementing very complex architecture and algorithm. l. Including a lot of code clones created over its evolution. 23

Complete vs Heuristic Strategy Clone A Overlapping fragment Call A Call B Clone A Straightforward refactoring Clone B System S Overlapping fragments as clones Overlapping Call Afragment is executed twice by both A and Call. BB Shared Function A Shared Function B System S’ 24

Complete Devided into Clone Chunk Clone A’ Clone chunk Clone AB Call B’ Clone AB Clone AB System S Call A’ Create shared function for each clone chunk created by overlapping fragment Remove cloned overlapping fragments Create for overlapping fragment Call A’ Call AB Call’ AB Shared Function A’ Call AB Shared Function B’ AB System S’ 25

Heuristic ⅠSelecting Merge Candidates Clone A Overlapping fragment Statement B Clone A Clone B System S Call A |A| > |B| Call A Statement B Shared Function A System S’ 26

Heuristic ⅡSelecting Merge Candidates Clone A Overlapping fragment Call B Clone A Clone B System S Call A A Statement |A| < |B| Call A A Statement Call B Shared Function B System S’ 27

Code Clone l. A code fragment with an identical or similar code fragment. l. Created by ‘copy-and-paste’ to reuse code. l. The existence of code clone affects the maintenance activities. Clone Set Code Clone File A File B 28

Software System Maintenance Activity l. Such as bug fixing and new feature addition. Maintenance Cost l. Determined by the scale and complexity of the system. The users of huge and decayed systems may evaluate the value of current running system. l. Keep paying the cost. l. Abandon the current systems and rebuild a new system. 29

Evaluation-OSS Ⅰ l. RQ 1: What are the reduction rates of Basic, Complete, and Heuristic Methods for popular OSS systems in written C/C++? System Use Total KLOC Clone KLOC(%) git Distributed versioning 157. 4 1. 9(1. 2) libcurl Client-side URL transfer 107. 5 11. 0(10. 2) Skynet Game framework 31. 4 0. 6(2. 0) Linux Operating system 11, 852. 6 864. 2(7. 3) Reduction by 3 Slimming Methods System Basic(% against total) Complete(%) Heuristic(%) git 63(0. 04) 1, 209(0. 8) 681(0. 4) libcurl 1. 766(1. 6) 6, 116(5. 7) 4, 876(4. 5) Skynet 6(0. 02) 246(0. 8) 212(0. 7) 117, 025(1. 0) 531, 285(4. 5) 372, 453(3. 1) Linux

Thanks for listening.