Mining Software Repositories to Assist Developers and Support

  • Slides: 38
Download presentation
Mining Software Repositories to Assist Developers and Support Managers Ahmed E. Hassan Soft. Ware

Mining Software Repositories to Assist Developers and Support Managers Ahmed E. Hassan Soft. Ware Architecture Group (SWAG) University Of Waterloo ahmed@alumni. uwaterloo. ca

Software Repositories • Track the evolution of a software project: – source control systems

Software Repositories • Track the evolution of a software project: – source control systems store changes to the code – defect tracking systems follow the resolution of defects – archived personnel communications record rationale for decisions throughout the life of a project • Used primarily for historical record supporting activities such as: checking bug status, or retrieving old code 2

Usage of Software Repositories Beyond Record Keeping • Prior Research: – SEL - NASA

Usage of Software Repositories Beyond Record Keeping • Prior Research: – SEL - NASA [Basilli, et. al]: • Fault analysis (Fortran 84) – 5 ESS - Lucent [Eick, Graves, Mockus, Perry, et. al]: • code decay, fault and effort predictions, visualizations, process – EMERALD - Nortel [Hudepohl, Khoshgoftaar, et. al] – Telephony - Nokia [Gall et. al]: • reveal hidden dependencies – Open Source [Chen et. al]: • index code for searching • Personal Industrial Experience • Open Source Phenomena: easy access to rich repositories for large projects 3

Survey on Usage of Source Control Repositories in Industry • Developers: – More likely

Survey on Usage of Source Control Repositories in Industry • Developers: – More likely to record the change rationale than list implementation alternatives or limitations – Use change msgs to understand legacy code and fix bugs • Senior developers use messages to: – Track the progress of the project – Monitor the quality of the code (specific developers) – Ensure code ownership boundaries • Change messages in open source projects are as descriptive as ones in industrial systems 4

Bugzilla CVS Mailings 5

Bugzilla CVS Mailings 5

Software Understanding Change Propagation Resource Allocation Bug Prediction Bugzilla Historical Information Extraction CVS Mailings

Software Understanding Change Propagation Resource Allocation Bug Prediction Bugzilla Historical Information Extraction CVS Mailings 6

Software Understanding Change Propagation Resource Allocation Bug Prediction Bugzilla Historical Information Extraction CVS Mailings

Software Understanding Change Propagation Resource Allocation Bug Prediction Bugzilla Historical Information Extraction CVS Mailings 7

Extracting Information from Source Control Repositories • CVS tracks changes at the line level,

Extracting Information from Source Control Repositories • CVS tracks changes at the line level, instead an evolutionary extractor tracks: – Addition, removal, or modification of code entities (functions, variables & macros): • function_1 is added/removed/modified – Addition and removal of dependencies: • function_1 no longer calls function_2 • Attaches additional CVS change information: developer name, co-changed entities, change reason 8

Complexity of Evolutionary Extraction main() { int a; /*call help*/ help. Info(); } V

Complexity of Evolutionary Extraction main() { int a; /*call help*/ help. Info(); } V 1: Undefined func. (Link Error) help. Info() { error. String! } main() { int a; /*call help*/ help. Info(); } V 2: Syntax error help. Info() { int b; } main() { int a; /*call help*/ help. Info(); } V 3: Valid code 9

Evolutionary Extractors: Challenges and Complexity • Unstable & Evolving Code • Scalability and Robustness

Evolutionary Extractors: Challenges and Complexity • Unstable & Evolving Code • Scalability and Robustness – Large amount of data/snapshots – Legacy systems are hard to parse • Accuracy – Entities with similar names throughout time 10

Assisting Software Developers 11

Assisting Software Developers 11

Understanding the Software Architecture Conceptual (proposed) Why? Who? When? Where? Concrete (reality) 12

Understanding the Software Architecture Conceptual (proposed) Why? Who? When? Where? Concrete (reality) 12

Dependency Investigation Questions (W 4 Approach) • Which low-level code entity is responsible for

Dependency Investigation Questions (W 4 Approach) • Which low-level code entity is responsible for a dependency? – Network (Send. Data) Scheduler (Print. To. Log) • Who added/removed the dependency? – Junior vs. senior/experienced developer • When was the dependency modified? – Late night / Just before release • Why was the dependency added/removed? – The rationale! 13

Source Sticky. Notes • Static dependencies give a static view of the system –

Source Sticky. Notes • Static dependencies give a static view of the system – not enough detail! • Need to extend static dependencies, but how? – Ask developers to fill Sticky. Notes for each change – Use software repositories to build these notes automatically 14

Case Study – Net. BSD (VM Subsystem) Conceptual (proposed) Why? Who? When? Where? Concrete

Case Study – Net. BSD (VM Subsystem) Conceptual (proposed) Why? Who? When? Where? Concrete (reality) 15

Unexpected Dependencies • Eight unexpected dependencies • All except two dependencies existed since the

Unexpected Dependencies • Eight unexpected dependencies • All except two dependencies existed since the first day: – Virtual Address Maintenance Pager – Pager Hardware Translations 16

Understanding Software Using Sticky. Notes • Traditional dependency graphs and program understanding models usually

Understanding Software Using Sticky. Notes • Traditional dependency graphs and program understanding models usually do not use historical information • In many open source projects, CVS comments are used for: – Communicating new features – Explaining design decisions – Narrating the progress of a project 17

Supporting Managers 18

Supporting Managers 18

Code Development Process • Central part in making software – Code changes to: –

Code Development Process • Central part in making software – Code changes to: – Implement/Enhance features – Repair faults • Acts as an early warning: – Does a complex code development process produce complex software? • Can we: – measure its complexity? – monitor the evolution of its complexity? – predict problems (delayed releases or faults)? 19

Development Process Chaos Domain Requirements Development Process Code/Design Team (size/structure) “Complexity is the business

Development Process Chaos Domain Requirements Development Process Code/Design Team (size/structure) “Complexity is the business we are in and complexity is what. Schedule limits us”. Fred Brooks, The Mythical Man-Month Pressure A chaotic code development process negatively affects its outcome, the software system, such as the occurrence of faults 20

Complexity of Development Process • “To implement feature A we modified one file” •

Complexity of Development Process • “To implement feature A we modified one file” • “To implement feature B we modified too many files – I don’t even remember half of them!” • Too much information to recall Decay of grasp of what is going in a complex system 21

Information Theory • Measures the amount of information in a message • Shannon Entropy

Information Theory • Measures the amount of information in a message • Shannon Entropy for a random prob distrib. P Hn(P) = - pk log 2(pk) • pk = 1: – If all pk = 1/n (equal prob. ) maximal entropy – If exists pi = 1 minimal entropy • Min. number of bits – Best compression 22

Info Theory in Dev Process 23

Info Theory in Dev Process 23

Evolution of Entropy 24

Evolution of Entropy 24

Adaptive System Sizing • System size changes over time • Standardized Shannon Entropy H:

Adaptive System Sizing • System size changes over time • Standardized Shannon Entropy H: H(P) = (log 2 n)-1 * Hn(P) = - (log 2 n)-1 * pk log 2(pk) • Many files are rarely modified (eg. util or libs): – A large n reduces the entropy – Use an active working set instead of n = # of files 25

Postgres – Development Chaos “This release marks a major step in the development team's

Postgres – Development Chaos “This release marks a major step in the development team's mastery of the source code we inherited from Berkeley. You will see we are now easily adding major features, thanks to the increasing size and experience of our world-wide development team” 6. 5 release notes 26

Case Study 2: KDE 2. 0 was also a very ambitious project, and the

Case Study 2: KDE 2. 0 was also a very ambitious project, and the nature of the changes delayed the project for a whole year”, Miguel de Icaza, founder of GNOME and Ximian 27

Mathematical Validation Details • Studied repositories of six open source software systems (Open. BSD,

Mathematical Validation Details • Studied repositories of six open source software systems (Open. BSD, Free. BSD, Net. BSD, Postgres, KDE, KOffice) • Built statistical regression models using the 2 nd and 3 rd years: – Faults – Changes – Entropy metric data to predict faults in 4 th and 5 th years 28

Results of Mathematical Validation • Prior faults better fault predictor than prior changes (~50%

Results of Mathematical Validation • Prior faults better fault predictor than prior changes (~50% improvement in prediction error) • Entropy metric better fault predictor than: – Changes (~13 -40% improvement) – Faults (~15 -38% improvement) • Results statistically significant at (95%) using a paired T-test 29

Development Chaos Summary • A new perspective on the complexity of software – focus

Development Chaos Summary • A new perspective on the complexity of software – focus on process instead of code • Entropy metrics used to study the evolution of complexity and predict faults • Results verified using 6 open source projects 30

Conclusion • Software repositories contain a wealth of valuable information about the evolution of

Conclusion • Software repositories contain a wealth of valuable information about the evolution of projects • Presented approaches and techniques to: – Recover data from source control repositories – Assist developers in understanding code structure – Support managers in predicting the occurrence of faults • MSR likely to take a central role in supporting software development practices and research 31

32

32

Thesis Contributions • Evolutionary Software Extractors (C-REX) • For Developers: – Source Sticky. Notes

Thesis Contributions • Evolutionary Software Extractors (C-REX) • For Developers: – Source Sticky. Notes – Development Replay Approach • For Managers: – Top Ten List – Software Development Chaos • All contributions were validated through: – case studies using open source systems, and – a survey of software practitioners 33

Research Overview • Develop approaches and techniques to : – Part I : Extract

Research Overview • Develop approaches and techniques to : – Part I : Extract Information from Repositories – Part II : Assist Developers in • Understanding the current code structure • Changing code and propagating changes – Part III: Support Managers in • Allocating limited testing resources • Predicting faults 34

Part II: Assisting Developers • Source Sticky. Notes: – Assist in architecture understanding –

Part II: Assisting Developers • Source Sticky. Notes: – Assist in architecture understanding – Attach historical information to the dependency graph • Development Replay: – Reenacts historical changes using proposed tools and strategies to assess their benefits – Permits us to perform empirical based estimation of the benefits of tools and strategies, e. g. change propagation 35

Part III: Supporting Managers • Top Ten List: – Assists managers in allocating limited

Part III: Supporting Managers • Top Ten List: – Assists managers in allocating limited resources • Development Process Chaos: – Develop a complexity metric based on the process followed by the developers to produce the code instead of on the code or the requirements 36

37

37

Research Hypothesis “Software repositories contain a wealth of valuable information about the evolution of

Research Hypothesis “Software repositories contain a wealth of valuable information about the evolution of a software project. By mining such historical information, we can develop techniques and approaches to support software developers and managers in their endeavors to build and maintain complex software systems” 38