COMPARING THE DEFECT REDUCTION BENEFITS OF CODE INSPECTION

COMPARING THE DEFECT REDUCTION BENEFITS OF CODE INSPECTION AND TEST -DRIVEN DEVELOPMENT By Veronika Movagharianpour and Adam Brakel

Introduction � � � Software Developers face challenges: Producing high-quality software with low-defect levels while doing it on-time and on-budget

4 Cases to Explore Case 1: No Code Inspections or Test Driven Development Case 2: Code Inspection Only Case 3: Test Driven Development Case 4: Code Inspection and Test Driven Development

No Code Inspections and No. Test Driven Development � � � We know from our reading and other student’s reports that programs will have bugs. That the everyday debugging that occurs does not catch all of them. So NOT testing in addition to the minimal amount carried out is just a poor design choice!

Code Inspection We all know what the process for Code Inspections is…

Error Detection Efficiency of Software Inspection Defect yield rates for SI ranges between 66 and 82 percent where the total number of defects prior to inspection is: The Defect Detection Yield was found to be:

Test Driven Development � � Writing automated unit tests before code Subsequent coding completed when all new and previous tests succeed Process completed iteratively until software is complete Successful unit tests execution verified after write code and refactor steps

The Process of TDD � � � Identifies and removes defects at the introduced point in the development process Early defect elimination is the TDD most significant benefit Leads to huge cost saving due to early defect removal

An Example of Test Driven Development � Example Explanation – � Run Demo -

Purpose & Research Questions: � 1. Which software defect reduction method is most effective at reducing software defects? H 1: Code inspection is more effective than TDD at reducing software defects. � 2. Are there interaction effects associated with the combined use of these methods? H 2: The combined use of code inspection and TDD is more effective than either method alone. � 3. What are the relative costs of these software defect reduction methods? H 3: Code Inspection and TDD differ in implementation cost

Research Method � � � Programming assignment: creating part of a spam filter using Java 554 non-commentary source statements on average 261 NCSS provided in a starter project

Participants Excluded by Group: � Participants: � undergraduate computer science students � With good Java programming and object-oriented design knowledge � Highly motivated to perform well

Experimental Procedures: � � � All Participants were trained in the Inspection Process and TDD using JUnit. They were then given a detailed description of the spam filter they were to code and were given two weeks to do so. Then the resulting code from each of the four groups was analyzed.

Measurements: Measurement explanation: - Using “yield” to measure selected method effectiveness - Yield measure is not reliably for TDD number of defects found by using the method Yield = ----------------------------------Total Number of defects prior to inspection

Cost of Testing Methods: � The total cost is the sum of three areas of activity; � 1. the original development hours by the code author. � 2. the hours spent by software inspectors and moderator (preparation time + meeting time). � 3. hours spent by the author correcting defects by inspection. � � TDD seems to save the implementation cost Inspection had the highest implementation cost

Implementation Cost:

Threats to Validity: � Internal: � Section bias: possibility of unequal participants division � Mortality bias: unequal number of members in each group � Maturation bias: unequal participants learning rates during experiment � Order bias: vulnerability to order bias caused by performed inspections order and whether they were performed on the first or second inspection day � Implementation bias: variability in the treatment conditions implementing or applying means

Threats to Validity (continue) � External: � Participants were undergraduate junior or senior students, not professional programmers � Code inspectors were also college students without code inspection experience � Code inspection process did not follow the normal regulations. Authors did not participate the inspections The iterative cycle of rework, follow-up, and reinspection did not performed � Inspectors performed multiple inspections of code performing the same function in a short time resulting in finding more defects in the later inspections

Results:

Implementation Cost Summaries:

Summary of Results

Summary and Conclusions: � Compare of defect rates and implementation costs associated with code inspection and test-driven development software defect reduction methods: � Inspection: More effective at reducing defects More expensive to implement � TDD May result in implementation cost savings � More research is require to validate this findings

¿¿¿ Questions ? ? ?