Total Defects Injected and Test Defect Rates of

Total Defects Injected and Test Defect Rates of 810 Experienced Engineers 12/23/2021 2

Total Defects Injected and Test Defect Rates of 810 Experienced Engineers • Developers typically inject about 100 defects per 1, 000 lines of the code (KLOC) they write – 1 defect/10 lines of code • There is considerable variation between engineers – Between 40 and 250 of Injected Defects/KLOC • About 50 Defects/KLOC at start of testing • Between 10 to 20 Defects/KLOC entering System testing 12/23/2021 3

The Testing Problem • Programs generally have about one branch instruction for every 10 or so lines of code. – For a 100, 000 LOC = 10, 000 branch instructions – Number of test paths • For a simple network of 16 switches, there are 70 possible paths from A to B. 12/23/2021 4

The Testing Problem • Number of test paths ! Because of the growing complexity of modern systems, it is impossible to test all of the ways in which such systems could be used. 12/23/2021 5

Successful Quality Strategies (Manufacturing) • Quality levels of a few defects per million parts – e. g. 6 sigma = less than 3. 4 defects per 1, 000 parts • Eight Elements: 1. Establish quality policies, goals, and plans. 2. Properly set up the machines. 3. Keep the machines supplied with high quality parts and materials. 4. Maintain the entire process under continuous statistical control. 5. Evaluate the machine outputs. 6. Properly handle all deviations and problems. 7. Suitably package, distribute, or otherwise handle the machine outputs. 8. Consistently strive to improve all aspects of the production and evaluation processes. What are the equivalent elements for software ? 12/23/2021 6

Eight Elements of Software Quality Management 1. Establish quality policies, goals, and plans. 2. Properly train, coach, and support the developers and their teams. (document the processes) 3. Establish and maintain a requirements quality-management process. 4. Establish and maintain statistical control of the software engineering process. 5. Review, inspect, and evaluate all product artefacts. 6. Evaluate all defects for correction and to identify, fix, and prevent other similar problems. 7. Establish and maintain a configuration management and change control system. 8. Continually improve the development process. 12/23/2021 7

Eight Elements of Quality Management Manufacturing Software 1. Establish quality policies, goals, and plans. 2. Properly set up the machines. 1. 3. Keep the machines supplied with high quality parts and materials. 3. 4. Maintain the entire process under continuous statistical control. 4. 5. Evaluate the machine outputs. 5. 6. Properly handle all deviations and problems. 6. 7. Suitably package, distribute, or otherwise handle the machine outputs. 8. Consistently strive to improve all 7. aspects of the production and 12/23/2021 evaluation processes. 2. 8. Establish quality policies, goals, and plans. Properly train, coach, and support the developers and their teams. Establish and maintain a requirements quality-management process. Establish and maintain statistical control of the software engineering process. Review, inspect, and evaluate all product artefacts. Evaluate all defects for correction and to identify, fix, and prevent other similar problems. Establish and maintain a configuration management and change control system. Continually improve the development process. 8