Statistical Software Quality Assurance Implies Information about defects

Statistical Software Quality Assurance • Implies – Information about defects is collected and categorized – An attempt is made to trace each defect to underlying cause – Use of Pareto Principle to identify vital causes (80% of defects can be traced to 20% of causes/mistakes) – Move to correct the problems that have caused the defects

Software Engineering II Lecture 30 Fakhar Lodhi

Recap

Example • Information about defects is collected for one year and categorized as follows: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Incomplete or erroneous specifications (IES) Misinterpretation of customer communication (MCC) Intentional deviation from specifications (IDS) Violation of programming standards (VPS) Error in data representation (EDR) Inconsistent component interface (ICI) Error in digital logic (EDL) Incomplete or erroneous testing (IET) Inaccurate or incomplete documentation (IID) Error in programming language translation or design (PLT) Ambiguous or inconsistent human computer interface (HCI) 12. Miscellaneous (MIS)

Error Category Serious Moderate Minor Sub Total IES 34 68 103 205 MCC 12 68 76 156 IDS 1 24 23 48 VPS 0 15 10 25 EDR 26 68 36 130 ICI 9 18 31 58 EDL 14 12 19 45 IET 12 35 48 95 IID 2 20 14 36 PLT 15 19 26 60 HCI 3 17 8 28 MIS 0 15 41 56 Total 128 379 435 942

Error Category % of Total Sub Total errors IES 205 22 MCC 156 17 IDS 48 5 VPS 25 3 EDR 130 14 ICI 58 6 EDL 45 5 IET 95 10 IID 36 4 PLT 60 6 HCI 28 3 MIS 56 6 Total 942 IES, MCC and EDR are vital errors - cause 53% of all errors

Error Category % of Serious errors Serious IES 34 27 MCC 12 9 IDS 1 1 VPS 0 0 EDR 26 20 ICI 9 7 EDL 14 11 IET 12 9 IID 2 2 PLT 15 12 HCI 3 2 MIS 0 0 Total 128 IES, EDR, PLT and EDL constitute about 80% of serious errors

Excel example • Now start corrective action focused on vital few • For example for EDR – Review the data representation techniques to identify the possible improvement areas – Adopt a use case tool for data modeling and perform stringent data design reviews

Error Index (EI) Used to develop an overall indication of improvement in software quality • Ei – the total number of errors uncovered during the ith step in the SE process • Si – number of serious errors • Mi – number of moderate errors • Ti – number of minor errors • PSi – product size at the ith step • ws, wm, wt – weighting factors for serious, moderate, and minor errors • Recommended values – 10, 3, 1 respectively

Error Index (EI) • At each step of the software process a Phase Index is computed PIi = ws(Si/Ei) + wm(Mi/Ei) + wt(Ti/Ei) • EI – cumulative effect on each PIi = ∑(i x PIi)/PSi – Weighting errors encountered in the SE processes more heavily than those encountered earlier – Used to develop an overall indication of improvement in software quality

Software Reliability • Defined as: – Probability of failure free operation of a computer program in a specified environment for a specified time – E. g. program X is estimated to have a reliability of 0. 96 over 8 elapsed hours. • What is meant by the term failure? – Failure is non-conformance to software requirements – Grading • From annoying to catastrophic • Time to fix from minutes to months • Ripples from fixing

Software reliability • Hardware versus software reliability – Hardware reliability is predicted on failure due to wear, rather than failure due to design – Software – no wear and tear • Mean time between failure – MTBF = MTTF + MTTR where MTTF is mean time to failure and MTTR is mean time to repair

Software reliability • Arguably MTBF is far better than defects/kloc – Each error does not have the same failure rate – User is concerned with failure and not with total error count