Q Software Measurement and Process Improvement Mehmet Tumer

Q Software Measurement and Process Improvement Mehmet Tumer Chicago Software Process Improvement Network C-SPIN 3/11/99 1

Introduction Role of measurement in software process improvement l Examples of software development process metrics l Recommendations for establishing and maintaining a measurement program l C-SPIN 3/11/99 2

Agenda Overview of Publitec l Project Management Metrics l Defect Based Metrics l Development Process Metrics l Quality System Metric l Tools for Metrics System l Summary l Feedback and Experience Sharing l C-SPIN 3/11/99 3

Publitec Internal software house l Established in the Netherlands in 1987 l Re-organization in 1991 l Established the Quality Assurance Group l C-SPIN 3/11/99 4

Process Improvement Approach Involvement by everyone l Decisions based on facts l Prioritization, implement critical processes first l Define Measurable processes l C-SPIN 3/11/99 5

Major Milestones Sep. ‘ 91 l Sep. ‘ 92 l Feb. ‘ 93 l Aug. ‘ 93 l Oct. ‘ 96 l Jan. ‘ 97 l C-SPIN Establish QA Group Procedures Implemented ISO 9001 Trial Audit ISO 9001 Certification SEI CMM training CMM assessment by independent organization 3/11/99 6

Objectives l Deliver at the required time within the estimated cost » Project Management Metrics l Deliver the expected functionality » Defect Metrics » Development Process Metrics l Justify Cost of Quality » Quality System Metric C-SPIN 3/11/99 7

Product and Process Measurement Guidelines Application: What does the metric indicate l Primitives: What is the basic data, how to capture l Implementation: How to calculate l Interpretation: Meaning and targets l C-SPIN 3/11/99 8

Measurement METRICS QUALITY SYSTEM METRICS C-SPIN DEVELOPMENT PROCESS METRICS DEFECT METRICS 3/11/99 PROJECT MANAGEMENT METRICS 9

Project Management Metrics l Cost performance Outlook » » l Project Plan Visibility » l Indicates the overall visibility of project plans Software development Effort Distribution » C-SPIN Provides a tool for measuring the Cost and schedule maintenance against the project plan At the end of the project provides historic data on the initial estimate, final estimate, and the actual cost and schedule. Provides the distribution of effort for the software development projects across different phases in the development life cycle. 3/11/99 10

Growth of a Software Problem ? REALITY APPEARANCES Requirement Definition C-SPIN Design Coding 3/11/99 Testing 11

Visibility C-SPIN 3/11/99 12

C-SPIN 3/11/99 13

C-SPIN 3/11/99 14

C-SPIN 3/11/99 15

C-SPIN 3/11/99 16

On Time Delivery C-SPIN 3/11/99 17

C-SPIN 3/11/99 18

Defect Based Metrics l C-SPIN Defect Days Number » The effectiveness of the software development process depends upon the timely detection and removal of detects across the entire life cycle. » The earlier in the development cycle a defect is identified, the cheaper it is to remove that defect. » This metric represents the number of days that defects spend in the software system from their introduction to their detection and removal 3/11/99 19

C-SPIN 3/11/99 20

Defect Based Metrics l Defect Distribution by Cause Category This metrics provides information on the reason of defects l Defect Distribution by Discovery Mechanism This defect indicates where in the development process, majority of the defects are identified. It is also possible to see the effectiveness of the review and testing process in identifying the defects before they are found by the customers. C-SPIN 3/11/99 21

C-SPIN 3/11/99 22

Discover Mechanism C-SPIN 3/11/99 23

Development Process Metrics These metrics are used to judge how effective the development process is in identifying and avoiding defects. These are plotted against the timeline to indicate the improvement in the process. Analysis (Design) Review Efficiency Indicates the percentage of analysis (design) defects found at the analysis (design) review, right after the analysis (design) activity. Long Stay Defects Percentage of the defects which took more than 100 days to remove after their introduction l Late Detect Defects Percentage of the defects which took more than 30 days to detect after their introduction C-SPIN 3/11/99 24 23

C-SPIN 3/11/99 25

Quality System Metric This metric is used to measure the effectiveness of the quality system l QUALITY LOSS l QUALITY INVESTMENT » Cost of Rework » Cost of Quality System » Cost of Scrap » Cost of Project QA Activities QUALITY COST = QUALITY LOSS + QUALITY INVESTMENT C-SPIN 3/11/99 2625

C-SPIN 3/11/99 27

Tools for Metrics System 4 Scheduling Tool 4 Tracking Tool (Time Registration Tool) 4 Defect Management Tool 4 Reporting Tool C-SPIN 3/11/99 28

Summary Improvement is not possible without measurement l Design measurable processes l Involve all Stakeholders in the description of the processes and metrics l C-SPIN 3/11/99 29

Summary Measure with a focus l Set targets and track achievement against the targets l Don’t ignore results, provide continuous feedback l Never use metrics to evaluate groups or individuals l C-SPIN 3/11/99 30

Summary Capture as much data as possible, report only those metrics that make sense l Audit the data collection process l Validate the conclusions with different metrics l Beware of aging metrics l C-SPIN 3/11/99 31