Working Group Session Empirical Software Engineering Inadequate Metrics

Working Group Session: Empirical Software Engineering Inadequate Metrics Application Tomoo Matsubara Consulting matsu@computer. org ISFST 2005 in Shanghai, China Tomoo Matsubara 1

Introduction n n Besides laissez-faire organizations which have no needs to improve, most organizations do measure their activities and results such as scale of a product, consumed hours, and built-in defects during development, for their improvement. But most of organizations fail to apply collected data to solve problems and to improve their activities effectively. There are several common inadequate applications that I frequently observe at the sites. ISFST 2005 in Shanghai, China Tomoo Matsubara 2

Metrics Without Objectives n Measure and collect data first before thinking how to use them. n n For example, when a given objective is cost saving by quality improvement, an organization should collect cost of quality data. But most organizations take defects counting with no COQ data. In traditional engineering, objective comes first and think how to solve problems, then conducting experiments and producing prototypes if necessary. ISFST 2005 in Shanghai, China Tomoo Matsubara 3

Inadequate Representation n n Most organizations collect data but fail to represent for effective use of data. For understanding problems and for observing status of ongoing improvement, representation of data is critical. n n Most of a common representation is table forms like accounting people use and only few people can understand what it means. For controllability perspective data need to show diagrammatic form. But in software organizations diagrams are not widely used. ISFST 2005 in Shanghai, China Tomoo Matsubara 4

Inadequate Classification/Categorization n For understanding problems, classification/categorization is critical. But inadequate classifications are common. n n No orthogonality like counting apples and oranges together. Inappropriate granularity that classify subset and superset separately. ISFST 2005 in Shanghai, China Tomoo Matsubara 5

Excessive Precision n For understanding a problem, excessive precision is harmful and hide real cause rather than disclose. n For example, for analyzing reworks, most organizations tend to measure rework and classify them by cause of rework precisely. It hide woods (real cause) but disclose trees (trivial causes). ISFST 2005 in Shanghai, China Tomoo Matsubara 6