Measuring Software Reuse Federal Chief Architects Forum 11

Measuring Software Reuse Federal Chief Architects Forum 11 May 2004 Dr. Jeffrey S. Poulin Lockheed Martin Distribution Technologies MD 0220 Owego, NY 13827 Tel: (607) 751 -6899 Jeffrey. Poulin@lmco. com http: //home. stny. rr. com/jeffreypoulin © Dr. Jeffrey S. Poulin, 2004

My background • Joined IBM Reuse Technology Center- 1991 – Responsible for metrics, reuse business case, and standards – Developed IBM’s first, accepted reuse metrics and ROI model • Worked on Reuse Metrics and Return-on-Investment (ROI) – Organized or helped organize numerous Reuse Conferences – 70+ publications, including editor of 3 conference proceedings – Author Measuring Software Reuse, Addison-Wesley, 1997 • System Architect / Technical Lead for Lockheed Martin – Programs from $15 – $500 million – Currently fielding a system to automatically read the addresses on all the mail in the UK and provide data to mail sorters. • http: //home. stny. rr. com/jeffreypoulin/ © Dr. Jeffrey S. Poulin, 2004

How do you start a Reuse Metric Program? • What are your goals? • What data do you have readily available? • Are the metrics you are considering simple and understandable? • Do your developers understand what future applications should “look like”? • Do your developers understand what components are available, how to find them, and how to use them? • How will you drive technical, organizational, and cultural change? © Dr. Jeffrey S. Poulin, 2004

Today’s Objectives 1. Convince you of the value of reuse! 2. Convince you it can be done. • Metrics and Motivators (“Governance”) • Payback- ROI (“RCR, ” “RCWR”, “Cultural Motivators”) • Barriers/Challenges • Organizing for Reuse • Case Studies / Questions © Dr. Jeffrey S. Poulin, 2004

Estimating the Value of Reuse © Dr. Jeffrey S. Poulin, 2004

Cost-Benefit Analysis C-B analysis requires assigning values to all known costs and benefits, including intangible items (e. g. , quality), summing the values, and applying a discount factor depending on the time period of the analysis. * * Poulin, Jeffrey S. , Debera Hancock and Joseph M. Caruso, “The Business Case for Software Reuse, ” IBM Systems Journal, Vol. 32, No. 4, 1993, pp. 567 -594. © Dr. Jeffrey S. Poulin, 2004

Cost-Benefit Analysis of Reusing Software Table: Benefits of Reusing Software © Dr. Jeffrey S. Poulin, 2004

Cost-Benefit Analysis of Reusing Software (cont. ) Table: Costs of Reusing Software © Dr. Jeffrey S. Poulin, 2004

Cost-Benefit Analysis of Reusing Software (cont. ) Table: Benefits of Producing Reusable Software © Dr. Jeffrey S. Poulin, 2004

Cost-Benefit Analysis of Reusing Software (cont. ) Table: Costs of Producing Reusable Software © Dr. Jeffrey S. Poulin, 2004

The Relative Cost of Reusing Software (RCR) Reuse does not come for free. The reuser must: 1. 2. 3. 4. 5. Locate Understand Integrate System test etc. with the reused component. However, reuse typically only takes about 20% of the effort of writing the same component from scratch. In this case, the relative cost to reuse the component equals 0. 2, and we have saved 80% of the development effort. Definition: Assume the cost to develop a new component equals one unit of effort. We call the portion of this effort that it takes to reuse a similar component without modification (black-box reuse) the Relative Cost of Reuse (RCR). © Dr. Jeffrey S. Poulin, 2004

The Range of Values for RCR Table: Summary of Relative Cost of Reuse (RCR) Values Recommended Default Value for RCR=. 2 © Dr. Jeffrey S. Poulin, 2004

The Relative Cost of Writing Reusable Software (RCWR) Writing reusable components requires additional effort beyond simply writing a component for one-time use. The reuse producer must: 1. 2. 3. 4. 5. Generalize for additional requirements Add more detailed documentation Test to increase trust Test for additional potential uses Prepare the component for distribution It typically takes about 50% more effort to write a component for reuse than to write the same component for one-time use. In this case, the relative cost to write for reuse equals 1. 5. Definition: Assume the cost to develop a new component for one-time use equals one unit of effort. We call the portion of this effort that it takes to write a similar “reusable” component the Relative Cost of Writing for Reuse (RCWR). © Dr. Jeffrey S. Poulin, 2004

The Range of Values for RCWR Table: Summary of Relative Cost of Writing for Reuse (RCWR) Values Recommended Default Value for RCWR= 1. 5 © Dr. Jeffrey S. Poulin, 2004

Reuse Metrics that You can Use © Dr. Jeffrey S. Poulin, 2004

Recommended Reuse Metrics 1. Reuse% © 2. Reuse Cost Avoidance (RCA) 3. Organizational ROI (ROIorg) © Dr. Jeffrey S. Poulin, 2004

1. Reuse% © Example: Your programming team developed and maintains 100 k LOC, and also uses 20 k LOC from an external source. Your team’s Reuse% equals: © Dr. Jeffrey S. Poulin, 2004

2. Reuse Cost Avoidance (RCA) Purpose: A Cost Avoidance metric used to show the financial benefit of reusing software to the individual project or team. RCA represents money you didn’t have to spend. We recognize cost avoidance: 1. 2. During development During service (i. e. , maintenance) © Dr. Jeffrey S. Poulin, 2004

2. Reuse Cost Avoidance (RCA) Reuse Cost Avoidance = Development Cost Avoidance + Service Cost Avoidance • Development: The benefits of reuse depend on the Relative Cost of Reuse (RCR). With an RCR=0. 2, reuse costs about 20% of the cost of new development. Therefore Development Cost Avoidance (DCA): DCA = RSI x (1 - RCR) x (New Code Cost) DCA = RSI x. 8 x (New Code Cost) • Service: Not having to fix errors results in a Service Cost Avoidance (SCA): SCA = RSI x (Your Error Rate) x (Your Error Cost) © Dr. Jeffrey S. Poulin, 2004

2. RCA Example • Development: If your development cost for new code equals $100 per line and your RCR = 0. 2, then your DCA for 20 k LOC RSI equals: Development Cost Avoidance = 20 k LOC x. 8 x $100 per line = $1. 6 million • Service: If your error rate for new code equals 1. 5 errors per k. LOC and if your cost to fix an error equals $10 k, then your SCA for 20 k LOC RSI equals: Service Cost Avoidance = 20 k LOC x 1. 5 errors per k. LOC x $10 k per error = $0. 3 million • Total Reuse Cost Avoidance: Reuse Cost Avoidance = DCA + SCA = $1. 6 million + $0. 3 million = $1. 9 million © Dr. Jeffrey S. Poulin, 2004

3. Organizational ROI Purpose: To show the total financial benefit to an organization from: 1. 2. Reusing software Building software to reuse • ROIorg = RCA - Additional Development Costs • Additional Development Costs (ADC) equals (the amount of source instructions you write for reuse by others) * (Relative Cost of Writing for Reuse - 1) * New Code Cost © Dr. Jeffrey S. Poulin, 2004

3. Organizational ROI Example • You have calculated your Reuse Cost Avoidance (RCA) (the net benefit of reuse) to equal $1. 9 million. However, you also invested extra effort in 16, 000 LOC Source Instructions Written for Reuse by Others (WRO). With an RCWR = 1. 5, your Additional Development Cost (ADC) equals: ADC = WRO x (RCWR - 1) x New Code Cost ADC = 16 k LOC x (1. 5 - 1) x $100 per line = $. 8 million • Organizational ROIorg = RCA - ADC = $1. 9 million - $0. 8 million = $1. 1 million © Dr. Jeffrey S. Poulin, 2004

Reuse Pays-off in Two Reuses! Easy to write, easy to reuse Nominal payback in 1 use + 1 reuse Hard to write, hard to reuse Observations: 1. The more difficult to write, the longer it takes to pay back. 2. The more difficult to reuse, the longer it takes to pay back. 3. With RCWR = 1. 5 and RCR = 0. 20 show breakeven at less than 1 use + 1 reuse. © Dr. Jeffrey S. Poulin, 2004

Defining Reuse with Metrics © Dr. Jeffrey S. Poulin, 2004

Defining Reuse with Metrics • Use Reuse Metrics to drive reuse goals • Most software metrics encourage writing code – We want to discourage writing code • The definition of a “what to count” affects programmer behavior* – Provide management emphasis on the metric * Poulin, Jeffrey S. , “Issues in the Development and Application of Reuse Metrics in a Corporate Environment, ” Fifth International Conference on Software Engineering and Knowledge Engineering , San Francisco, CA, 16 -18 June 1993, pp. 258 -262. © Dr. Jeffrey S. Poulin, 2004

Major Reuse Level Metric Models • Almost everyone reports “Percent Reuse: ” Reused Software Percent Reuse = Total Software – Easy to understand – Easy to calculate – De facto standard to communicate the level of reuse activity • No one defines what to count! © Dr. Jeffrey S. Poulin, 2004

The Boundary Problem © Dr. Jeffrey S. Poulin, 2004

Goal and Assumptions Goal: To promote formal reuse across organizations, not just to reward “good programming practice. ” Assumptions: 1. Informal reuse (software sharing) commonly happens within organizations 2. Sharing software between organizations happens less often 1, 2 1 Bollinger, T. B. , and Pfleeger, S. L. , “Economics of Reuse: Issues and Alternatives, ” Inf. Software Technology, Vol. 32, No. 10, December 1990, pp. 643 -52. 2 Kraut, Robert E. and Lynn A. Streeter, “Coordination in Software Development, ” Communications of the ACM, Vol. 38, No. 3, 1995, pp. 69 -81. © Dr. Jeffrey S. Poulin, 2004

Boundary example: Small Project Figure: Internal and External Reuse on a Small Project © Dr. Jeffrey S. Poulin, 2004

Boundary example: Large Project Figure: Internal and External Reuse on a Large Project © Dr. Jeffrey S. Poulin, 2004

Example Organization for Reuse Project Manager . . . Reuse Development Manager Development Manager Reuse Team REUORG. pre Figure: Organizing a Large Software Development Project for Reuse © Dr. Jeffrey S. Poulin, 2004

Program A • A planned suite of 80+ applications in seven domains, primarily in: 1. Resource Management, which contains 10 applications, 2. Logistics, which contains 13 applications. 3. Personnel, which contains 18 applications, and 4. Information Management, which contains 16 applications. • The Reuse Development Team created 30 domain-specific components to provide services to all (or most) applications. – Services included: Scheduling, Progress Status, Electronic Signature, Help, Business Graphics, Data Validation, Access Control, Mail Interface, File Transfer, Audit Trail, Print, etc. – These comprise almost 20% of the software in the first release – Reuse Cost Avoidance of approximately $7 million. © Dr. Jeffrey S. Poulin, 2004

Program A • In addition, the program obtained and used (without modification) components from Customer-supplied sources. • Resulted in Reuse Cost Avoidance (RCA) of > $1. 9 million. * * Army Reuse Center (ARC), "Reuse Benefits Continue to Add Up, " The Army Reuse Center News, Vol. 3, No. 3, October 1994, p. 3. © Dr. Jeffrey S. Poulin, 2004

“What do we Count? ” © Dr. Jeffrey S. Poulin, 2004

Example: Measuring “uses” vs “reuses” • A project reported 11 k LOC of reuse • 5120 lines came from one 10 -line reusable macro in the same module • The original code: 1 Do i : = 1 to 512 MACRO ( i ); 2 “Unrolled” to yield: MACRO ( 1 ); MACRO ( 2 ); MACRO ( 3 ); MACRO ( 4 ); MACRO ( 5 ); MACRO ( 6 ); . . . MACRO ( 509 ); MACRO ( 510 ); MACRO ( 511 ); MACRO ( 512 ); 1 Should be counted as 2 source instructions and 10 reused instructions. 2 Was reported as 512 source instructions and 5120 reused instructions. © Dr. Jeffrey S. Poulin, 2004

Example: Measuring Generated Code • Situation: – – – • A development team used a “WYSIWYG” tool to lay out a large number of window-based forms that input and extract data from a Human Resources database. The tool generated 97, 000 lines of “C” and “Java” to implement the X-window interface and database interactions. The team added 6, 000 lines of “C” to implement the company’s unique business logic. The team reported: – – Producing 103, 000 lines of code A reuse level of “ 94%” 1 Should be counted as 97, 000 lines of generated plus 6, 000 lines of new source instructions. 2 No reuse; reuse would be based on the 6, 000 lines of new development. © Dr. Jeffrey S. Poulin, 2004

Example: Measuring “white-box” vs “black-box reuse” “White-box” reuse saves a little effort here. . . Development Maintenance “Black-box” reuse saves a lot of effort here. . . plus here Figure: The benefits of reusing without modification dwarf those obtainable by modifying the software. © Dr. Jeffrey S. Poulin, 2004

Why you should discourage “white-box” reuse - How do we measure the extent of the modification? Did the programmer modify 1% or 99% of the component? - What benefits did we receive? Modifying can cost more than writing the new code!* Cost Relative to Totally New Code 2. 0 ase t. C rs Wo 1. 5 Breakeven 1. 0 t es B 0. 5 se Ca Reuse Saves Money 0. 0 0. 5 Fraction of Code Modified 1. 0 Example: Reusing Software is Not Always Cost-Effective. Figure: The Relative Costs of White-box Reuse * Stutzke, Richard D. , “Software Estimating Technology: A Survey, ” Crosstalk: The Journal of Defense Software Engineering, Vol. 9, No. 5, May 1996, pp. 17 -22. © Dr. Jeffrey S. Poulin, 2004

Conclusion: Being clear about “what to count” is essential! • Define what you want to happen • Measure the benefits of what happens • Provide an equitable basis for incentives © Dr. Jeffrey S. Poulin, 2004

A Reuse Strategy for Large Organizations © Dr. Jeffrey S. Poulin, 2004

The Three Classes of Software Application “Application. Specific” Layer Intel App#1 Intel App#2 Components Citicorp App#1 Citicorp App#2 Components Allstate App#1 Allstate App#2 100% Components up to 85% “Domain. Dependent” Layer “Domain. Independent” Layer Microprocessor Business Components Financial Business Components Insurance Business Components 15 -20% General-purpose utilities, APIs, Abstract Data Types, System Service Components 0% © Dr. Jeffrey S. Poulin, 2004

An Reuse Strategy for Large Organizations • Phase I: Pilot Domain-Independent Reuse – – Provide low-level, commonly “re-used” assets Make the assets widely available Establish procedures to manage, control, encourage their use Set a goal of 15 -20% reuse based on these assets • Phase II: Expand into Domain-Dependent Reuse – Incrementally identify and create assets that are core to your business – Focus on components needed by at least two projects – Build on the procedures established in Phase I – Set higher (but realistic goals) © Dr. Jeffrey S. Poulin, 2004

Multi-tiered Web Architecture (J 2 EE) App#1 App#2 I. T. Business Components App#1 App#2 Financial Business Components App#1 App#2 Insurance Business Components General-purpose utilities, APIs, Abstract Data Types, System Service Components © Dr. Jeffrey S. Poulin, 2004

Multi-tiered Web Architecture (. NET) App#1 App#2 I. T. Business Components App#1 App#2 Financial Business Components App#1 App#2 Insurance Business Components General-purpose utilities, APIs, Abstract Data Types, System Service Components © Dr. Jeffrey S. Poulin, 2004

Project R • “I've read your material and used your metrics calculations in presenting the case for a companysponsored and sanctioned software reuse mandate…” • “I've been spearheading the effort to gain widespread acceptance…” • “When it comes to "making the numbers", they don't want to drink the Kool-Aid. ” • “I'm trying to fortify the justification for creating a team of engineers to manage, maintain, harvest, acquire, and develop software components that can be used across the [xxx] brand…[using] the ROI for a single common component. ” © Dr. Jeffrey S. Poulin, 2004

Conclusion 1. Reuse is undeniably valuable to your organization! • The ROI is ~80% on whatever you reuse (code, patterns, documents) 2. Reuse can be done! • Metrics help provide incentive • Organizational boundaries inhibit reuse • A common architecture provides the framework • Organize, plan, and provide tools that reduce the boundaries! • Start with low-level components – then Grow components that are important to your business! © Dr. Jeffrey S. Poulin, 2004