COSMIC Function Points Evaluation for Software Maintenance Anandi

COSMIC Function Points Evaluation for Software Maintenance Anandi Hira and Barry Boehm University of Southern California ISEC 2018, Hyderabad, India

Outline Motivatio n Dataset Metrics Analysis Approach Results Conclusio n

Motivation • 2003: Despite research in cost and effort estimation, still a problematic area • Most effort models use SLOC (Source Lines of Code) • Cannot be estimated until end of project • IFPUG Function Points (FPs) -> successful standard in many environments • Inapplicable to projects outside of Management Information Systems (MIS) domain • COSMIC Function Points (CFPs) developed a size metric based on “basic principles” to apply to wider range of application domains.

COSMIC Function Points (CFPs) Successes • Web applications • Real-time systems • Embedded systems • MIS applications Drawbacks 1. Another paper found weak correlation between CFPs and cost for small projects 2. 2 papers state COSMIC method less effective for banking & scientific software

Research Questions Address these questions with UCC’s dataset: 1. Can COSMIC Function Points (CFPs) be used to effectively estimate small, maintenance tasks’ effort? 2. Can using COSMIC Function Points (CFPs) and the Software Non-functional Assessment Process (SNAP) together lead to effective effort estimates for effort estimates. ? • IFPUG developed SNAP to complement FPs for better estimates

Outline Motivatio n Dataset Metrics Analysis Approach Results Conclusio n

Unified Code Count (UCC) Project Description Project Types • Code metrics tool • Add New Functionality • New language parsers • Implemented in C++ • New features, such as GUI • 45 to 1425 logical SLOC • Modify Existing Functionality • 2010 to 2014 • Cyclomatic Complexity support (modify • Modularized architecture existing language parsers with mathematical operations and algorithms • 4 -month increments

UCC’s Project Types 450 Normalized Effort (hours) 400 Adding Functionality 350 Modifying Functionality 300 All Data Points 250 200 Adding Functionality Linear Trendline 150 Modifying Functionality Linear Trendline 100 All Data Points Linear Trendline 50 0 0 2 4 6 8 COSMIC FPs (CFPs) 10 12 14

Outline Motivatio n Dataset Metrics Analysis Approach Results Conclusio n

COSMIC Software Model

Calculate COSMIC Function Points (CFPs) Functional Processes Data Groups DG 1 …. . DG n FP 1 FP 2 …. FP n Software Totals Transactions Entry Exit Read Write Totals

IFPUG Software Model

Calculate IFPUG SNAP Complexity of Components Multiplier Factor Subcategory Low Average High Data Entry Validation 2 3 4 Logical Operations 4 6 10 Mathematical Operations 3 4 7 Data Formatting 2 3 5 Internal Data Movements 4 6 10 User Interface 2 3 4 Total Non-Functional Requirements Size Total

Outline Motivatio n Dataset Metrics Analysis Approach Results Conclusio n

Model Format based on COCOMO® II Model

Accuracy Measures • • • R 2: how closely the regression curve fits the data points MAR: Mean of Absolute Residuals Md. AR: Median of Absolute Residuals MMRE: Mean Magnitude of Relative Error Md. MRE: Median Magnitude of Relative Error PRED(25): Percentage of estimates within 25% of actuals

Leave-One-Out Cross Validation 1 n 2 n 1 … 17

Outline Motivatio n Dataset Metrics Analysis Approach Results Conclusio n

Adding New Functionality Research Question 1: Effort Estimation Effectiveness of COSMIC Function Points (CFPs)

Norm. Effort = 137. 718 + 16. 434 x CFP 1. 0997 450 Normalized Effort (hours) 400 350 300 250 200 150 100 50 0 0 2 4 6 8 COSMIC FPs (CFPs) 10 12 14 Sample 11 R 2 0. 516 MAR 51 hrs Md. AR 49 hrs MMRE 21% Md. MRE 15% PRED(25) 73%

Adding New Functionality Research Question 2: Effort Estimation Effectiveness of COSMIC Function Points (CFPs) and IFPUG SNAP together

Norm. Effort = 138. 542 + ((15. 538 x CFP) + (-0. 602 x SNAP))1. 0997 • Though SNAP accounts for mathematical operations and algorithmic complexity, grouping into complexity ranges may cause the ineffectiveness • 3 data points with relatively fewer mathematical operations/algorithms have unacceptably low prediction accuracy Sample 11 R 2 0. 709 MAR 57 hrs Md. AR 47 hrs MMRE 23% Md. MRE 15% PRED(25) 64%

Modifying Existing Functionality Research Question 1: Effort Estimation Effectiveness of COSMIC Function Points (CFPs)

Spearman’s Rho Correlation Test Normalized Effort (hours) 250 200 150 100 50 0 0 1 2 3 COSMIC FPs (CFPs) 4 5 6 Sample 19 Coefficient (Rho) 0. 298 p-value 0. 214 Coefficient of Determinatio n 0. 089

Modifying Existing Functionality Research Question 2: Effort Estimation Effectiveness of COSMIC Function Points (CFPs) and IFPUG SNAP together

Norm. Effort = 61. 159 + ((0. 922 x CFP) + (4. 089 x SNAP))1. 0997 • Only 1 project had estimate 40% from actual. • Smallest project • SNAP complexity range grouping sized it similar to other projects Sample 19 R 2 0. 709 MAR 19 hrs Md. AR 18 hrs MMRE 14% Md. MRE 15% PRED(25) 95%

Outline Motivatio n Dataset Metrics Analysis Approach Results Conclusio n

Threats to Validity • Dataset of convenience • Not student assignments • Product with users all over • All environments differ from one another in some way • Publicity of Dataset • Authors working to make public • Small project sizes • Complements existing studies • Use COCOMO® model for potential scalability • Statistical Significance • Other research paper indicates needing 10 data points

Summary of Results CFPs Effective for UCC’s Tasks • Adding New Functionality CFPs with SNAP Effective • Adding New Functionality • Unacceptable estimates for 3 • Unacceptable estimates for 4 projects with relatively less mathematical operations and algorithms • Modifying Functionality • Very weak correlation • Modifying Functionality • Acceptable prediction accuracy

Conclusions • COSMIC’s method addresses some drawbacks of IFPUG’s FPs • Applicability to software outside MIS domain • Limits to complexity size • COSMIC FPs, like IFPUG FPs, unable to size complexity stemming from mathematical operations and algorithms • Enhancing COSMIC’s method to size data manipulation can make it a more effective size metric