Evaluation of a Business Application Framework Using Complexity

Evaluation of a Business Application Framework Using Complexity and Functionality Metrics Hikaru Fujiwara†, Shinji Kusumoto†, Katsuro Inoue†, Toshifusa Ootsubo‡, Katsuhiko Yuura‡ †Graduate School of Engineering Science, Osaka University, Japan. ‡Business Solution Systems Division, Hitachi Ltd. This research is partially supported by International Information Science Foundation Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 1

Contents 1. Background 2. Approach 2. 1 Purpose 2. 2 Reuse 3. Evaluation Method 4. Case Study 4. 1 Case Study 1 4. 2 Case Study 2 5. Conclusion 2 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

1. Background (1/4) • It becomes important to develop high-quality software cost-effectively. • Reuse is one of the most famous techniques to attain it. • In object-oriented software development, developers reuse a particular library called framework. • A framework is a collection of classes that provide a set of services for a particular domain. 3 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

1. Background (2/4) • A department of Hitachi Ltd. develops application software for many local governments. • The developers have been using the original reuse technique based on a conventional module-based reuse. 4 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

1. Background (3/4) • 47 prefectures in Japan. • 3229 local governments, 671 cities, 1991 towns, 567 villages. • A department of Hitachi Ltd. develops application software for many local governments. • The requirement of the application is different depending on local ordinance. 5 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

1. Background (4/4) • The department is going to introduce the framework in order to improve the efficiency. • It is difficult to transfer the new framework to the development. • To motivate the developers, we need to show the benefit of using framework quantitatively. 6 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

Contents 1. Background 2. Approach 2. 1 Purpose 2. 2 Reuse 3. Evaluation Method 4. Case Study 4. 1 Case Study 1 4. 2 Case Study 2 5. Conclusion 7 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

2. Approach 2. 1 Purpose • We evaluate the usefulness of the framework quantitatively from two viewpoints. – saving cost – improving software quality • We conducted two case studies to evaluate it. – Application : Applications for local governments – Language : Java 8 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

2. 2 Reuse Conventional module-based reuse (1/2) • An application consists of a main module and some screen control modules. • The main module controls business logics and screen transitions of the application. • Each screen control module corresponds to one screen. A：Database renewal program Screen transition controller X 1：Data Inquiry Y 1：Showing the Inquiry Result Z 1：Updating the Record (a) Health insurance application for a local government A 9 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

2. 2 Reuse Conventional module-based reuse (2/2) A：Database renewal program Screen transition controller X 1：Data Inquiry Y 1：Showing the Inquiry Result Z 1：Updating the Record (a) Health insurance application for a local government A B：Database renewal program Screen transition controller X 2：Data Inquiry Y 2：Showing the Inquiry Result Z 2：Updating the Record (b) Health insurance application for a local government B 10 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

2. 2 Reuse Framework-based reuse • The framework is intended to reuse the processing of the transition of the screens in addition to module-based reuse. – Typical transitions of screens are prepared. • data inquiry, data renewal, data addition, data removal, etc. Database Renewal Program A B F 1： Framework Database renewal Framework Data Inquiry Showing the Inquiry Result Specific Parameters to a Local Government A B Updating the Record 11 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

Contents 1. Background 2. Approach 2. 1 Purpose 2. 2 Reuse 3. Evaluation Method 4. Case Study 4. 1 Case Study 1 4. 2 Case Study 2 5. Conclusion 12 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

3. Evaluation Method • We compare the framework-based reuse with the conventional module-based reuse from the viewpoints of cost and quality. (Case Study 1) Developing applications that have the same functions. framework-based reuse Specification of function fa Ca FW FW：framework Pa conventional reuse Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 13

3. Evaluation Method (Case Study 2) Adding functions to the existing application. Ca framework-based reuse Specification of function fa FW Ca+b FW Specification of function fb Pa Ca+b+c FW Specification of function fc Pa+b+c conventional reuse 14 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

3. Evaluation Method Metrics used in the case studies • We could not collect the actual effort and the number of faults. • We used following metrics to indirectly evaluate the productivity and quality. – OOFP (Object-Oriented Function Point) measures functionality. productivity. – C&K metrics (Chidamber and Kemerer’s metrics) measure complexity. quality. 15 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

3. Evaluation Method OOFP（Object Oriented Function Points§) • OOFP is an adaptation of FP(Function Point) to enable the measurement of object-oriented analysis and design specifications. – FP is measured from logical files(Internal Logical Files: ILFs, External Interface Files: EIFs) and transactions (inputs, outputs, inquiries). – OOFP is measured from logical files (ILF, EIF) and transactions (Service Requests: SRs). • Classes correspond to logical files • Methods correspond to transactions §: G. Caldiera, G. Antoniol, R. Fiutem, C. Lokan, “Definition and Experimental Evaluation of Function Points for Object-Oriented Systems”, IEEE, Proc. of METRICS 98, pp. 167 -178 (1998). Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 16

3. Evaluation Method C&K metrics (Chidamber and Kemerer’s metrics) viewpoints metrics how to calculate metrics of a class C inheritance DIT the depth of C in the inheritance tree NOC the number of immediate sub-classes subordinated to C CBO the number of couplings between C and any other class RFC (the number of the methods in C) ＋(the number of the methods called by C) WMC the sum of the complexity of the methods in C LCOM Assume that methods M 1. . . Mn ∈ C, Ii is a set of instance variables used in Mi, (the number of pairs (Mk, Ml) such that Ik∩Il＝φ) － (the number of pairs (Mk, Ml) such that Ik∩Il≠φ) coupling method 17 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

Contents 1. Background 2. Approach 2. 1 Purpose 2. 2 Reuse 3. Evaluation Method 4. Case Study 4. 1 Case Study 1 4. 2 Case Study 2 5. Conclusion 18 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

4. Case Study 4. 1 Case Study 1 • Four applications Ca, Cb, Cc, Cd are developed using the framework-based reuse. • Four applications Pa, Pb, Pc, Pd are developed using the conventional reuse. • Ci and Pi (i = a, b, c, d) implement the same function fi. function fa fb fc framework-based Ca Cb Cc conventional Pa Pb Pc fd Cd Pd 19 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

4. 1 Case Study 1 • We compared Ci with Pi from the viewpoints of productivity and quality. • We measured OOFP and C&K metrics from newly developed part of each application. framework-based reuse Ca FW Specification of function fa FW：framework Pa conventional reuse FW：framework Software Engineering Research Group, Graduate School of Engineering Science, Osaka University : newly developed 20

4. 1 Case Study 1 Result of case study 1 number of classes OOFP CBO RFC WMC LCOM Ca Cb Cc Cd 5 5 11 8 176 180 418 252 3. 8 5. 4 4. 1 14. 4 18. 2 33. 1 17. 8 7. 4 7. 6 8. 1 6. 4 21. 4 23. 2 28. 7 13. 8 Pa Pb Pc Pd 25 25 29 28 526 671 672 2. 1 2. 3 3. 0 2. 4 5. 8 5. 9 7. 4 8. 6 3. 3 3. 4 4. 3 2. 1 2. 0 15. 7 C&K metrics are the average values per class 21 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

4. 1 Case Study 1 Analysis of OOFP frameworkbased (C) conventional (P) function fa 176 526 fb 180 526 fc 418 671 fd 252 672 OOFP(Ci) < OOFP(Pi) Application development using framework reduces the effort of development. 22 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

4. 1 Case Study 1 Analysis of OOFP • To develop the framework, initial investment (effort) is needed. • The OOFP of the framework FW is 1298. • The framework-based reuse is about 2. 5 times more effective than the conventional reuse. • The department develops the similar applications repeatedly. • It will save the effort after three or four applications have been developed, whereas the investment for the framework was spent. 23 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

4. 1 Case Study 1 Analysis of CBO and RFC average CBO function fa fb fc fd average RFC frameworkbased (C) conventional (P) 3. 8 5. 4 4. 1 2. 3 3. 0 2. 4 14. 4 18. 2 33. 1 17. 8 5. 9 7. 4 8. 6 CBO(Ci) > CBO(Pi), RFC(Ci) > RFC(Pi) All of the methods called by Ci are included in the framework. If the framework is high quality, the complexity does not affect the quality of the overall application program. 24 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

4. 1 Case Study 1 Analysis of WMC and LCOM average WMC function fa fb fc fd average LCOM frameworkbased (C) conventional (P) 7. 4 7. 6 8. 1 6. 4 3. 3 3. 4 4. 3 21. 4 23. 2 28. 7 13. 8 2. 1 2. 0 15. 7 WMC(Ci) > WMC(Pi), LCOM(Ci) > LCOM(Pi) There are many simple methods, that set/get the values of the attribute. The complexity does not affect the quality of the overall application program. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 25

Contents 1. Background 2. Approach 2. 1 Purpose 2. 2 Reuse 3. Evaluation Method 4. Case Study 4. 1 Case Study 1 4. 2 Case Study 2 5. Conclusion 26 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

4. 2 Case Study 2 • At first, application Ca and Pa are developed. • Function fb, fc, fd are continuously added to Ca and Pa. • Ci and Pi (i = a, a+b+c, a+b+c+d) implement the same function fi. frameworkbased conventional function fa Ca Pa fa+b+c+d Ca+b+c+d Pa+b+c+d 27 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

4. 2 Case Study 2 • We compared the differences between the two successive applications from the viewpoint of productivity and quality. • We measured OOFP and C&K metrics from newly developed part of each application. Ca framework-based reuse Specification of function fa FW Ca+b FW Specification of function fb Pa Ca+b+c FW Specification of function fc Pa+b conventional reuse Software Engineering Research Group, Graduate School of Engineering Science, Osaka University Pa+b+c 28

4. 2 Case Study 2 Result of case study 2 number of classes OOFP CBO RFC WMC LCOM Ca Ca+b+c+d 5 7 15 20 176 251 578 743 3. 8 5. 0 5. 9 5. 8 14. 4 16. 7 30. 1 28. 3 7. 4 7. 6 8. 3 7. 9 21. 4 20. 1 28. 6 25. 6 Pa Pa+b+c+d 25 29 39 46 526 615 849 1084 2. 1 2. 8 3. 7 4. 0 5. 8 6. 9 8. 6 10. 5 3. 3 3. 9 2. 1 2. 0 1. 8 10. 0 C&K metrics are the average values per class Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 29

4. 2 Case Study 2 Analysis of OOFP fa function fa+b+c+d frameworkconventional based (C) (P) 176 526 75 89 251 615 327 234 578 849 165 235 743 1084 OOFP(Ca+b+c - Ca+b) > OOFP(Pa+b+c - Pa+b) The adaptability of the framework to the function fc is not good. It is necessary to add new components for the functions like fc to the framework. 30 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

4. 2 Case Study 2 Analysis of OOFP fa frameworkconventional based (C) (P) 176 526 function fa+b 251 615 fa+b+c 578 849 fa+b+c+d 743 1084 OOFP(Ca+b+c+d) < OOFP(Pa+b+c+d) The framework-based reuse is more effective to reduce the effort of development than the conventional reuse. 31 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

4. 2 Case Study 2 Analysis of CBO and RFC average of CBO frameworkbased (C) fa function fa+b+c+d 1. 2 0. 9 -0. 1 3. 8 5. 0 5. 9 5. 8 average of RFC conventional (P) 0. 9 0. 3 frameworkbased (C) 2. 1 2. 3 2. 8 13. 4 3. 7 4. 0 -1. 8 conventional (P) 14. 4 1. 1 16. 7 1. 7 30. 1 28. 3 1. 9 5. 8 6. 9 8. 6 10. 5 RFC(Ca+b+c - Ca+b) > RFC(Pa+b+c - Pa+b) Ca+b+c calls a lot of methods included in the framework, in order to handle many data items in the function fc. If the framework is high quality, the complexity does not affect the overall application program. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 32

Contents 1. Background 2. Approach 2. 1 Purpose 2. 2 Reuse 3. Evaluation Method 4. Case Study 4. 1 Case Study 1 4. 2 Case Study 2 5. Conclusion 33 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

5. Conclusions • We have experimentally evaluated the usefulness of the framework quantitatively from the viewpoints of quality and saving cost. • As the result of the case studies, the framework-based reuse is more effective than the conventional reuse. • In order to show the usefulness of the framework, we are going to apply the framework to many software development projects in future. 34 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

35 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

Appendix How to measure OOFP(1) OOFP=OOFPILF+OOFPEIF+OOFPSR • For each ILF, OOFPILF is measured using DETs (Data Element Types) and RETs (Record Element Types). • For each EIF, OOFPEIF is measured using DETs and RETs. • For each SR, OOFPSR is measured using DETs and FTRs (File Types Referenced). • Finally, OOFP is calculated by summing up OOFPILF, OOFPEIF and OOFPSR. 36 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

Appendix How to measure OOFP (2) • To calculate OOFP from Java source code, we suppose that the concepts in Java correspond to ones in OOFP as follows. OOFP Logical files Java ILFs Classes within the application. EIFs Classes outside of the application. DETs Number of simple attributes (such as integers, strings) RETs Number of complex attributes (such as objects) Transactions SRs Methods within the application. DETs Number of simple arguments, instance variables and class variables. FTRs Number of complex arguments, instance variables, class variables and objects. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 37

Appendix Case Study 2; Analysis of WMC and LCOM (1) average of WMC frameworkbased (C) function fa fa+b+c 0. 2 0. 7 fa+b+c+d -0. 4 average of LCOM conventional (P) 7. 4 0. 0 7. 6 0. 0 8. 3 7. 9 0. 6 frameworkbased (C) 3. 3 -1. 3 3. 3 8. 5 3. 3 3. 9 -3. 0 conventional (P) 21. 4 -0. 1 20. 1 -0. 2 28. 6 25. 6 8. 2 2. 1 2. 0 1. 8 10. 0 LCOM(Ca+b+c - Ca+b) > LCOM(Pa+b+c - Pa+b) There a lot of set/get methods in Ca+b+c , in order to handle many data items in the function fc. 38 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

Appendix Case Study 2; Analysis of WMC and LCOM (2) average of WMC frameworkbased (C) function fa fa+b+c 0. 2 0. 7 fa+b+c+d -0. 4 average of LCOM conventional (P) 7. 4 0. 0 7. 6 0. 0 8. 3 7. 9 0. 6 frameworkbased (C) 3. 3 -1. 3 3. 3 8. 5 3. 3 3. 9 -3. 0 conventional (P) 21. 4 -0. 1 20. 1 -0. 2 28. 6 25. 6 8. 2 2. 1 2. 0 1. 8 10. 0 LCOM(Ca+b+c+d - Ca+b+c) < LCOM(Pa+b+c+d - Pa+b+c) There are complex transitions of the screens in function fd. In Pa+b+c+d , the LCOM values of the classes implement these transactions are high. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 39

Appendix Case Study 2; Analysis of WMC and LCOM (3) average of WMC frameworkbased (C) average of LCOM conventional (P) frameworkbased (C) conventional (P) function fa 7. 4 3. 3 21. 4 2. 1 fa+b+c+d 7. 6 8. 3 7. 9 3. 3 3. 9 20. 1 28. 6 25. 6 2. 0 1. 8 10. 0 WMC(Ci) > WMC(Pi), LCOM(Ci) > LCOM(Pi) We examined the classes of Ci that have high WMC and LCOM values. It is found that there are many simple methods, that set/get the values of the attribute. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 40

Appendix Process of Case Study • Development of the framework • Case Study 1 – Two developers – Develop Ca, Cb, Cc, Cd – Develop Pa, Pb, Pc, Pd • Case Study 2 – Two developers – Develop Ca, Ca+b+c, Ca+b+c+d – Develop Pa, Pa+b+c, Pa+b+c+d • Apply the metrics to the applications. • Analysis 41 Software Engineering Research Group, Graduate School of Engineering Science, Osaka University