AN ONTOLOGY TO SUPPORT TMMIBASED TEST PROCESS ASSESSMENT

AN ONTOLOGY TO SUPPORT TMMI-BASED TEST PROCESS ASSESSMENT Eda Gülçin Çiflikli ICterra Information and Communication Technologies Ankara, Turkey Ahmet Coşkunçay Department of Computer Engineering, Ataturk University Erzurum, Turkey

Outline ■ Introduction ■ Background ■ Related Works ■ Structure of the TMMI Assessment Ontology ■ An Example Usage of the TMMI Assessment Ontology ■ Conclusion

Introduction ■ Software process reference models: – General purpose: ■ SPICE, ■ CMMI … – Process specific: ■ SMmm (for maintenance), ■ R-CMM (for requirements engineering), ■ MCM (for measurement), ■ SA-CMM (for acquisition) … – Software testing: ■ Test Maturity Model integration (TMMi), ■ Test Process Improvement (TPI) Model, ■ Metrics Based Verification and Validation Maturity Model (MB-V 2 M 2) … ■ Reference models: mostly textual -> understandability and ambiguity issues, engineers like graphical representations more ■ Tools for process assessment: lack interoperation -> lack benchmark, knowledge sharing

Introduction ■ An ontology would help in: – establishing a shared understanding of the reference models, – reusing the assessment outputs and outcomes, – enabling interoperability between tools supporting process improvement ■ An ontology to support TMMI-based assessment is developed that: – creates a mapping infrastructure between TMMi model and outputs generated by an organization as a consequence of implementing and assessing software testing processes – identifies and formally defines the concepts and their relationships that are necessary to perform a TMMi-based assessment – presents TMMi assessment domain knowledge visually

TMMI ■ Short form of Test Maturity Model Integration ■ Developed by TMMi Foundation ■ A complementary reference model to CMMI ■ Addresses to software quality and test issues to support organizations with evaluating and improving their test process ■ Described as text in a natural language ■ Inherits the concepts such as maturity level, process area, goal and practice

TMMI (Cont’d) Maturity Level Process Areas Defect Prevention (5) Optimization Test Process Optimization Quality Control Test Measurement (4) Measured Software Quality Evaluation Advanced Peer Reviews Test Organization Test Training Program (3) Defined Test Lifecycle and Integration Non-functional Testing Peer Reviews Test Policy and Strategy Test Planning (2) Managed Test Monitoring and Control Test Design and Execution Test Environment (1) Initial - ■ Has a staged architecture ■ Highest objective for an organization to achieve is the maturity level 5 ■ In order to achieve one maturity level, an organization should satisfy all process areas of that maturity level and the lower maturity levels.

TMMI Assessment ■ Three possible formal assessment methods: – TMMi Accredited Assessment Method which is developed by a TMMi accredited service provider – TMMi Assessment Method (TAM), developed and provided by the TMMi Foundation – The Standard CMMI Appraisal Method for Process Improvement (SCAMPI) A ■ SCAMPI-A Method is selected: – Commonly used – Familiarity

SCAMPI-A ■ Essentially designed for CMMI ■ Considered a valid method to perform TMMi assessments ■ Provides benchmark-quality ratings – To enable identifying the strengths and weaknesses of an organization – To prioritize improvement plans – To derive maturity level ratings – To identify risks relative to maturity determinations ■ used to decide practice implementation and goal ratings based on collected objective evidence: – Artifacts – Affirmations ■ Goals are characterized as Fully Implemented, Largely Implemented, Partially Implemented, Not Implemented or Not Yet.

Ontology ■ Formal explicit description of concepts, properties and interrelations in a specific domain ■ Ontologies: – help to clarify the structure of the knowledge by machine-interpretable common vocabulary definitions – enable knowledge sharing – build specific knowledge basis to make domain assumptions explicit

Related Works • Liao et al. (2005) • an OWL-based ontology for software processes: Software Process Ontology (SPO) • presented how to extend SPO by using continuous representation of CMMI and ISO/IEC 15504 • a web based process assessment tool is under development • Soydan et al. (2006) • CMMI-SW ontology • staged representation of CMMI-SW model defined in OWL • Sharifloo et al. (2008) • Ontology for CMMI-ACQ model • Implemented based on SUMO (an upper ontology)

Related Works • Ryu et al. (2008) • OWL ontology for MND-TMM (Ministry of National Defense-Testing Maturity Model) • top level concepts of MND-TMM defined • Purpose: share knowledge and help organizations improve their testing processes • under development… • Gazel et al. (2012) • OWL ontology for both continuous and staged representations of CMMI -Dev • an Eclipse Plugin for CMMI assessment • Purpose: present an integrated infrastructure for process improvement and CMMI based assessment activities

■ Structure of TMMI Assessment Ontology Composed of three separate onologies ■ Web Ontology Language (OWL) and Protégé 5. 1. 0 platform is used ■ Reuses existing ontologies in several parts

TMMI Ontology Includes: ■ 4 top level classes ■ 3 object properties

TMMI Ontology (Cont’d) ■ Higher maturity levels are defined as a subclass of lower maturity levels to ensure staged foundation of TMMI

TMMI Ontology (Cont’d) ■ Process_Area class consist of mid-level subclasses to categorize process areas according to maturity level they take place

TMMI Ontology (Cont’d) ■ The targeted maturity level determine which generic goals applicable therefore generic goal 3 is defined as a subclass of generic goal 2

TMMI Ontology (Cont’d) ■ Each level in the Goal class hierarchy has more specific “achieved. By” relationship with the related class in the Practice class hierarchy

SCAMPI-A Ontology Includes: ■ 6 top level classes ■ 4 object properties

SCAMPI-A Ontology (Cont’d) In SCAMPI-A method each process generates assessment outputs, however the same assessment output may be revised and updated in the next process. Therefore Assessment_Output class is grouped according to phases.

Organization Ontology ■ Includes 3 top level classes and 2 object properties ■ Each organization, project or work product can be defined as an individual to the related class

TMMI Assessment Ontology

An Example ■ Presents: – Goal ratings for each project – Work products identified in each goal – … ■ Can be extended by: – More projects – More organizations Meaning: a linked structure of assessment data

Conclusion ■ TMMi Assessment Ontology to support TMMi-based assessments ■ An ontology with a modular structure (3 ontologies that can be reused: TMMi, SCAMPI-A, organization) ■ A linked data environment for assessments: – can be extended with assessment data from other projects and organizations – benefit from data search, display, inference, storage ■ Possibilities: – Visually represent TMMi and SCAMPI-A domain knowledge – Improves understandability of TMMi and SCAMPI for both people and machines – Allows semantic interoperability between assessment tools (exchange information correctly and completely) – Can be used in benchmarking ■ “Plan and Prepare for the Appraisal” and “Report Result” phases of SCAMPI-A may be a subject to another project in order to make the infrastructure more usable for assessors.

Thanks! Ahmet Coşkunçay ahmet. coskuncay@gmail. com Department of Computer Engineering, Ataturk University Erzurum, Turkey

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. Gibson, D. L. , Goldenson, D. R. , Kost, K. : Performance Results of CMMI-Based Process Improvement. Software Engineering Institute, Technical report CMU/SEI-2006 -TR-004 (2006) April, A. , Huffman Hayes, J. , Abran, A. , Dumke, R. : Software maintenance maturity model (SMmm). J. Softw. : Evol. Process. 17(3), 197– 223 (2005) Beecham, S. , Hall, T. , Rainer, A. : Defining a requirements process improvement model. Softw. Qual. J. 13(3), 247– 279 (2005) Salmanoglu, M. , Coskuncay, A. , Yildiz, A. , Demirors, O. : An exploratory case study for assessing the measurement capability of an agile organization. Softw. Qual. Prof. Mag. 20(2), 36– 47 (2018) Cooper, J. , Fisher, M. , Sherer, S. W. : Software Acquisition Capability Maturity Model (SA- CMM) Version 1. 02 (No. CMU/SEI-99 -TR-002). Software Engineering Institute, Carnegie Mellon University, Pittsburgh (1999) Farooq, A. , Dumke, R. R. : Developing and applying a consolidated evaluation framework to analyze test process improvement approaches. In: Cuadrado. Gallego, J. J. , Braungarten, R. , Dumke, Reiner R. , Abran, A. (eds. ) IWSM/Mensura -2007. LNCS, vol. 4895, pp. 114– 128. Springer, Heidelberg (2008). https: //doi. org/10. 1007/978 -3 -540 -85553 -8_10 Tarhan, A. , Giray, G. : On the use of ontologies in software process assessment: a systematic literature review. In: 21 st International Conference on Evaluation and Assessment in Software Engineering (2017) Noy, N. F. , Mc. Guinness D. L. : Ontology Development 101: A Guide to Creating Your First Ontology. Stanford University (2001) Veenendaal, E. : Test Maturity Model Integration (TMMi) Release 1. 0. TMMi Foundation. https: //www. tmmi. org/wpcontent/uploads/2016/09/TMMi. Framework. pdf TMMi Assessment Options. https: //www. tmmi. org/assessment-options/ Software Engineering Institute: Standard CMMI Appraisal Method for Process Improvement (SCAMPI) A, Version 1. 3: Method Definition Document (2011) Chandrasekaran, B. , Josephson, J. R. , Benjamins, V. R. : What are ontologies, and why do we need them? IEEE Intell. Syst. 14, 20– 26 (1999) Kalibatiene, D. , Vasilecas, O. : Survey on ontology languages. In: Grabis, J. , Kirikova, M. (eds. ) BIR 2011. LNBIP, vol. 90, pp. 124– 141. Springer, Heidelberg (2011). https: //doi. org/10. 1007/978 -3 -642 -24511 -4_10 Ryu, H. , Ryu, D. , Baik, J. : A strategic test process improvement approach using an ontological description for MND-TMM. IEEE (2008) Gazel, S. , Tarhan, A. , Sezer, E. : An ontology based infrastructure to support CMMI-based software process assessment. Gazi Univ. J. Sci. 25, 155– 164 (2012) Soydan, G. H. , Kokar, M. M. : An OWL ontology for representing the CMMI-SW model. In: Workshop on Semantic Web Enabled Software Engineering (2006)