6132021 CMM Overview Professor Ron Kenett Tel Aviv

6/13/2021 CMM Overview Professor Ron Kenett Tel Aviv University School of Engineering 1

6/13/2021 : יעדים טיפוסיים l Faster time to market l Improve quality l Increase customer satisfaction l Enhance productivity l Develop right products in required time windows 2

6/13/2021 תהליך השיפור P 1: Assessment P 2: Projects Launch P 3: Process analysis P 5: Implementation and on going control P 4: Management Dashboards 7

6/13/2021 ESPRIT European Systems and Software Initiative Israeli Process Improvement Experiments p IAI p Magic p Motorola p Onyx p ECI 23683 TESTART 23690 MAGICIAN 23705 PREV-DEV 23750 SPIP 27582 ARMT 8

6/13/2021 ESPRIT European Systems and Software Initiative Best Practice Manual Israel ESPINODE מרכז הידע לשיפור תהליכי פיתוח תוכנה מדריך לשיטות מתקדמות בפיתוח תוכנה KPA בע"מ Email: espinode@kpa. co. il http: //www. kpa. co. il/espinode 9

6/13/2021 The Software Engineering Institute Capability Maturity Model The Capability Maturity Model, Version 1. 1 CMU/SEI-93 -TR-24, Software Engineering Institute Carnegie Mellon University, Pittsburgh, 1993 Watts S. Humphrey, Managing the Software Process Addison-Wesley, Reading Mass. , 1989 10

6/13/2021 SEI Capability Maturity Model 5. OPTIMIZING Continuous Improvement 4. MANAGED Process Control 3. DEFINED Process Definition 2. REPEATABLE Process Discipline 1. INITIAL Ad Hoc Levels of Process Maturity 11

6/13/2021 Level 1: Just do it. Level 2: Think before you act, and think after you act, just to make sure you did it right. Activity to produce Results Planning Activity to produce Results input to Evaluation to improve 12

6/13/2021 Level 3 input to Standards Planning Activity input to to produce Results input to Evaluation to improve Use your lessons learned. 13

6/13/2021 Level 4 input to Standards Planning Activity input to to forecast to produce Results input to Evaluation to improve Predict the results you need and expect and then create opportunities to get those results 14

6/13/2021 Level 5 input to Standards Planning Activity input to to forecast to produce Results input to Evaluation to improve Create lessons learned, and use lessons learned to create more lessons learned, and use more lessons learned to create even more lessons learned, and use even more lessons learned to create. . . 15

6/13/2021 SEI Capability Maturity Model השפעה על שקיפות הפיתוח והיכלות לתכנן ולהתריע 5. OPTIMIZING Continuous Improvement 4. MANAGED Process Control 3. DEFINED Process Definition 2. REPEATABLE Process Discipline 1. INITIAL Ad Hoc 16

6/13/2021 The process is a "black box" at Level 1 IN OUT 17

6/13/2021 Milestones provide visibility at Level 2 IN OUT 18

6/13/2021 Internal processes become visible at Level 3 IN OUT 19

6/13/2021 At Level 4 visibility is quantitative, objective IN OUT 20

6/13/2021 High degrees of insight at Level 5 IN OUT 21

6/13/2021 SEI Capability Maturity Model השפעה על עמידה ויעדים ועלויות פיתוח 5. OPTIMIZING Continuous Improvement 4. MANAGED Process Control 3. DEFINED Process Definition 2. REPEATABLE Process Discipline 1. INITIAL Ad Hoc 22

6/13/2021 Targets usually missed at Level 1 Target Time/$/. . . 23

6/13/2021 Targets more often met at Level 2 Target Time/$/. . . • Plans are more realistic • Results still vary widely 24

6/13/2021 Performance improves at Level 3 Target Time/$/. . . • Variability decreases • Actual results improve 25

6/13/2021 Process under statistical control at Level 4 Target Time/$/. . . • Results highly predictable • Variability narrows significantly 26

6/13/2021 Level 5: a platform for ongoing improvement Target Level 5 Time/$/. . . • Defect prevention and continuous improvement provide ongoing performance gains 27

6/13/2021 CMM structure overview Maturity Level Process Capability Key Process Areas Goals Key Practices 28

6/13/2021 The CMM comprises 18 KPAs Level Key Process Areas Optimizing Process Change Management Defect Prevention Technology Change Management Managed Software Quality Management Quantitative Process Management Defined Organization Process Focus Organization Process Definition Training Program Integrated Software Management Software Product Engineering Intergroup Coordination Peer Reviews Repeatable Requirements Management Project Planning Project Tracking & Oversight Subcontract Management Quality Assurance Configuration Management Initial 29

6/13/2021 The CMM Structure Maturity Levels Key Process Areas RM PP 3 2 PT SM QA 4 5 CM Goals Common Features Activities Performed Commitment to Perform Ability to Perform Measurement and Analysis Verifying Implementation Key Practices 30

6/13/2021 Quality increases and risk decreases as the process matures Level Characteristic Optimizing Continuous process capability improvement Managed Product quality planning; tracking of measured software processes Defined Software process defined and institutionalized to provide product quality control Repeatable Management oversight and tracking of projects; stable planning and product baselines Initial Impact QUALITY RISK Ad hoc (unpredictable, chaotic) 31

6/13/2021 Understanding the Managed and Optimizing Levels Level 3 "Stability" Level 4 "Control" Level 5 "Continuous Improvement" Upper limit Cost of Poor Quality Zone of Performance ? Lower Limit Chronic Waste 32

6/13/2021 A Software Process Immaturity Model Level 0 Foolish Characteristic Key Process Area Negligent: • Software reuse Failure to allow successful development process -1 Stupid Obstructive: counter-productive process imposed -2 Lunatic Contemptuous: disregard for good software engineering institutionalized Result Sense • Development environments • Repositories • Automatic programming Idiocy Source: A. Finkelstein, ACM SIGSOFT Software Engineering Notes, Oct 1992 33

6/13/2021 Process appraisal steps Form & Train Team Findings Software Quality Assurance Maturity Questionnaire Finding Few standards and procedures for process and products have been formally adopted § There are no independent means to assure that defined standards and procedures are followed Consequences Quality gains are not sustainable § SDM is inconsistently applied § Potential for conflict between groups § Quality is at risk § Defined processes may be abandoned § Frequent emergency releases § § Collect Data Interviews • Discussions • Document • Reviews KPA Profile Process Change Management Technology Change Management Defect Prevention Software Quality Management Quantitative Process Management Peer Reviews Intergroup Coordination Software Product Engineering Integrated Software Management Training Program Organization Process Definition Organization Process Focus Subcontract Management Requirements Management Analyze Data & Formulate Findings Quality Assurance Configuration Management Project Tracking & Monitoring Project Planning 34

6/13/2021 CMM-based process profile Process Change Management Technology Change Management Defect Prevention Software Quality Management Quantitative Process Management Peer Reviews Intergroup Coordination Software Product Engineering Integrated Software Management Training Program Organization Process Definition Organization Process Focus Subcontract Management Requirements Management Legend Quality Assurance Not satisfied Configuration Management Partially satisfied Project Tracking & Monitoring Fully satisfied Project Planning Not applicable 35

6/13/2021 36

6/13/2021 סטטיסטיקה מתוך אבחונים פורמליים Source: SEI, 1994, number of organizations: 261 1997, number of organizations 606 37

6/13/2021 Examples from data reported in http: //www. sei. cmu. edu/technology/measurement/profile. html p AT&T Network Systems Advanced Software Construction Center u u p Maturity Level-2 May-95 Lockheed Martin Aircraft Services Division u u Maturity Level-2 Oct 95 Motorola Transmission Products Group u Hewlett Packard u p Maturity Level-2 Aug-95 p Systems and Electronics Group at Texas Instruments u u p Maturity Level-2 Sep-93 Maturity Level-3 Jun-94 CITIL (Citicorp Information Technology Industries Limited) u u Maturity Level-4 Aug-95 38

6/13/2021 More examples from data reported in http: //www. sei. cmu. edu/technology/measurement/profile. html p Mc. Donnell Douglas Aerospace (St. Louis) u u p Raytheon Electronic Systems u u p Level-3 Aug Maturity -94 Maturity Level-3 1991 IBM Federal Systems Company u u Maturity Level-5 1989 p Tracor, Inc. u u p Boeing Defense & Space Group u u p Maturity Level-3 September 1996 Maturity Level-5 Aug-96 Motorola Electronic India Ltd. (MEIL) u u Maturity Level-5 Aug-96 39

6/13/2021 CMM structure overview Maturity Level Process Capability Key Process Areas Goals Key Practices 40

6/13/2021 Establishing a repeatable process • Requirements Management REPEATABLE • Project Planning • Project Tracking & Oversight • Subcontract Management INITIAL • Quality Assurance • Configuration Management 41

6/13/2021 CMM Level 2 Key Process Areas Requirements Management Software Quality Assurance Software Project Planning Software Configuration Management Software Project Tracking and Oversight Software Subcontract Management 42

6/13/2021 Requirements Management p Purpose: u p Involves: u p To establish a common understanding between the customer and the software project of the customer's requirements that will be addressed by the software project. Establishing and maintaining an agreement with the customer on the requirements for the software project Forms the basis for estimating, planning, performing, and managing the project 43

6/13/2021 Selected key practices Requirements Management Commitment to Perform: p An organizational policy specifies that: Requirements are documented. u Requirements are reviewed by the software manager and other affected groups. u Plans, products, and activities are changed when the requirements change. u 44

6/13/2021 Requirements Management Activities Performed Goal 1: System requirements allocated to software controlled to establish a baseline for software engineering and management use. p The software engineering group reviews the allocated requirements before they are incorporated into the software project. Incomplete and missing allocated requirements are identified. u Commitments resulting from the allocated requirements are negotiated with the affected groups. u p The allocated requirements are managed and controlled. 45

6/13/2021 Requirements Management Activities Performed Goal 2: Software plans, products, and activities are kept consistent with the system requirements allocated to software. The software engineering group uses the allocated requirements as the basis for plans, work products, and activities. p Changes to the allocated requirements are reviewed and incorporated into the project. p u p The impact to existing commitments is assessed, and changes are negotiated as appropriate. Changes to plans, products, and activities are identified, documented, managed, and tracked. 46

6/13/2021 Software Project Planning p Purpose: u p To establish reasonable plans for performing the software engineering and for managing the software project. Involves: Developing estimates u Negotiating commitments u Establishing the plan u 47

6/13/2021 Selected key practices Software Project Planning Commitment to Perform: p A software project manager is designated to negotiate commitments and develop a plan. p An organizational policy specifies that: u u u requirements are used as a basis for planning commitments are negotiated. other groups’ involvement is negotiated and documented. affected groups review estimates and commitments. senior management reviews external commitments. development plan is managed and controlled. Ability to Perform: p Documented and approved statement of work exists. 48

6/13/2021 Software Project Planning Goal 1: Software estimates are documented for use in planning and tracking the software project. Activities Performed Documented procedures are used to derive estimates for product size (and size changes), effort and costs, critical computer resources, and schedule. p Estimated capacity requirements for the software facilities, environments, and tools are based on size estimates and other characteristics. p Planning data are recorded p 49

6/13/2021 Software Project Planning Goal 2: Software project activities and commitments are planned and documented. Activities Performed p p p Software project planning is initiated in the early stages of, and in parallel with, the overall project planning. A software life cycle with stages of manageable size is defined. A software development plan is developed according to a documented procedure. The SDP is documented, reviewed, managed, and controlled(. Software work products that help maintain control of the software project are identified. 50

6/13/2021 Software Project Planning Goal 2: Software project activities and commitments are planned and documented. Activities Performed Cont’d Software technical, cost, resource, and schedule risks are identified, assessed, and documented. p Plans for software engineering facilities and support tools are prepared. p 51

6/13/2021 Software Project Planning Goal 3: Affected groups and individuals agree to their commitments related to the software project Activities Performed The software group participates on the project proposal team. p The software group participates with other affected groups in overall project planning. p External software commitments are reviewed with senior management according to a documented procedure. p Plans for the involvement of the software group with related groups, and vice versa, are negotiated, the support efforts are budgeted, and agreements documented. p 52

6/13/2021 Project Tracking & Oversight p Purpose: u p To provide adequate visibility into actual progress so that management can take effective actions when the software project's performance deviates significantly from the software plans. Involves: Monitoring project results u Comparing results to estimates and plans u Taking corrective actions u 53

6/13/2021 Selected key practices Project Tracking & Oversight Commitment to Perform: p A software project manager designated to be responsible for project’s activities and results. p An organizational policy specifies that: u u u a documented SDP is used and maintained as the basis for tracking the project manager is kept informed of the software project’s status and issues corrective actions are taken when the plan is not being achieved changes to the commitments are made with the agreement of the affected groups senior management reviews all new external commitments and changes 54

6/13/2021 Software Project Tracking & Oversight Goal 1: Actual results and performances are tracked against the software plans. Activities Performed A software development plan is used for tracking activities and communicating status. p Results are tracked: p u u u size of software products software effort and costs critical computer resources software schedule software engineering technical activities software risks more. . . 55

6/13/2021 Software Project Tracking & Oversight Goal 1: Actual results and performances are tracked against the software plans. Activities Performed Cont’d Actual measurement data are recorded. p SW group conducts regular internal reviews to track performance against the SDP. p Formal reviews of software results are conducted at milestones according to documented procedure. p 56

6/13/2021 Software Project Tracking & Oversight Activities Performed Goal 2: Corrective actions are taken and managed to closure when actual results and performance deviate significantly from the software plans. The SDP is revised according to documented procedure. p Corrective actions, determined through tracking software items, are taken as necessary. p Replanning data are recorded. p 57

6/13/2021 Software Project Tracking & Oversight Goal 3: Changes to software commitments are agreed to by the affected groups and individuals. Activities Performed External software commitments and changes to them are reviewed with senior management according to a documents procedure. p Approved changes to software commitments are communicated to the software group and other related groups. p 58

6/13/2021 Software Quality Assurance p Purpose: u p To provide management with appropriate visibility into the process being used by the software project and of the products being built. Involves: Reviewing and auditing products and activities to determine whether the project is adhering to established plans, standards, and procedures u Providing the results to the project manager and other managers, as appropriate u 59

6/13/2021 Selected key practices Software Quality Assurance Commitment to Perform: p An organizational policy specifies that: the SQA function is in place for all projects u the SQA group has an independent reporting channel to senior management u senior management periodically reviews SQA activities and results u 60

6/13/2021 Software Quality Assurance Goal 1: Software quality assurance activities are planned. Activities Performed An SQA plan is prepared for the software project according to a documented procedure. p The SQA plan is reviewed by the affected groups and individuals. p The SQA plan covers: p responsibilities and authority of the SQA group u evaluations, reviews, and audits to be conducted u 61

6/13/2021 Software Quality Assurance Activities Performed Goal 2: Adherence of software products and activities to the applicable standards, procedures, and requirements is verified objectively. The SQA group’s activities are performed in accordance with the SQA plan. p The SQA group participates in the preparation and review of the project’s SDP, standards, and procedures. p The SQA group reviews the software engineering activities to verify compliance. p The SQA group audits the designated software work products. p 62

6/13/2021 Software Quality Assurance Goal 3: Affected groups and individuals are informed of software quality assurance activities and results. Activities Performed The SQA group periodically reports the results of its activities to the software engineering group. p Deviations identified in the software activities and software work products are documented and handled according to a documented procedure. p The SQA group conducts periodic reviews of its activities and findings with the customer’s SQA personnel, as appropriate. p 63

6/13/2021 Software Quality Assurance Activities Performed Goal 4: Noncompliance issues that cannot be resolved within the software project are addressed by senior management. Deviations not resolvable with the software task leaders, software managers, or project manager are documented and presented to the senior manager designated to receive non-compliance items. p Non-compliance items presented to the senior manager are periodically reviewed until resolved. p 64

6/13/2021 Software Configuration Management p Purpose: u p To establish and maintain the integrity of the products of the software project throughout the project's software life cycle. Involves: Identifying the configuration of the software at given points of time u Controlling changes to the configuration u Maintaining the integrity and traceability of the configuration u 65

6/13/2021 Software Configuration Management Selected key practices Commitment to Perform: p An organizational policy specifies that: u u u SCM responsibility for each project is explicitly assigned SCM is implemented throughout the project’s life cycle SCM is implemented for externally deliverable products, designated internal work products and support tools projects have a repository for storing configuration items and associated SCM records baselines and SCM activities are audited on a periodic basis 66

6/13/2021 Software Configuration Management Goal 1: Software configuration management activities are planned. Activities Performed An SCM plan is prepared for each software project according to a documented procedure. p The SCM plan covers the SCM activities to be performed, including those performed by the software engineering group and other software-related groups(. p 67

6/13/2021 Software Configuration Management Goal 2: Selected software work products are identified, controlled, and available. Activities Performed The SCM plan is the basis for performing SCM activities. p A configuration management library system is established as a repository for the software baselines. p Work products to be placed under SCM are identified. p Products from the software baseline library are created, and their release is controlled, according to a documented procedure. p 68

6/13/2021 Software Configuration Management Goal 3: Changes to identified software work products are controlled. Activities Performed Change requests and problem reports for all configuration items/units are initiated, recorded, reviewed, approved, and tracked according to documented procedure. p Changes to baselines are controlled according to a documented procedure. p 69

6/13/2021 Software Configuration Management Goal 4: Affected groups and individuals are informed of the status and content of software baselines. Activities Performed The status of the configuration items is recorded according to a documented procedure. p Standard reports documenting the SCM activities and the contents of the software baseline are developed and made available to affected groups and individuals. p Software baseline audits are conducted according to a documented procedure. p Results of the software baseline audits are reported to the project software manager. p 70

6/13/2021 Establishing a defined process • Organization Process Focus • Organization Process Definition DEFINED • Training Program • Integrated Software Management REPEATABLE • Software Product Engineering • Intergroup Coordination • Peer Reviews 71

6/13/2021 Organization Process Focus p Purpose: u To establish the organizational responsibility for software process activities that improve the organization's overall software process capability. p Involves: u Developing and maintaining an understanding of processes used u Coordinating process improvement activities 72

6/13/2021 Organization Process Definition p Purpose: u To develop and maintain a usable set of software process assets that improve process performance across the projects and provide a basis for cumulative, long-term benefits to the organization. p Involves: u Developing and maintaining the organization’s standard software process and related process assets 73

6/13/2021 Training Program p Purpose: u To develop the skills and knowledge of individuals so they can perform their roles effectively and efficiently. p Involves: u Identifying training needs u Developing or procuring training to meet the needs 74

6/13/2021 Integrated Software Management p Purpose: u To integrate the software engineering and management activities into a coherent, defined software process that is tailored from the organization's standard software process and related process assets p Involves: u Developing a project’s defined process u Managing the project using this defined process 75

6/13/2021 Software Product Engineering p Purpose: u To consistently perform a well-defined engineering process that integrates all the software engineering activities to produce correct, consistent software products effectively and efficiently. p Involves: u Performing the engineering tasks using the defined process and appropriate tools and methods 76

6/13/2021 Intergroup Coordination p Purpose: u To establish a means for the software engineering group to participate actively with the other engineering groups so the project is better able to satisfy the customer's needs effectively and efficiently. p Involves: u Participating with other project engineering groups to address system-level requirements, objectives, and issues 77

6/13/2021 Peer Reviews p Purpose: u To remove defects from the software work products early and efficiently. p Involves: u Methodical examination of software work products by the producers’ peers to identify defects and areas where changes are needed 78

6/13/2021 Establishing a managed process MANAGED • Quantitative Process Management DEFINED • Software Quality Management 79

6/13/2021 Quantitative Process Management p Purpose: u To control the process performance of the software project quantitatively. Software process performance represents the actual results achieved from following a software process. p Involves: u Establishing goals for the performance of a project’s defined process u Taking measurements of process performance u Analyzing the measurements u Making adjustments to maintain process performance within acceptable limits 80

6/13/2021 Software Quality Management p Purpose: u To develop a quantitative understanding of the quality of the project's software products and achieve specific quality goals. p Involves: u Defining quality goals for software products u Establishing plans to achieve these goals u Monitoring and adjusting plans, work products, activities, and quality goals to satisfy customer needs 81

6/13/2021 Establishing an optimizing process • Defect Prevention OPTIMIZING • Technology Change Management MANAGED • Process Change Management 82

6/13/2021 Defect Prevention p Purpose: u To identify the cause of defects and prevent them from recurring. p Involves: u Analyzing defects that were encountered in the past u Taking specific actions to prevent the occurrence of those types of defects in the future 83

6/13/2021 Technology Change Management p Purpose: u To identify new technologies (i. e. , tools, methods, and processes) and track them into the organization in an orderly manner. p Involves: u Identifying, selecting, and evaluating new technologies u Incorporating effective technologies into the organization 84

6/13/2021 Process Change Management p Purpose: u To continually improve the software processes used in the organization with the intent of improving software quality, increasing productivity, and decreasing the cycle time for product development. p Involves: u Defining process improvement goals u Identifying, evaluating, and implementing improvements to the organization’s standard process and the projects’ defined processes on a continuous basis 85