www agiledigm com Empowering People through Process Executive

  • Slides: 52
Download presentation
www. agiledigm. com Empowering People through Process Executive Overview of CMMI® 18 Aug 2010

www. agiledigm. com Empowering People through Process Executive Overview of CMMI® 18 Aug 2010 NASA IT Summit Margaret Kulpa Chief Operating Officer of Agile. Digm, Inc. ® CMMI is registered in the U. S. Patent and Trademark Office by Carnegie Mellon University. © Agile. Digm, Inc.

Copyright Notices The presentation material in this seminar is copyrighted by Agile. Digm, Incorporated,

Copyright Notices The presentation material in this seminar is copyrighted by Agile. Digm, Incorporated, 2010. For further information, please contact: Agile. Digm, Inc. 11 Twelve Oaks Trail Ormond Beach, Florida 32174 U. S. A. Phone: +1. 386. 673. 1384 Email: info@agiledigm. com Terms like these are often used in the following material: CMMI SCAMPIsm and SCAMPI Lead Appraisersm CMMI is registered in the U. S. Patent and Trademark Office by Carnegie Mellon University. SM SCAMPI and SCAMPI Lead Appraiser are service marks of Carnegie Mellon University. Executive Overview of CMMI © 2010 2

Who is Agile. Digm? Agile. Digm, Incorporated is a US based corporation working with

Who is Agile. Digm? Agile. Digm, Incorporated is a US based corporation working with organizations around the world – providing consulting, training, and appraisals for organizations that build software-intensive systems – Software Engineering Institute (SEI) Partner – Scrum Training Institute (STI) Partner Authors of Interpreting the CMMI , best selling book on using the CMMI. Unique experience working with one of the only CMMI Maturity Level 5 Agile companies. CMMI is registered in the U. S. Patent and Trademark Office by Carnegie Mellon University. SCAMPI and SCAMPI Lead Appraiser are service marks of Carnegie Mellon University. SM Executive Overview of CMMI © 2010 3

What is the CMMI? • CMMI is an acronym for Capability Maturity Model Integration.

What is the CMMI? • CMMI is an acronym for Capability Maturity Model Integration. • Approach used by organizations to improve. • CMMI is a collection of best practices – Over 100 organizations formally assisted in its development, including NASA • There is a wide range of freedom in how an organization decides to embrace the CMMI. You can – – Go slow or fast – Choose the areas you need to focus on – Choose the appraisal mode you are comfortable with (SCAMPI A, B, C, High Maturity) Executive Overview of CMMI © 2010 4

Three Constellations • CMMI for Development – Development and maintenance of products – Integrates

Three Constellations • CMMI for Development – Development and maintenance of products – Integrates software engineering, systems engineering, collaborative teams, acquisition from the supplier’s side – Not just for software development! – Can be used for developing complex systems (healthcare, bridges, valves, cleanroom technology) • CMMI for Services – Organizations that provide services (from taxis to call centers) • CMMI for Acquisition – Acquiring products and services • Common Process Areas are embedded within all three • This presentation focuses on CMMI for Development. Executive Overview of CMMI © 2010 5

CMMI Concepts What the CMMI is not: – CMMI is not a process, standard,

CMMI Concepts What the CMMI is not: – CMMI is not a process, standard, directive, regulation, or description. – Not a list of “Shalls” that might require big changes in how you do your work. You can make the CMMI in your organization as intrusive and painful as you want --- OR NOT! It’s up to you. Executive Overview of CMMI © 2010 6

Does the CMMI tell me how to do stuff? • NO! • The CMMI

Does the CMMI tell me how to do stuff? • NO! • The CMMI documents best practices from many organizations. • The CMMI describes WHAT to do, not HOW to do it. • Organizations need to determine how to apply the best practices of the CMMI to how the organization works in order to improve how stuff gets done. Executive Overview of CMMI © 2010 7

Why Should I Use the CMMI? • Structures the effort (Where do I start?

Why Should I Use the CMMI? • Structures the effort (Where do I start? How do I know when I am done? ) • Helps identify an organization’s improvement objectives and priorities • Provides benefit from a larger community’s experience • Has an an appraisal methodology to diagnose the state of improvement efforts. Executive Overview of CMMI © 2010 8

Why Should I Use the CMMI? • CMMI focuses on “Process” • A process

Why Should I Use the CMMI? • CMMI focuses on “Process” • A process is a set of practices performed to achieve a given purpose (from CMMI Glossary) • Four major elements of process are tools, methods, materials, and people – Major determinants of product cost, schedule, and quality are tools, methods, materials, and people. Executive Overview of CMMI © 2010 9

CMMI Performance Results Summary This is the summary of improvements from all sources collected

CMMI Performance Results Summary This is the summary of improvements from all sources collected regardless of maturity levels. Improvement Nbr. of Data Median Lowest Highest Points Cost 34% 3% 87% 29 Schedule 50% 2% 95% 22 Productivity 61% 11% 329% 20 Quality 48% 2% 132% 34 Customer Satisfaction 14% -4% 55% 7 4. 0 : 1 1. 7 : 1 27. 7 : 1 22 Category Return on Investment Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI 2006 -TR-004 © 2010 10 Executive Overview of CMMI

Cost – Benefits and Impact • Cost metrics (and reductions in cost) defined as:

Cost – Benefits and Impact • Cost metrics (and reductions in cost) defined as: – – Cost of delivery – Cost of quality/Cost of poor quality Overhead rate – Costs of rework/Defect find & fix cost Software unit costs – Number/Cost of process staff Variation in cost performance index (CPI) • Organizations have reported improved – Budget estimation accuracy – Average in cost performance index (CPI) Cost example on next slide – cost per function reduced by 52% Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006 -TR-004 Executive Overview of CMMI © 2010 11

Cost Savings at DB Systems Gmb. H Source: Performance Results of CMMI-Based Process Improvement,

Cost Savings at DB Systems Gmb. H Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006 -TR-004 Executive Overview of CMMI © 2010 12

Schedule – Benefits and Impact • Schedule metrics (and reductions) defined as: – Variation

Schedule – Benefits and Impact • Schedule metrics (and reductions) defined as: – Variation in schedule – Schedule performance index (SPI) – Number of days late – Days variance from plan – Slippage of project delivery • Organizations have reported improved or increased – Cycle time – Average SPI – Proportion of milestones met – Estimation accuracy • Example from NCR Corporation (on next slide) – Average days schedule variance reduced from approximately 130 days to less than 20 days one year after reaching CMMI maturity level 2 Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006 -TR-004 Executive Overview of CMMI © 2010 13

Schedule Improvement at NCR Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical

Schedule Improvement at NCR Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006 -TR-004 Executive Overview of CMMI © 2010 14

Productivity – Benefits and Impact • Productivity measures (and improvements) defined as: – –

Productivity – Benefits and Impact • Productivity measures (and improvements) defined as: – – Time comparisons by build – Source statements per month Lines of code per labor hour – Number of releases per year Testing rates – Software production Function points (FP) per full time equivalent (FTE) staff • Example from IBM Australia (on next slide) – Over 20 percent improvement in account productivity as the organization moved from SW-CMM maturity level 3 toward CMMI maturity level 5. Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006 -TR-004 Executive Overview of CMMI © 2010 15

Productivity Improvement at IBM Australia Account Productivity (FP/FTE) Source: Performance Results of CMMI-Based Process

Productivity Improvement at IBM Australia Account Productivity (FP/FTE) Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006 -TR-004 Executive Overview of CMMI © 2010 16

Quality – Benefits and Impact • “Quality” most frequently defined as reductions in numbers

Quality – Benefits and Impact • “Quality” most frequently defined as reductions in numbers of defects. • Measurements include: – Defect counts by phase of injection – Defect counts by phase of discovery – Total defect density • Example from IBM Australia (on next slide) – 40 percent reduction in overall production and over 80 percent reduction in Severity 1 problems, as the organization moved from SW-CMM maturity level 3 toward CMMI maturity level 5 Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006 -TR-004 Executive Overview of CMMI © 2010 17

Quality Improvement at IBM Australia Source: Performance Results of CMMI-Based Process Improvement, Aug 2006,

Quality Improvement at IBM Australia Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006 -TR-004 Executive Overview of CMMI © 2010 18

Customer Satisfaction – Benefits and Impact • Customer satisfaction most frequently defined by the

Customer Satisfaction – Benefits and Impact • Customer satisfaction most frequently defined by the results of customer surveys. – Award fees are sometimes used as surrogate measures. • An example from Lockheed Martin Management and Data Systems – Increased award fees by 55 percent Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006 -TR-004 Executive Overview of CMMI © 2010 19

ROI – Benefits and Impact • ROI defined by cost avoidance measures and improvements

ROI – Benefits and Impact • ROI defined by cost avoidance measures and improvements including: – Rework avoided due to fewer defects – Improved productivity – Increased revenue due to shorter cycle times • Example from Raytheon Corporation, anonymous site – 6: 1 ROI in a CMMI maturity level 3 organization Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006 -TR-004 Executive Overview of CMMI © 2010 20

From personal experience … • Ensuring I am working on the latest (correct) version

From personal experience … • Ensuring I am working on the latest (correct) version of the code • Running tests that prove requirements are satisfied (not that the system is just not blowing up) • Predicting the number of projects to start up and the number of people I need ahead of time • Increasing communication throughout the organization (and hearing things I sometimes did not want to hear) • Not making phone calls to my QA person while she was in labor. Executive Overview of CMMI © 2010 21

CMMI Structure CMMI is a collection of best practices – These “best practices” reside

CMMI Structure CMMI is a collection of best practices – These “best practices” reside in 22 focus areas called “Process Areas. ” – Each process area has specifically stated goals that have to be met in order to have “maturity” or “capability” in the process area. – Each process area has specific practices, subpractices, typical work products (outputs), examples, and other informative material – Most of the goals are specific to the process area, but there are some generic or “global” goals that are common to all of the Process Areas like: • Provide Resources for this area • Train People for this area • Identify Stakeholders for this area Let’s look at the book … Executive Overview of CMMI © 2010 22

Structure of the CMMI • Maturity Level or Capability Level – Specific and Generic

Structure of the CMMI • Maturity Level or Capability Level – Specific and Generic Goals • Specific and Generic Practices – Subpractices, Typical Work Products, Examples • Two types of Goals and Practices – We will see these examples in the Example Process Area ☺ – Specific Goals pertain to one Process Area (SG 1 in Project Planning is “Establish Estimates”) – Specific Goals have associated Specific Practices (For SG 1 above, SP 1. 4 is Determine Estimates of Effort and Cost – There also Generic Goals and Generic Practices that pertain to all of the Process areas throughout the model. They are “Global. ” Executive Overview of CMMI © 2010 23

Example Process Area (Maturity Level 3) Decision Analysis and Resolution The purpose of Decision

Example Process Area (Maturity Level 3) Decision Analysis and Resolution The purpose of Decision Analysis and Resolution (DAR) is to analyze possible decisions using a formal evaluation process that evaluates identified alternatives against established criteria. The CMMI then continues with half a page of Introductory Notes that further describe what this Process Area is about. Note: Underlines and emphasis added by the presenter here and in the following slides. I have omitted some of the Informative Material. Informative material is very useful in interpreting the intent of the process area/practice/subpractice and should not be ignored. Executive Overview of CMMI © 2010 24

Decision Analysis and Resolution • SG 1: Evaluate Alternatives – – – SP 1.

Decision Analysis and Resolution • SG 1: Evaluate Alternatives – – – SP 1. 1 Establish Guidelines for Decision Analysis SP 1. 2 Establish Evaluation Criteria SG – Specific Goal SP 1. 3 Identify Alternative Solutions SP – Specific Practice SP 1. 4 Select Evaluation Methods *** SP 1. 5 Evaluate Alternatives SP 1. 6 Select Solutions Note: Other Process Areas may have more Specific Goals and more Specific Practices. There is more informative material presented in the book. ***This is the practice used in the following example. Executive Overview of CMMI © 2010 25

Decision Analysis and Resolution SP 1. 4 Select the Evaluation Methods Typical Work Products:

Decision Analysis and Resolution SP 1. 4 Select the Evaluation Methods Typical Work Products: 1. Selected evaluation methods Subpractices 1. Select the methods based on the purpose for analyzing a decision and on the availability of the information used to support the method. Typical evaluation methods include the following: – Modeling and simulation – Engineering studies – Manufacturing studies – Cost studies – Business opportunity studies Executive Overview of CMMI © 2010 26

Decision Analysis and Resolution – Surveys – Extrapolations based on field experience and prototypes

Decision Analysis and Resolution – Surveys – Extrapolations based on field experience and prototypes – User review and comment – Testing – Judgment provided by an expert or group of experts 2. Select evaluation methods based on their ability to focus on the issues at hand without being overly influenced by side issues. 3. Determine the measures needed to support the evaluation method. – Consider the impact on cost, schedule, performance, and risks. Executive Overview of CMMI © 2010 27

Generic “Common” Goals and Practices for All Process Area Generic Goal 2 Institutionalize a

Generic “Common” Goals and Practices for All Process Area Generic Goal 2 Institutionalize a Managed Process GP 2. 1 Establish an Organizational Policy GP 2. 2 Plan the Process GP 2. 3 Provide Resources GP 2. 4 Assign Responsibility GP 2. 5 Train People GP 2. 6 Manage Configurations GP 2. 7 Identify and Involve Relevant Stakeholders GP 2. 8 Monitor and Control the Process GP 2. 9 Objectively Evaluate Adherence GP 2. 10 Review Status with Higher-Level Management GG – Generic Goal GP -Generic Practice Generic Goal 3 Institutionalize a Defined Process GP 3. 1 Establish a Defined Process GP 3. 2 Collect Improvement Information © 2010 Executive Overview of CMMI 28

Two representations of the model • Staged Representation (or “View”): – – Most like

Two representations of the model • Staged Representation (or “View”): – – Most like other currently accepted models Focuses on all process areas contained within a level Used for Maturity Level ratings for contract awards Start with any Process Area within a level. • Continuous Representation (or “View”): – Was designed to focus on specific, individual process areas – Process areas are arranged into functional categories – Start with any Process Area. • Staged is the most “popular” CMMI representation. – When your customers ask for a level or when an organization says “We are Level N”, they are referring to a Maturity Level of the staged representation. Executive Overview of CMMI © 2010 29

Not to Worry • The information contained is the same in both! • The

Not to Worry • The information contained is the same in both! • The only difference is in your approach for implementation and appraisal. Executive Overview of CMMI © 2010 30

Process Areas by Maturity Level (Staged) Executive Overview of CMMI © 2010 31

Process Areas by Maturity Level (Staged) Executive Overview of CMMI © 2010 31

Process Areas by Category (Continuous) Process Management Project Management Organizational Process Focus Organizational Process

Process Areas by Category (Continuous) Process Management Project Management Organizational Process Focus Organizational Process Definition (with IPPD) Organizational Training Organizational Process Performance Organizational Innovation & Deployment Project Planning Project Monitoring and Control Supplier Agreement Management Integrated Project Management (with IPPD) Risk Management Quantitative Project Management Engineering Support Requirements Management Requirements Development Technical Solution Product Integration Verification Validation Measurement & Analysis Process and Product Quality Assurance Configuration Management Decision Analysis & Resolution Causal Analysis and Resolution Executive Overview of CMMI © 2010 32

CMMI “Staged” Maturity Levels Level 1 – Initial Level 2 – Managed Level 3

CMMI “Staged” Maturity Levels Level 1 – Initial Level 2 – Managed Level 3 – Defined Level 4 – Quantitatively Managed Level 5 - Optimizing “Continuous” Capability Levels Level 0 - Incomplete Level 1 – Performed Level 2 – Managed Level 3 – Defined Level 4 – Quantitatively Managed Level 5 - Optimizing ØProcess Management ØProject Management ØEngineering ØSupport Process Areas (22) There are no “shalls”. Appraisals yield Ratings It’s a Model, not a specification Process Areas (22) Active PA Subset Specific Goals Specific Practices Executive Overview of CMMI Realm of Appraisals © 2010 33 Generic Goals Generic Practices It can be hierarchical. It’s highly detailed. . It’s complex, but nothing is complicated.

What is a SCAMPIsm Appraisal? • SCAMPIsm – Standard CMMI Appraisal Method for Process

What is a SCAMPIsm Appraisal? • SCAMPIsm – Standard CMMI Appraisal Method for Process Improvement – Used for baselining current behaviors to begin improvement activities or see into ongoing improvement activities – To assign a formal rating for contract award or fees (if requested by appraisal sponsor) – Several appraisal methods (class A, B, C, plus High Maturity) – Uses the CMMI as reference model. • SCAMPI goal is to provide an accurate picture of your organization relative to the reference model. Executive Overview of CMMI © 2010 34

Reported Appraisals CMMI® For Development SCAMPIsm Class A Appraisal Results 2009 End-Year Update Red

Reported Appraisals CMMI® For Development SCAMPIsm Class A Appraisal Results 2009 End-Year Update Red indicates countries with reported SCAMPI Class A Appraisals Executive Overview of CMMI © 2010 35

Maturity Profile – End of Year 2009 • The percentage of organizations at each

Maturity Profile – End of Year 2009 • The percentage of organizations at each maturity level based on most recent appraisals of 2753 organizations. Executive Overview of CMMI © 2010 36

Appraisal Classes A, B, and C Executive Overview of CMMI © 2010 37

Appraisal Classes A, B, and C Executive Overview of CMMI © 2010 37

Appraisal Team (SCAMPI “A”) • Minimum team size is four - A Lead Appraiser

Appraisal Team (SCAMPI “A”) • Minimum team size is four - A Lead Appraiser and three additional team members. • At least one team member must be from the appraised organization. • All team members must be trained in the official CMMI and SCAMPI courses • Conflicts of interest must be avoided – Exclude process authors and individuals directly impacted by the outcome of the appraisal • Team members cannot be managers of any of the selected projects or groups • Appraisal sponsors cannot be team members. Executive Overview of CMMI © 2010 38

What does the SCAMPI Team do? • Reviews an organization’s documentation – • Conducts

What does the SCAMPI Team do? • Reviews an organization’s documentation – • Conducts interviews – • To ensure the documentation is actually used Maps results to the CMMI – • Processes, procedures, policies, status reports, meeting minutes Practices, Goals, Process Areas Assigns ratings (if requested and allowed by the type of SCAMPI). Executive Overview of CMMI © 2010 39

What does the Team NOT do? • Does not go looking for the documentation

What does the Team NOT do? • Does not go looking for the documentation – Easter egg hunts are not allowed! • Does not judge the “goodness” of the process • Does not judge how well you are doing your job. Executive Overview of CMMI © 2010 40

PPQA Example Evidence ID Goal/Practice SG 1 Objectively Evaluate Processes and Work Products SP

PPQA Example Evidence ID Goal/Practice SG 1 Objectively Evaluate Processes and Work Products SP 1. 1 Objectively Evaluate Processes QA audit report SP 1. 2 Objectively Evaluate Work Products and Services SG 2 Provide Objective Insight SP 2. 1 Communicate and Ensure Resolution of Noncompliance Issues QA audit report SP 2. 2 Establish Records QA Reports, Status of Corrective Actions, Report of Quality Trends Executive Overview of CMMI Potential Artifacts © 2010 41 QA audit report

Simplified Point of the SCAMPI: • Do you have a (documented) process? • Do

Simplified Point of the SCAMPI: • Do you have a (documented) process? • Do you follow it? • How closely does it map to the CMMI? Executive Overview of CMMI © 2010 42

Conclusion (1 of 2) • Organizations can and have benefited from using the CMMI.

Conclusion (1 of 2) • Organizations can and have benefited from using the CMMI. • Implement the CMMI to support your organization’s business objectives, goals, and culture. • “Engage your brain. ” Interpretation of the information must be used. • Do not ignore the informative information and the subpractices. They help to understand the intent of the practice or Process Area. • Use a SCAMPI to guide your implementation, measure your progress, and verify your achievements. Executive Overview of CMMI © 2010 43

Conclusion (2 of 2) • My favorite websites for more information: – www. sei.

Conclusion (2 of 2) • My favorite websites for more information: – www. sei. cmu. edu (main SEI website) – seir. sei. cmu. edu (contains helpful presentations and articles from the community) Executive Overview of CMMI © 2010 44

Questions? • Time for some questions? Executive Overview of CMMI © 2010 45

Questions? • Time for some questions? Executive Overview of CMMI © 2010 45

Contact Information The presentation material in this seminar is copyrighted by Agile. Digm, Incorporated,

Contact Information The presentation material in this seminar is copyrighted by Agile. Digm, Incorporated, 2010. For further information, please contact: Margaret Kulpa Chief Operating Officer Agile. Digm, Inc. 11 Twelve Oaks Trail Ormond Beach, FL 32174 USA +1 386 673 1384 Margaret. kulpa@agiledigm. com www. agiledigm. com Executive Overview of CMMI © 2010 46

Backup Slides Executive Overview of CMMI © 2010 47

Backup Slides Executive Overview of CMMI © 2010 47

Combined View of Staged and Continuous Executive Overview of CMMI © 2010 48

Combined View of Staged and Continuous Executive Overview of CMMI © 2010 48

Maturity Levels (MLs) • A maturity level signifies the level of performance that can

Maturity Levels (MLs) • A maturity level signifies the level of performance that can be expected from an organization. There are 5 maturity levels. Following are the key features of each maturity level (ML): – – ML 1 (Initial) – ad hoc processes ML 2 (Managed) – basic project management system ML 3 (Defined) – process standardization ML 4 (Quantitatively Managed) – quantitative management “by the numbers” – ML 5 (Optimizing) – continuous process improvement ML 2 provides a foundation for ML 3, ML 3 for ML 4, and ML 4 for ML 5. Executive Overview of CMMI © 2010 49

Appraisal Sponsor • SCAMPI appraisals require an appraisal sponsor. – Usually this is a

Appraisal Sponsor • SCAMPI appraisals require an appraisal sponsor. – Usually this is a senior manager. – The appraisal sponsor is responsible for • Verifying that the appraisal team leader has the appropriate experience, knowledge, and skills to lead the appraisal. • Ensuring that the appropriate organizational units (projects and support) participate in the appraisal. • Supporting the required confidentiality and nonattribution rules. – Some appraisals have multiple sponsors. Executive Overview of CMMI © 2010 50

“Typical” Appraisal Roadmap May 2010 Plan • Plan SCAMPI activities • Monitor and adjust

“Typical” Appraisal Roadmap May 2010 Plan • Plan SCAMPI activities • Monitor and adjust plan August 2010 (1 – Week) Train Team • Train Appraisal Team • Begin PIID creation • Finalize Plan • Organization Develops PIID documents October 2010 (1 – Week) Reviewing PIIDs is a key focus of the required readiness review Readiness Review December 2010 (1 – 3 Weeks) • Conduct Appraisal Executive Overview of CMMI © 2010 51 Appraisal Onsite

Presenter – Margaret Kulpa • Chief Operating Officer of Agile. Digm, Incorporated • She

Presenter – Margaret Kulpa • Chief Operating Officer of Agile. Digm, Incorporated • She is the primary author of the book Interpreting the CMMI: A Process Improvement Approach, Second Edition, Auerbach (2008) which details how to identify, define, and improve business, systems and software processes on both enterprise-wide and project levels. • Ms. Kulpa has developed and taught courses in software quality assurance, project management, peer reviews, requirements development and management, process baselines and modeling, safety and security, and other software engineering-related subjects. She has developed courses in CMMI and has served as an external member of numerous SCAMPIsm Class A Appraisals. Executive Overview of CMMI © 2010 52