CMMI Based Process Improvement Risk Management Concept Version

CMMI Based Process Improvement Risk Management “And through all this welter of change and development, your mission remains fixed, determined, inviolable -- it is to win our wars. ” General Douglas Mac. Arthur © Copyright 2001 The MITRE Corporation

CMMI Based Process Improvement Risk Management Reasons “If you always do what you’ve always done, you’ll continue to get what you always got” and we can no longer afford it • • • Only 16% of all Information Technology (computer and software) projects complete on time and on budget 31% are cancelled before completion The remaining 53% are late and over budget, with the typical cost growth exceeding the original budget by more the 89% – Average overrun of project budgets was 189% – The average schedule overrun for projects that were in difficulty was 222% Of the IT projects that are completed, the final product contains only 61% of the originally specified features If no formal systems engineering effort is included, projects run the risk of 50% to 100% development cost overruns DSB recommendation: Employ developers who demonstrate CMM Level 3 or equivalent. Certification must be less that 2 years old. Eliminate the “escape clause” “Charting the Seas of Technology: The CHAOS Study” The Standish Group, January 1995 Report of the Defense Science Board Task Force on Defense Software, Nov 2000 INCOSE Systems Engineering Handbook © Copyright 2001 The MITRE Corporation

CMMI Based Process Improvement Risk Management Process Improvement Objectives Establish a Standard Risk Management Process That Will Assist in Achieving Overall Center Objectives ESC Objectives Standard Risk Management Process Objectives ãAddress Changes in Major Stakeholders ãAddress Acquisition and Operational Risks ãConsistency with Current AF and DOD Policy ãTool Independent ãProgram Office Perspective ãValue-Add Integral to Everyday Program Management ãAccessible by Sponsor via Web Technology ãAddress Risk to the Enterprise © Copyright 2001 The MITRE Corporation · Shorter Time to Market · Integrated Command Control and Combat Support · Harmonize Capabilities, Interoperability, User Needs, Budget, and Technology · Dealing With Uncertainty · Life-Cycle Systems Engineering · Streamline Communications

CMMI Based Process Improvement CMMI Risk Management Goals and Practices Plan and Prepare for Risk Management Identify and Analyze Risks Determine Establish p Risk Management is a continuous, forward-looking Define Risk a Risk Sources process that is an important part of. Management business and technical Identify Parameters and Strategy Risks management Categories processes. Risk management needs to address issues that could endanger critical objectives. A continuous risk management approach is applied to ensure Responsibility effective anticipation and. Risk mitigation of risks with critical impact across the project life cycle. CMMI Continuous From Project Planning Evaluate, Mitigate Risks Representation, V 1. 0, August 2000 and Project Monitoring Classify, and Control DAR © Copyright 2001 The MITRE Corporation Implement Risk Mitigation Plans Develop Risk Mitigation Plans Prioritize Risks

CMMI Based Process Improvement Risk Management Defined o “Risk is a measure of the inability to achieve system life cycle objectives. . . ” † such as the following: • • Assurance of Program Viability Provision of Operational Capability Delivery Within Negotiated Baseline Assurance of Operational Asset Survival Assurance of Mission Success Assurance of Personnel Safety and Performance Assurance of Integration with Operational Environment o “Risk has two components: • The probability (or likelihood) of failing to achieve particular system life cycle objectives • The consequences of failing to achieve those objectives” † † Adapted from AFMCP 63 -101, 9 July 1997 © Copyright 2001 The MITRE Corporation Risk o “Risk is a measure of the inability to achieve system life cycle objectives. . ” † such as the following: • Assurance of Program Viability • Provision of Operational Capability • Delivery Within Negotiated Baseline • Assurance of Operational Asset Survival • Assurance of Mission Success • Assurance of Personnel Safety and Performance • Assurance of Integration with Operational Environment o “Risk has two components: • The probability (or likelihood) of failing to achieve particular system life cycle objectives • The consequences of failing to achieve those objectives” †

CMMI Based Process Improvement Program Life Cycle and Risk Management A B Concept & Technology Development Pre - Systems Acquisition © Copyright 2001 The MITRE Corporation IOC C System Development & Demonstration Production & Deployment OT&E FRP Decision Review Systems Acquisition (Engineering and Manufacturing Development, Demonstration, LRIP and Production) FOC Operations & Support Sustainment

CMMI Based Process Improvement Objectives and the System Life Cycle Objectives Assurance of Program Viability Provision of Operational Capability Delivery Within Negotiated Baseline Assurance of Operational Asset Survival Assurance of Mission Success Assurance of Personnel Safety and Performance Assurance of Integration with Ops. Environment © Copyright 2001 The MITRE Corporation Concept Acq. Package Mission Def. Strat. Develop Integ. O&S

CMMI Based Process Improvement Process Life Cycle Context Players in the Life of a

CMMI Based Process Improvement Program Life Cycle and Risk Management Process Modeling Mission Area Planning Risk Zone Vision & Goals Experimentation Mission Shift PPBS Risk Zone Operational & Technical Architecture(s) Existing Direction ORD Risk Zone Budgeting & Tradeoffs Threat Change SRD/TRD Development User Requirements Definition Risk Zones Funding & Direction Funding & Contract Award Depot Start DT&E Complete System Life Cycle Direction A B IOC Form User Group IOC C Concept & Technology Development Pre - Systems Acquisition Mission Integration Package Development Production & Deployment System Development & Demonstration OT&E FRP Decision Review Define Post Award Work Packages and IPTs Operations & Support Contract Award Form Working Level IPT Product Design & Development Product Integration Back to Development FOC Fielding Partial Decision Full Operations & Support Begin Operations O&M Planning Begin Sustainment TCTOs, Reprocurement, Modifications, SLEP Engineering, Mission Shift Test. Assets Test Assets Operate and Maintain Form Test & Integration Working Group Certification and Accreditation Security Net Worthiness Interoperability Deficiency Reports Incident Reports Maint Data Analysis Production Readiness DT&E Mission Integration Establish Test Environments Labs Support Elements Test Assets Training Back to Development OT&E No Production Decision Yes IOC Infrastructure Readiness Conduct Training, Connectivity, Installation, Deployment © Copyright 2001 The MITRE Corporation Source Selection Package Development Sustainment Systems Acquisition (Engineering and Manufacturing Development, Demonstration, LRIP and Production) Phase. Out&&Disposal Phase RFP Preparation System Disposal Post Award Conference Acquisition Strategy ASP Form Program Office Contract Award Concept Definition Acquisition Strategy Development Requirements Feasibility Risk Zone Requirement Feasibility Program Planning IPT Acquisition Strategy Funding & Direction Risk Zone Capstone Architecture Spiral Feedback Concept Definition Shortfalls & Opportunities Finalize Production Readiness OT&E System Fixes

CMMI Based Process Improvement Risk Management Space Concept Definition Acquisition Strategy Mission Integration Package Development Operations & Support IOC A B FOC C Operational Risk Management Concept & Technology Development FRP System Development & Demonstration Decision & Production Review OT&E Deployment Operations & Support Acquisition Risk Management Pre - Systems Acquisition © Copyright 2001 The MITRE Corporation Systems Acquisition (Engineering and Manufacturing Development, Demonstration, LRIP & Production) Sustainment

CMMI Based Process Improvement Risk Management Process Step 2 Step 1 Identify Risks & Hazards Prepare Revise Risk Plan Is the Risk Plan Working Yes ? (Continue Monitoring) No Yes (Continue Monitoring) Step 7 Monitor Handling Plans Step 3 New Phase or Key Stakeholder ? No Assess & Prioritize Risks Yes Key Milestone No Approaching (Continue ? Step 4 Monitoring) Decide on Control Options (Continue Monitoring) No Step 6 Implement Risk Handling Plans © Copyright 2001 The MITRE Corporation “n” Mo Since Last Assessment ? Yes Step 5 Establish Handling Plans

CMMI Based Process Improvement System Life Cycle Application of Risk Process Concept Definition Acquisition Strategy Mission Integration Package Development Operations & Support IOC A B FOC C Operational Risk Management Concept & Technology Development FRP System Development & Demonstration Decision & Production Review OT&E Deployment Operations & Support Acquisition Risk Management Pre - Systems Acquisition (Engineering and Manufacturing Development, Demonstration, LRIP & Production) Applied Continuously Through Life Cycle © Copyright 2001 The MITRE Corporation Sustainment

CMMI Based Process Improvement Risk Management Process Step 1 - Prepare Action 1: Obtain Buy-In From Program Manger on Risk Assessment and Management Action 2: Identify & Notify Key Program/ Mission Stakeholders Action 3: Identify and Distribute Key Program/Mission Objectives & Requirements Action 4: Identify, Review, and Distribute Applicable Risk /Hazard Taxonomies Commit Form the Team Know the Mission Think Risks PM Risk Management Becomes a Program Priority • Stakeholders Become Co-Sponsors • Stakeholders Commit to Sufficient Resource Bu y-I n Risk Management Becomes a Management Priority • Manager Becomes Advocate of Risk Management • Manager Commits Energy and Resource to Effort IN yu r. B e rity d o l i Pr ho n e sio ak s t i S M ID Risks Risk Management Becomes a Mission Priority • Process is Focused on Successful Mission • Stakeholders Become Familiar with Program and Mission Stakeholders Identify Mission Uncertainties • Risk Manager Makes Various Risk Data and Information Available to All Stakeholders • Each Stakeholder Formulates Individual Concerns/Uncertainties © Copyright 2001 The MITRE Corporation 1

CMMI Based Process Improvement Risk Management Process Step 2 - Identify the Risks and Hazards Action 1: Assemble Stakeholders for Risk Assessment Establish Team Action 2: Review Program/Mission Objectives, Taxonomies and Risk Assessment Process Develop Understanding Action 3: Conduct Risk Identification Action 4: Group Related Risks Action 5: Consolidate Related Risks & Write “If Then” Risk Statements Identify Classify Write Conduct Risk Management Meetings • Initial Meeting Sets Tone and Opens Channels of Communication • Subsequent Meetings Need to be Held as Program Progresses Understand Mission, How Risk Will be Managed, and Tools Available • Compare Current Mission with Past Missions (Taxonomies) • Make Sure Everyone Understands How Risk Management Will be Done Identify Program Risks • The Inability to Achieve Program Objectives • When will it Happen Classify the Risks • Can Use a Predefined Structure (Taxonomy, WBS, CPT etc. ) • Can Use a Self-Organized Structure Consolidate Like Risk and Write Risk Statements • Capture Concise Description to be Acted Upon • Risk Statement = Condition + Consequence © Copyright 2001 The MITRE Corporation a 1

CMMI Based Process Improvement Risk Management Process Step 3 - Assess and Prioritize Risks Action 1: Identify & Get Consensus on Impact / Severity for Each Risk Action 2: Identify & Get Consensus on Probability of Occurrence for Each Risk Impact ? Probability ? Action 3: Identify Time Window when Risk Could Occur Action 4: Reassess Any Existing Risks in Database When ? Old Risks ? Action 5: Prioritize Risks by Impact, Probability & Time Action 6: Identify Handling Bands Prioritize Coarse Sort A • Identify Consequences or Level of Impact to the Program If the Risk Occurs • Establish or use Predetermined Impact Categories (e. g. Critical, Serious, Moderate, Minor, Negligible) • Determine the Probability of Occurrence • Establish or Use Predetermined Probability Bands ( e. g. Very Unlikely, Probably, Likely, Very Likely) • For Each Risk Identify the Time Period When the Risk Is Likely to Occur • Establish or Use Predetermined Time Periods ( e. g. Near, Midterm, Far) • Incorporate Existing Identified Risks With Newly Identified Risks • Reassess Existing Risks Following Actions 1, 2, and 3. • Fold Existing Risks and Newly Identified Risk Together • Prioritize Risks • Involves Grouping Risks Using Impact, Probability and Timing • Objective Is to Identify Most Serious Program Risks • Identify Risk Handling Bands • Place Risks in to Appropriate Handling Band • Objective Is to Establish Preliminary Resource Constraints © Copyright 2001 The MITRE Corporation 1

CMMI Based Process Improvement Risk Management Process Step 4 Decide on Handling Options Action 1: Identify Handling Options Within Each Risk Band Action 2: Identify Which Risks will be Assumed or Watched Action 3: Identify Which Risks will be Avoided, Transferred or Mitigated Action 4: Assign Plan OPRs for Avoided, Transferred, or Mitigated Risks (Active) Action 5: Establish or Update Risk Database Options Easy Risks Hard Risks Responsibility Capture Choose Risk Handling Options Decide Which Risk Will: • Be Assumed • Be Watched (set “Triggers” or “Cues”) • Avoided • Transferred • Mitigated Assign Responsibility for Risk Planning • Avoid Risk - Research, Design, Fund etc. • Transfer Risk - To Whom, Acceptance • Mitigate Risk - Strategy, Resources etc. Ranked Risks High Medium Low Avoid Transfer Mitigate Monitor Assume a “Cues” Action Plans Cont Plans “Triggers” Establish and Update a Risk Database © Copyright 2001 The MITRE Corporation 1

CMMI Based Process Improvement Risk Management Process Step 5 - Establish Handling Plans Action 1: Develop Draft Handling Plans and Associated Resource Requirements Action 2: Program Manager Review and Approval of Handling Plans Action 3: Handling Plan are Funded, Directed, and Integrated with Program Management Develop Plans & Estimates Review and Approve Fund, Direct, Integrate Draft the Handling Plans • Avoided, Transferred and Mitigated Risks • Contingency and Risk Status Change Plans • 1 -3 Pages, Standard Format, Matches Database A Integrated DR AF T Program Manager Review and Approval Approved • Program Manager Buy-in of the Handling Plan • Formal Process to Insure That Resources Required Are Allocated • Opportunity to Improve the Handling Plan and Provide Team Perspective • Process Is Iterative and May Require a Number of Changes to Proposed Plans • Can Provide an Opportunity to Expose and Adjudicate Different Points of View Funded, Directed and Integrated with Program Management • Usually Requires Expenditure of Resources (E. G. Cost Estimates And/or Budget Actions) • For a Handling Plan to Have Impact It Must Be Enforceable • Appropriate Changes to Program Directives and Execution Documents and Monitored © Copyright 2001 The MITRE Corporation 1

CMMI Based Process Improvement Risk Management Process Step 6 - Implement Risk Handling Action 1: Finalize Risk Management Plan & Management Infrastructure Finalize RMP Action 2: Provide Mechanism to Monitor Triggers, Cues and Handling Plans Action 3: Implement Handling Plans as Authorized, Funded, & Scheduled Work with Exit Criteria Action 4: Provide Reporting on Handling Plan Results & Progress in Meeting Exit Criteria Monitoring Approach Implement Monitor Progress Complete Risk Management Plan (RMP) • RMP Can Be Completed - the Program Now Has a Good Understanding of Program Risk • Risk Management Program Management Provide for a Mechanism to Monitor • Handling Plans • Triggers and Cues Implement the Handling Plans • Implement = Knowledge + Resources + Authority to Act • Communicate, Communicate Risk Handling © Copyright 2001 The MITRE Corporation a Handling Status Review …. TAKE ACTION! Risk Management Database …. UPDATE! 1

CMMI Based Process Improvement Risk Management Process Step 7 - Monitor Handling Plans Action 1: Periodically Review Handling Plan Results Action 2: Stop or Modify Handling Plans and Resources, if required Action 3: Retire Risks When Handling Plans are Successfully Completed Review Modify or Stop Retire High Establish Handling Bands Periodic Reassessment Handling Band ? Est. Trigger & Contingency Plan Yes “Accept” Risk ? No Trigger Monitoring Trigger Occurs ? Yes No Yes Continue Implementation of Contingency Plan Done or OBE ? Implement Contingency Plan Trigger Monitoring Time to Reassess ? STOP Yes © Copyright 2001 The MITRE Corporation Yes Is It Working ? Work The “Issue” Yes Update Database Medium No a Low No Handling Band ? Yes Monitor Trigger Action 4: Update Risk Database for Handling Plan Progress & Risk Retirement Handling Band Accept Do Nothing Until Reassessment Yes Prepare, Approve and Implement Handling Plan No No “Accept” Risk ? Yes Handling Plan Monitoring Monitor Handling Plans Yes Time to Reassess ? No Handling Plan Monitoring STOP Is It Working ? Establish Cues for Increasing Risk No Yes Time to Reassess ? No Yes Done or OBE ? Monitor Cues Cue Monitoring No Cue Occurs ? Yes No Cue Monitoring Reassess Individual Risk Immediately 2

CMMI Based Process Improvement Risk Management Process Step 7 - Monitor Handling Plans STEP 7 - MONITOR HANDLING PLANS Action 1: Periodically Review Action Plan Results Action 3: Retire Risks When Action Plans are Successfully Completed Step 7 Decisions Action 2: Stop or Modify Action Plans and Resources, if required Action 4: Update Risk Database for Action Plan Progress & Risk Retirement PROACTIVE LOOK AHEAD • To “n” Review New “n” New Key Mo. Mo & Identify Risks Is This Phase or Key Milestone Since Last. New Handling Phase or Key Milestone Since Last • To Create Plans Pre DEC Working Not Working Stakeholder Approaching Assessment Rebuild RMP & ORD JAN NOV ? ? ? ? to Step 2 to Insure • Return Communicate Buy-In all Risk and Hazards Have Major Program OCT Re-Establish FEB Yes Yes Post Risk to Senior of Present Set of Been Identified ORD Decision Maker SEP Return to to Step 1 1 AUG MAR Stakeholders Return to to Step 2 2 Return to Step 1 1 Revise. RMP Risk Plan Revise Dev. APR Period MAY JUL New Stakeholders There. JUN may be other reasons or events that will require a Ops. Env. Review Establish A Regular Are Added or Integ. reworking of the process ……. . it is imperative that the Changed Throughout Cycle & Go Back Through the Return to Step 1 possibility and Revise the program team be open to this and be prepared the Program Process Starting With Step 2 Risk Management Plan to fix the problem because …… O&M Life Cycle Risk Management Program Management © Copyright 2001 The MITRE Corporation 2

CMMI Based Process Improvement Risk Management Process “Life is tough, but it’s tougher if you’re stupid” John Wayne as Sergeant John M. Stryker, USMC, in “The Sands of Iwo Jima” © Copyright 2001 The MITRE Corporation

CMMI Process Life Cycle Context Program Life Cycle Concept Definition Process Modeling JV 2020 Vision & Goals EFX, ATD, ACTD Experimentation Mission Area Planning Shortfalls & Opportunities PPBS Spiral Feedback STAR Threat Change EAF Mission Shift Existing Direction Program Planning IPT © Copyright 2001 The MITRE Corporation Capstone Architecture User Requirements Definition Budgeting & Tradeoffs Operational & Technical Architecture(s) Requirement Feasibility Program Direction

CMMI Process Life Cycle Context Program Life Cycle Acquisition Strategy Capstone Architecture Funding User Requirements SRD/TRD Development (1) APB PMD Ops & Tech Architectures Requirements Feasibility Form Program Office © Copyright 2001 The MITRE Corporation Acquisition Strategy Development ASP RFP Preparation (2) Source Selection Contract Award

CMMI Process Life Cycle Context Program Life Cycle Package Development Contract Award Post Award Conference Define Post Award Work Packages and IPTs System Architecture Development Package Design & Development Product Integration Package Design & Development Back to Development Form Working Level IPT Production Spiral “n” Requirements Package Tradeoff. Test. Package Consensus Readiness Design Refinement and Decision. Articles and Contract Action © Copyright 2001 The MITRE Corporation Test Ready? Product Integration Partial Full Spiral “n” DT&E Development

CMMI Process Life Cycle Context Program Life Cycle Mission Integration Test Articles Form Test & Integration Working Group Certification and Accreditation Establish Test Environments Back to Development OT&E No Production Decision Yes Depot Start Infrastructure Readiness Conduct Training, Connectivity, Installation, Deployment © Copyright 2001 The MITRE Corporation Finalize Production Readiness OT&E System Fixes

CMMI Process Life Cycle Context Program Life Cycle IOC Form User Group Operations & Maintenance Begin Operations O&M Planning Phase Out & Disposal © Copyright 2001 The MITRE Corporation Begin Sustainment Spiral N+1 (e. g. TCTOs, Reprocurement, Modifications, SLEP Engineering, Mission Shift) Operate and Maintain