The EIA731 Capability Maturity Model its Relationship to

The EIA-731 Capability Maturity Model, & its Relationship to Other Systems Engineering Standards Paul Davies Thomson Racal Defence Ltd

Themes l What Standards are ‘Out There’? l Where did they come from? l What topics do they cover? l What’s their current status? l Who uses them, and what for? l Where are they going? l Which one is best to use now? l How do they support business process improvement?

The ‘Frameworks Quagmire’ - What Standards are out there? PSP People CMM SW-CMM SE-CMM CMMI** ISO 15504 (SPICE) IPDCMM* SECM (EIA/IS 731) SECAM SSECMM MIL-STD -499 B* IEEE 1220 EIA/IS 632 * Not released ** CMMI-SE/SW released IPPD not yet released MIL-Q -9858 SDCE SA-CMM FAAi. CMM SDCCR IEEE Stds. 730, 828 829, 830, 1012, 1016 1028, 1058, 1063 NATO AQAP 1, 4, 9 EQA Trillium DO 178 B DOD IPPD AF IPD Guide Baldrige Tick. IT Q 9000 ISO 10011 BS 5750 MIL-STD-1679 DODSTDDOD-STD 2168 -2167 A MIL-STD 498 ISO/IEC 12207 ISO 9000 Series ISO 15288* EIA 632 Also see www. software. org/quagmire EIA/IEEE J-STD-016 IEEE 1074 IEEE/EIA 12207 SOFTWARE PRODUCTIVIT YCONSORTIUM

The ‘Frameworks Quagmire’ - What Standards are out there? PSP Meaning MIL-Q of color MIL-STD-1679 SW-CMM -9858 DODSDCE Red A Capability Maturity Model People SCE STDDOD-STD CMM Green A U. S. Government or military 2168 IEEE Stds. 730, 828 -2167 A NATO document SA-CMM 829, 830, 1012, 1016 AQAP 1, 4, 9 ISO 1028, 1058, 1063 Purple An international standard 15504 EQA MIL-STD(SPICE) Blue Commercial, professional, or FAACMMI** industrial 498 association Baldrige i. CMM BS Trillium IPDdocument (mostly U. S. ) EIA/IEEE 5750 CMM* DOJ-STD-016 ISO/IEC Black Other DOD 178 B SECM 12207 SE-CMM IPPD (EIA/IS 731) SECAM SSECMM MIL-STD -499 B* Color SDCCR IEEE 1220 EIA/IS 632 * Not released ** CMMI-SE/SW released IPPD not yet released AF IPD Guide Tick. IT Q 9000 ISO 10011 ISO 9000 Series ISO 15288* EIA 632 Also see www. software. org/quagmire IEEE 1074 IEEE/EIA 12207 SOFTWARE PRODUCTIVIT YCONSORTIUM

The ‘Frameworks Quagmire’ - What Standards are out there? PSP SW-CMM People CMM FAAi. CMM SDCCR CMMI** SE-CMM SECM (EIA/IS 731) IEEE 1220 EIA/IS 632 * Not released ** CMMI-SE/SW released IPPD not yet released MIL-STD-1679 DODSTDDOD-STD 2168 -2167 A Software Development and IEEE Stds. 730, 828 NATO 829, 830, 1012, 1016 Standards, AQAP 1, 4, 9 Documentation 1028, 1058, 1063 including Lifecycle models. MIL-STDEQA IPDCMM* SECAM SSECMM SDCE SCE ISO 15504 (SPICE) MIL-Q -9858 Trillium DO 178 B DOD IPPD AF IPD Guide Baldrige Tick. IT Q 9000 ISO 10011 BS 5750 498 ISO/IEC 12207 ISO 9000 Series ISO 15288* Also see www. software. org/quagmire EIA/IEEE J-STD-016 IEEE 1074 IEEE/EIA 12207 SOFTWARE PRODUCTIVIT YCONSORTIUM

The ‘Frameworks Quagmire’ - What Standards are out there? PSP SW-CMM People CMM FAAi. CMM SDCCR SDCE SCE ISO 15504 (SPICE) IEEE Stds. 730, 828 829, 830, 1012, 1016 1028, 1058, 1063 Quality, CMMI** SE-CMM NATO AQAP 1, 4, 9 EQA Quality Management Baldrige BS Trillium and IPDQuality Audit Standards 5750 CMM* SECM (EIA/IS 731) SECAM SSECMM MIL-Q -9858 IEEE 1220 EIA/IS 632 * Not released ** CMMI-SE/SW released IPPD not yet released DO 178 B DOD IPPD AF IPD Guide Tick. IT Q 9000 ISO 10011 MIL-STD-1679 DODSTDDOD-STD 2168 -2167 A MIL-STD 498 ISO/IEC 12207 ISO 9000 Series ISO 15288* Also see www. software. org/quagmire EIA/IEEE J-STD-016 IEEE 1074 IEEE/EIA 12207 SOFTWARE PRODUCTIVIT YCONSORTIUM

The ‘Frameworks Quagmire’ - What Standards are out there? PSP SW-CMM People CMM FAAi. CMM SDCCR SDCE SCE CMMI** SE-CMM ISO 15504 (SPICE) IPDCMM* SECM (EIA/IS 731) Integrated Product SECAM (& Process) SSEImprovement IEEE CMM 1220 EIA/IS Standards 632 * Not released ** CMMI-SE/SW released IPPD not yet released MIL-Q -9858 IEEE Stds. 730, 828 829, 830, 1012, 1016 1028, 1058, 1063 NATO AQAP 1, 4, 9 EQA Trillium DO 178 B DOD IPPD AF IPD Guide Baldrige Tick. IT Q 9000 ISO 10011 BS 5750 MIL-STD-1679 DODSTDDOD-STD 2168 -2167 A MIL-STD 498 ISO/IEC 12207 ISO 9000 Series ISO 15288* Also see www. software. org/quagmire EIA/IEEE J-STD-016 IEEE 1074 IEEE/EIA 12207 SOFTWARE PRODUCTIVIT YCONSORTIUM

The ‘Frameworks Quagmire’ - What Standards are out there? PSP People CMM SDCCR SW-CMM SDCE SCE ISO 15504 Quality-Related Standards (SPICE) FAALinked. CMMI** to IPPD & i. CMM IPDBusiness Capability CMM* SE-CMM SECM (EIA/IS 731) SECAM SSECMM IEEE 1220 EIA/IS 632 * Not released ** CMMI-SE/SW released IPPD not yet released MIL-Q -9858 IEEE Stds. 730, 828 829, 830, 1012, 1016 1028, 1058, 1063 NATO AQAP 1, 4, 9 EQA Trillium DO 178 B DOD IPPD AF IPD Guide Baldrige Tick. IT Q 9000 ISO 10011 BS 5750 MIL-STD-1679 DODSTDDOD-STD 2168 -2167 A MIL-STD 498 ISO/IEC 12207 ISO 9000 Series ISO 15288* Also see www. software. org/quagmire EIA/IEEE J-STD-016 IEEE 1074 IEEE/EIA 12207 SOFTWARE PRODUCTIVIT YCONSORTIUM

The ‘Frameworks Quagmire’ - What Standards are out there? PSP SDCCR MIL-Q -9858 MIL-STD-1679 SW-CMM DODSDCE People Software CMM’s and DOD-STD SCE STDCMM 2168 IEEE Stds. 730, 828 Capability Evaluation -2167 A NATO SA-CMM 829, 830, 1012, 1016 AQAP 1, 4, 9 ISO Standards 1028, 1058, 1063 15504 EQA MIL-STD(SPICE) FAACMMI** 498 Baldrige i. CMM BS Trillium IPDEIA/IEEE 5750 CMM* DOJ-STD-016 ISO/IEC DOD 178 B SECM 12207 SE-CMM IPPD (EIA/IS 731) SECAM SSECMM IEEE 1220 EIA/IS 632 * Not released ** CMMI-SE/SW released IPPD not yet released AF IPD Guide Tick. IT Q 9000 ISO 10011 ISO 9000 Series ISO 15288* Also see www. software. org/quagmire IEEE 1074 IEEE/EIA 12207 SOFTWARE PRODUCTIVIT YCONSORTIUM

The ‘Frameworks Quagmire’ - What Standards are out there? PSP SW-CMM People CMM FAAi. CMM SDCCR CMMI** SE-CMM IPDCMM* SECM (EIA/IS 731) SECAM SSECMM MIL-STD -499 B* SDCE SCE ISO 15504 (SPICE) IEEE 1220 EIA/IS 632 * Not released ** CMMI-SE/SW released IPPD not yet released MIL-Q -9858 IEEE Stds. 730, 828 829, 830, 1012, 1016 1028, 1058, 1063 NATO AQAP 1, 4, 9 EQA Trillium DO 178 B DOD IPPD AF IPD Guide EIA 632 Baldrige Tick. IT Q 9000 BS 5750 MIL-STD-1679 DODSTDDOD-STD 2168 -2167 A MIL-STD 498 ISO/IEC 12207 ISO 9000 Series ISO 10011 Systems Engineering ISO 15288* Standards Also see www. software. org/quagmire EIA/IEEE J-STD-016 IEEE 1074 IEEE/EIA 12207 SOFTWARE PRODUCTIVIT YCONSORTIUM

The ‘Frameworks Quagmire’ - What Standards are out there? PSP People CMM FAAi. CMM SE-CMM SDCCR SW-CMM SCE CMMI** ISO 15504 (SPICE) IPDCMM* SECM (EIA/IS 731) SECAM SSECMM SDCE IEEE Stds. 730, 828 829, 830, 1012, 1016 1028, 1058, 1063 NATO AQAP 1, 4, 9 EQA Trillium Baldrige DO 178 B DOD IPPD BS 5750 Tick. IT IEEE 1220 EIA/IS 632 * Not released ** CMMI-SE/SW released IPPD not yet released MIL-Q -9858 ISO 9000 AF IPD Systems Engineering Series Q 9000 Guide MIL-STD-1679 DODSTDDOD-STD 2168 -2167 A MIL-STD 498 ISO/IEC 12207 IEEE 1074 Capability Maturity Models ISO 10011 (including Evaluation. ISO Methods) 15288* Also see www. software. org/quagmire EIA/IEEE J-STD-016 IEEE/EIA 12207 SOFTWARE PRODUCTIVIT YCONSORTIUM

The ‘Frameworks Quagmire’ - What Standards are out there? PSP SDCCR SW-CMM People CMM MIL-Q -9858 SDCE SCE IEEE Stds. 730, 828 829, 830, 1012, 1016 1028, 1058, 1063 NATO AQAP 1, 4, 9 EQA FAAi. CMMI** SE-CMM MIL-STD 498 Baldrige BS EIA/IEEE Integrated Capability 5750 Maturity Models DOJ-STD-016 ISO/IEC (“the whole nine yards”) DOD 178 B SECM 12207 IPPD (EIA/IS 731) IPDCMM* SECAM SSECMM MIL-STD-1679 DODSTDDOD-STD 2168 -2167 A IEEE 1220 EIA/IS 632 * Not released ** CMMI-SE/SW released IPPD not yet released Trillium AF IPD Guide Tick. IT Q 9000 ISO 10011 ISO 9000 Series ISO 15288* Also see www. software. org/quagmire IEEE 1074 IEEE/EIA 12207 SOFTWARE PRODUCTIVIT YCONSORTIUM

Status of ‘Lead’ Standard in each Framework

What EIA 632 is about

EIA 632 Overview

EIA 632 Process Requirements

What EIA-731 is about l Defines Focus (Process) areas generally felt to be of benefit for most businesses l Defines capability levels for each FA l Suggests questions to ask to evaluate whether the maturity level has been reached by the organisation performing the activity l Provides recommended scoring scheme l Provides recommended conduct of an evaluation l Allows tailoring of FA’s and questions to specific business needs

How is EIA 731 arranged? l SE Domain - 3 Categories q 19 Focus Areas §Themes - 1 -5 per Focus area -Specific Practices within each theme l Capability Domain q Generic Practices §Applied to any SE Domain Focus Area / Theme §‘An activity which, when applied…, enhances the capability to perform those practices’ §Grouped by Maturity Level §e. g. GP 2. 2 Verify Compliance with approved plans & processes. . . q Generic Attributes §‘Effectiveness’ and ‘Value’ of products achieved by SE Domain Themes §Descriptions of Effectiveness & Value for each Maturity Level §Applicable to any Focus Area

SECM FAs for TC 1. 0 SE Technical l l l FA 1. 1 Define Stakeholder and System Level Requirements FA 1. 2 Define Technical Problem FA 1. 3 Define Solution FA 1. 4 Assess and Select FA 1. 5 Integrate System FA 1. 6 Verify System FA 1. 7 Validate System

SECM FAs for TC 2. 0 SE Management l l l l FA 2. 1 Plan and Organize FA 2. 2 Monitor and Control FA 2. 3 Integrate Disciplines FA 2. 4 Coordinate with Suppliers FA 2. 5 Manage Risk FA 2. 6 Manage Data FA 2. 7 Manage Configurations FA 2. 8 Ensure Quality

SECM FAs for TC 3. 0 SE Environment l FA 3. 1 Define and Improve the Systems Engineering Process l FA 3. 2 Manage Competency l FA 3. 3 Manage Technology l FA 3. 4 Manage Systems Engineering Support Environment

SECM Capability Levels l Level 0 l Level 1 l Level 2 l Level 3 l Level 4 l Level 5 Initial Performed Managed Defined Measured Optimizing

Level 0 “Initial” Development Process characterised as Chaotic or Ad Hoc General Properties Not all practices accepted as enabling Systems Engineering are performed General failure to perform activities No easily identifiable work products No evidence tasks are accomplished

Level 1 “Performed” Development Process characterised as Informal General Properties All basic practices accepted as enabling Systems Engineering are performed Planning and tracking of practices is inconsistent Dependence upon individual’s knowledge (“heroes”)

Level 2 “Managed”: Characteristics Development Process characterised as Formal General Properties Activities are planned, tracked and verified Processes are defined but programme specific Adherence to specified process is monitored and enforced

Level 3 “Defined”: Characteristics Development Process characterised as Institutionalised / Established General Properties Standard SE development process across the organisation Tailored if required for specific programmes Process metrics are collected Qualitative process improvement occurs

Levels 4 & 5: Characteristics Level 4 Process Improving Measurable quality goals established for work products Processes are quantitatively improved Level 5 Process Optimised Continuous quantitative process improvement

Which model is ‘Best’ to use? l It depends which you are most comfortable with l 731 is very specific - perhaps good if you don’t know where to start, and want a list of possibles to select l CMM’s give you a guideline for process improvement; but perhaps assume a prior level of maturity? l How do you want to measure yourself: At intervals (staged)? q Or continuously - harder work, but quicker feedback q l Let’s hear the speakers l …. And debate it afterwards!