Systems Engineering Execution Boeing Strengthening Systems Engineering Execution

Systems Engineering Execution - Boeing Strengthening Systems Engineering Execution at Boeing …. . And Meeting Customer Needs Dev A. Banerjee Division Director – Systems and Flight Engineering (SAFE) Boeing St. Louis 6 th Annual Systems Engineering Conference October 23, 2003 MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 1 / 22 May 2021

Systems Engineering Execution - Boeing Outline • Assessment of USAF for better SE – Incentivizing Contractors (Excerpts from Dr. Sambur guidance) – Mapping of USAF SE Elements to PMBP, CMMI and SEPM – Joint Do. D/Industry Suggested Actions • Strengthening SE Execution at Boeing – Establish Management Commitment – Provide Infrastructure – Monitor and Support Execution MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 2 / 22 May 2021

Systems Engineering Execution - Boeing Incentivizing Contractors for Better SE Excerpts from Dr. Sambur Memo • • “An immediate transformation imperative for all our programs is to focus more attention on the application of Systems Engineering (SE) principles and practices throughout the system life cycle. ” “A more robust SE environment can only be achieved through joint cooperative efforts with our contractors. ” “ SAF/AQ and all other MDAs will not sign out any future ASPs that lack the necessary and sufficient attention to SE. ” “Additionally, we are identifying ways to improve SE throughout the acquisition process, including workforce issues such as education and training, ; tools such as policies, instructions, and guidance; and institutions such as the proposed Center of Excellence for SE under the Air Force Institute of Technology (AFIT). ” “---the Air Force leadership will take a strong and renewed interest in how our acquisition community addresses SE disciplines and practices. ” Marvin Sambur Assistant Secretary of the Air Force (Acquisition) Policy Memo, dated April 09, 2003 USAF Committed to Robust SE Practice for Future Acquisitions MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 3 / 22 May 2021

Systems Engineering Execution - Boeing Mapping of Air Force SE Elements to PMBP, CMMI, and SEPM – existing pieces of our SE Infrastructure Sambur SE Elements (1) - Examples - Systems Requirements Document Conduct and thoroughness of reviews Test and Evaluation Master Plan (TEMP) Provides fully integrated engineering solutions Effective SE interface control and management Program Management Best Practices CMMI Process Areas Requirements Management System(s) Requirements Development Use of Independent Reviews Systems Integration (Proposed) Affordability System Safety Plan 2. 2 1. 2, 1. 3, 1. 4 1. 2, 1. 5, 1. 6 Product Integration 1. 5, 2. 6 Decision Analysis and Resolution Technical Solution 1. 4 1. 2, 1. 3, 1. 4, 1. 5, 1. 6 System Safety Risk Management Plan Risk Management Configuration Management Plan Baseline Management Systems Engineering Management Plan Organization Life cycle management w/production and retrofit strategic plans Integrated Master Plan Integrated Master Schedule Integrated Planning Integrated Schedules Earned Value Management Source Selection Plan Supplier Management Technical Performance Measurement Closed Loop Corrective Action Contractor Performance Assessment Rating System Requirements Management Requirements Development Verification Validation Interface Definition / Management (Proposed) Trades Studies (Proposed) System Maturity Matrix CMMI Process Area to SEPM Section Management Information System Risk Management 1. 4, 2. 5 Configuration Management 1. 6, 2. 7 Organizational Process Focus Organizational Process Performance Organizational Innovation and Deployment Integrated Project Management (IPPD) Integrated Teaming 3. 1 3. 3 2. 1, 2. 2, 2. 3 2. 1 Project Planning 2. 1, 2. 2, 2. 6 Project Monitoring and Control 2. 2, 2. 5 Supplier Agreement Management Integrated Supplier Management 2. 4 Measurement and Analysis 2. 2 Causal Analysis and Resolution Quantitative Project Management 2. 1 Organizational Process Definition Organizational Training 3. 1 3. 2 Process and Product Quality Assurance 2. 8 Organizational Environment for Integration 3. 4 Help Needed All Program Management Best Practices Apply Program Quality Plan Logistics Management Plan Customer informed of configuration changes and impacts Provide Support Customer Communication/Contact Plan Program Management Best Practice Proposed new PMBP awaiting approval (1) Source Memo: "Incentivizing Contractors for Better Systems Engineering", dated 6 Jan 2003 from Dr. Marvin Sambur, Assistant Secretary of the Air Force for Acquisition MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 General Goals and Processes Suggested new Program Management Best Practice Quality and Mission Assurance Best Practice 4 / 22 May 2021

Systems Engineering Execution - Boeing Suggested Actions from the Joint Do. D / Industry Meeting on SE – 11 April 2003 • SE needs to be considered across the Program Life Cycle • Develop case studies on SE, with both good and bad program examples • Make use of disciplined assessment process to monitor rate and level of SE implementation • Recommend minimum or normal set of SE products or deliverables • Create a consistent set of SE measures/metrics • Define an appeal process and take corrective action if a program deviates from the minimum level of SE needed for sound engineering practice MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 5 / 22 May 2021

Systems Engineering Execution - Boeing Strengthening Systems Engineering Execution at Boeing I. Establish Management Commitment • Command Media • VP Engineering guidance to Chief Engineers • EIB & PM Council direction to Bus • Goals/Metrics II. Provide Infrastructure • Comprehensive SE process assets (processes, tools, templates, etc) • Training, SMEs • Management tools • SE HILT activity across sites Corrective Action MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 III. Monitor and Support Execution • Oversight by SE HILT - SE review of proposals and programs • Disciplined assessment process • Metrics: - Monitoring Rate - Compliance Rate 6 / 22 May 2021

Systems Engineering Execution - Boeing I. Establish Management Commitment • Command Media – “Systems Engineering Process Requirements”- in-work – Systems Engineering Process Documents – Systems Engineering Process Manual • Leadership Endorsement / Direction – VP Engineering advisory to site Chief Engineers emphasizing commitment to SE – PM Council endorsement • Goal – Initiate effort to define goal/metric addressing SE execution Leadership Advisories and Goals will Reinforce Expectations for SE Implementation across Boeing MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 7 / 22 May 2021

Systems Engineering Execution - Boeing II. Systems Engineering Infrastructure • SE Process Assets, Management Tools – – The SEPM outlines detailed requirements for effective SE process CMMI to provide detailed set of process assets (process documentation, templates, etc) Program Management Best Practices (PMBPs) address some key aspects of systems engineering Gaps in Process Assets (e. g. handbooks in specialty areas) to be filled by SE Horizontal Integration Leadership Team (HILT) • Training, SMEs – – – Considerable training in SE process exists throughout Boeing The SE HILT to consolidate SE Training CMMI compliant process assets in development SE HILT Leadership, coupled with Site CMMI activities, will ensure a comprehensive SE infrastructure MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 8 / 22 May 2021

Systems Engineering Execution - Boeing Systems Engineering HILT / Process Action Teams (PATs) Anaheim Canoga Park El Segundo Florida Houston Huntington & Seal Beach Huntsville Long Beach Mesa / Philly Puget Sound St. Louis Wichita System of Systems Survivability / Vulnerability Systems Engineering HILT Process Action Teams (PATs) Network Systems Centric Info. Engineering Affordability & Comm. R, M&T Human Factors Safety Operations Analysis Strengthen/Integrate SE Across Boeing MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 9 / 22 May 2021

Systems Engineering Execution - Boeing R, M&T Human System Interface System Safety Ops Analysis System of Systems (So. S) Survivability/ Vulnerability SE Training Database yone Ever Proj ec SE S t Teams ; peci alist s Sr. S E Lds; s; IPT PMs ; CE s Inter Adv Basic 4 6 7+ Intermediate 2 4 4 1 2 Advanced Formal Edu - Graduate - Workshops - Cross Trng SE Leader Exper (yrs) 4 -8 8 -13 13 -18 Typical Guidance • SE Specialists and Technical Fellows more in-depth training in a selected domain. • IPT Leaders and Program Managers more general training across the SE disciplines. MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 10 / 22 May 2021 # of Courses (Processes & Tools) Affordability Basic Training Topic By SE Discipline SE Analysis & Control Certificate Level Basic Training Advanced Network Centric - Info & Comm Candidates • UCLA Ext • UC Irvine • AFCEA • U of KS Intermediate Source of Training • SE Processes • So. SE Process • Do. D Sys Acq Process • Sys Arch Devel I Te nte ch rn ni al ca Do l. T m ra ain in / In in t g Re er n la al te P d ro Tr c ai es ni sng Ex te rn al Tr ai ni ng • Info & Comm Arch • Networks • Info Operations • Adv Tech Courses Tr a Le ini ve ng l Available SE Training for Network Centric-Info & Comm (Sample)

Available Network Centric Training (Sample) Network -Centric Communications Training Program: Topics Internal Process -Related Training Architectures Networks Information Operations Ø Analog vs. Digital Comm Ø Internet Ø Information assurance Ø Comm Spectrum Ø Interoperability Ø Information Warfare Ø SATCOM Ø Performance Ø C 4 ISR Ø Wireless Comm/ Terrestrial/ Cellular Ø Protocols/controls Ø Information Protection Ø Laser Com Ø Low Probability of Intercept Ø Evolving Technologies Ø Processing Ø Techniques Ø Ø Vulnerabilities Ø Network protection Ø Processing Techniques Advanced/evolving technologies Information/data management Ø Anti- jam Techniques Ø Performance In-house training plans focused in 2 areas: domain/technology training and process training SE Process Requirements Management Risk Management Trade Study Methodology System- of-Systems Engineering Process Detailed System- of - Systems Process Systems OO/UML Process OO Modeling in UML OO Analysis and Design So. SE Tool Kit The Do. D Systems Acquisition Process System Architecture Development High Level Architecture for Simulation DII - COE C 4 ISR Architecture Framework Principles and Applications Network Centric Communications Training Progra External Training Universities / Associations / Gov. Labs / Agencies / Commercial Companies Network -Centric Communications Training Program: UCLA Extension – Short Course Programs Internal Domain / Technology -Related Training Communications; Network Technology; Space Mission Analysis Information & Communication Architectures Satellite Communications Systems Open Architectures Phased Array Antennas Laser Communications Software Architecture RF Subsystem Design Networks Network - Centric Warfare Web Services Information Operations Government Information Technology Thrust Modulation Techniques Working with C 4 ISR Fundamentals of Digital Signal Processing Time and Frequency Reference Concepts Advanced Technology Courses Advanced Satellite Communications Payloads Satellite Network Architectures Advanced Fiber Optics. Satellite Communications Payloads & Systems Design High Frequency Satellite Communications UC Irvine– Certificate Program Communications & Network Engineering (18 units required) Advanced Communications Systems Engineering (18 units required) Armed Forces Communications & Electronics Association (AFCEA) Website courses: http: //www. afcea. org/education/index. htm Aviation & Electronics Schools of America– individual short courses (3 - 6 days) Fiber Optics Systems for RF Telecommunications Network Technologies and Practices Defense Information Systems Agency (DISA) offers COE course University of Kansas -the Information and Telecommunication Technology Center: Lawrence Livermore National Laboratory Software/Commercial Companies offering related technology/software training: Robelle, Sun Microsystems, Borland USA, Object Management Group, etc. Industry Seminars - Networking Cisco Alta Vista- - Search/Meta. Data Topia Entrust - Intelligent Agents - Security

Systems Engineering Execution - Boeing III. Monitor and Support Execution on Proposals / Programs 1. Sites provide resources to execute SE and monitor execution. 2. SE HILT provide ‘integration mechanism across Boeing’ • Two Key Metrics to be monitored: - “Monitoring Rate” Metric • - “Implementation Level” Metric SE Evaluation Templates required for program reviews: - proposal coverage - kickoff planning - program in-process reviews 3. SE HILT analyze metrics with sites; Jointly conduct root cause analyses and develop Corrective Action Plans 4. SE HILT monitor implementation & results of corrective actions • Effective monitoring • Root cause analysis • Implementation of corrective action MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 will strengthen the execution of Systems Engineering… consistent with customer desires. 12 / 22 May 2021

Systems Engineering Execution - Boeing Conduct In-Progress Reviews on Programs RFP System Life Cycle Pre-Concept / Requirements Development Concept Definition Proposal Program Start Preliminary Definition SRR Detail Definition PDR Production Support CDR Ensure SE Integrity Through Tailored Reviews during Major Program Milestones MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 13 / 22 May 2021

Systems Engineering Execution - Boeing The Metrics: - “Monitoring Rate” Metric: Was the SE Process Monitoring Activity conducted? - Measure instances as % of opportunities. Example: At each site, measure the number of proposals reviewed for SE compliance at Gate C as a percentage of the total number of Gate C proposals reviewed by the site. - “Implementation Level” Metric: When the SE Process Monitoring Activity was conducted, what was the level of execution implementation? Example: For each proposal reviewed, evaluate the level of coverage of SE process execution. MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 14 / 22 May 2021

Systems Engineering Execution - Boeing is Committed to SE Execution across the Enterprise MUNROBANERJEEPRESENTATIONSSE_NDIA_23 OCT 03 15 / 22 May 2021