AEGIS Surface Warfare ClosedLoop Systems Engineering Using Operational

AEGIS Surface Warfare Closed-Loop Systems Engineering Using Operational Realistic Test & Evaluation Calvin Foster Mark Davis Robert Doyle Northrop-Grumman IT Deputy Planning Consultants, Inc SUW CSC Advanced Marine T&E Manager Systems Engineer Senior Engineer CDR Mike Good, US Navy PEO Integrated Warfare Systems Littoral Warfare Manager 22 October 2003

AGENDA • Challenge • AEGIS Flight IIA Destroyer Configuration • Closed-loop SE / T&E Process & Team • At-Sea Testing • Summary 2

CHALLENGE • Demonstrate DDG 51 Flight IIA Surface Warfare (SUW) Performance vs. Massed Fast Patrol Boat (FPB) Raid 3

FLIGHT IIA SUW CONFIGURATION Engagement Armed Helo w/Hellfire Missile. 50 Cal MG Phalanx Block 1 B Close In Weapons System Surface Mode 5”/62 Caliber Gun Standard Missile Surface Mode Armed Helo w/Radar, FLIR SPY-1 D Radar SPS-67 Radar Mk 46 Optical Sight System SQS-53 Sonar Sensor 4

WHY “CLOSED-LOOP”? • Directly Link Requirements to Results • Instills Rigor; At-Sea Test Time More Efficient • Ensures Capture of Critical Data • Ensures Operational + Technical Balance • Enhances Visibility of Results – Developer – Program Office – Fleet 5

DOD-STD-2167 A INTERPRETED FOR AEGIS SYSTEM DEFINITION AND DESIGN OPERATIONAL REQUIREMENTS DEFINITION COMPUTER PROGRAM IMPLEMENTATION COMPUTER PROGRAM DEFINITION AND DESIGN COMPUTER PROGRAM TESTING SYSTEM INTEGRATION TESTING PRODUCT BASELINE ALLOCATED BASELINE FUNCTIONAL BASELINE TOP LEVEL REQUIREMENTS OPERATIONAL TEST PLAN, PROCEDURES, REPORT 2167 A PROCESSES SYSTEM REQUIREMENTS ANALYSIS AND SYSTEM DESIGN CPAP SYSTEM DEMONSTRATION & SYSTEM QUALIFICATION TEST (SQT) SOFTWARE REQUIREMENTS ANALYSIS SYSTEM REQUIREMENTS DEFINITION PRELIMINARY DESIGN A SPEC TEST PLAN, PROCEDURES, REPORT MRP DETAILED DESIGN ELEMENT REQUIREMENTS DEFINITION SYSTEM TEST & EVALUATION, MULTI-ELEMENT INTEGRATION & TEST (MEIT) CODING AND CSU TESTING B 1 SPEC TEST DOCUMENTATION CSC INTEGRATION AND TESTING SDR CSCI TESTING COMPUTER PROGRAM PERFORMANCE REQUIREMENTS DEFINITION ENGINEERING TEST AND EVALUATION (ET&E) SYSTEM INTEGRATION AND TESTING TEST MATRICES, PROCEDURES, REPORT PPS, CPRS PDR TPR LEGEND NAVY REVIEW ELEMENT COMPUTER PROGRAM INTEGRATION AND TEST COMPUTER PROGRAM DESIGN PDD, CPDD, IDS, CPDP LOCKHEED MARTIN INTERNAL NAVY INVITED TEST PROCEDURES, REPORT CDR IPR UNIT TEST COMPUTER PROGRAM DETAILED DESIGN MODULE/CSU TEST, PROCEDURES, MDF DDR CR CODE MODULE/CSUs AND DATA MODULE. CSUs AND DATA MULTIPLE ITEMS REVIEWED BY ELEMENT CDR: CRITICAL DSGN REV CPAP: CMPTR PRGRM ACCEPT PNL CPDD: CMPTR PRGRM DSGN DOCMT CPDP: CMPTR PRGRM DEV PLAN CR: CODE REV CSC: CMPTR S/W COMPONENT CSCI: CMPTR S/W CONFIG ITEM CSU: CMPTR S/W UNIT DDR: DETAILED DSGN REV ET&E: ENGNRNG TESTS & EVAL IDS: INTERFACE DSGN SPEC IPR: IN-PROCESS REV MDF: MODULE DEV FOLDER MEIT: MULTI-ELEM INTEG & TEST MRP: MISSION READINESS PNL PDD: PRGRM DSGN DOCMT PDR: PRELIM DSGN REV PPS: PRGRM PERF SPEC SDR: SYS DSGN REV SQT: SYS QUAL TEST TPR: TEST PRGRM REV

AEGIS COMBAT SYSTEMS ENGINEERING PROCESS SEC Test Plan/Execute Issue Closure WSPA Issues Report Out Element/System Test Objective Development Performance Assessment Baseline Definition/CP Development 30 Day WSPA 60 Day Performance Review Board IWS 1 B 1 Dep (Chair) COR (Secretariat) PHD DD LM WAMs ATRC SWDG AEGIS Techrep Retest Requirements LIMS Additional Data Required Ongoing Investigations Training; Physics Impact Assessment TTP Development TTP Viability New Requirements Future Baseline Design Performance Review Board (As Required) IWS 1 B 1 Dep (Chair) DD (Secretariat) PHD COR LM WAMs ATRC SWDG AEGIS Techrep 7

CLOSED LOOP SE / T&E System Engineering Requirements Operational Requirements DT/OT Deficiencies New Capability Previous Test Results Test Objectives (TOs) Measures of Effectiveness Test Kinematics Identify Performance Deficiencies Identify Training Deficiencies Simulation/Validation Retire Completed TOs Report System Performance: Quicklooks Test Objective Assessment WSPA/TESPA Semi-Annual Prog. Office Review WSPR/TESPR TO Assignment to Test Event Scenario Development Performance Predictions Test Execution and Data Analysis Plan Observations Results Test Plan Certification Testing 8

SUW SYSTEMS ENGINEERING COUNCIL AEGIS CSE / T&E SUW SEC “Panel” SEC Admin/Support TO Development Edit/Revise Initial Approval Submit/Input into the TO Database • DMAP • WSPA/TESPA – TORs • Test Reports – Quicklooks – Final Analysis • Sensor Performance • Element Performance Range Requirements IPT Target Requirements IPT Ordnance Requirements IPT Warfare Analysis/M&S IPT DMAP Scenario Certification 1 Week Scenario Pre-Cert 1 -2 Months Draft TP Scenario Finalization Scenario Development TO Approval TO Development TO Assignment Event View Element/AWS Reps/SME IPT 3 Months New/Different TO (Pre-Event & Post-Event) 3 Months Final TP Due To The Ship Formal Report • • Data Analysis WG Target/Ordnance Requirements Surface Warfare Analysis Modeling & Simulation Analyze Event Test Objective WG • • Conduct Event ORG View Scenario WG PEOIWS 1 B 1 C CSE Review More Data Improvements Conclusions & Recommendations 9

SYSTEMS ENGINEERING Test Execution and Analysis Test Objectives • Derived From Operational Requirements Document (ORD) and T&E Master Plan (TEMP) • Match to Validated Threat • Create Measures of Performance • Verify Capability to Test AT-SEA T&E Test Planning • Insert TO’s Into Combat System Ship Qualification Trial (CSSQT) SUW Test Phase • ID & Upgrade Range Capabilities, Target Performance and Data Collection Systems • Confirm All Systems (Ship and Range) Are Ready • Execute Test • Gather Weapons, Range and Target Data • Conduct Quick-Look Analysis Onsite • Conduct Detailed Data Analysis Working Group • Close Loop With Updated Discrepancies and Findings For Future Testing. . . 10

READINESS BUILDING BLOCKS Surface Events Blo cks DDG 84 CSSQT/801 DT HSMST Event AFWTF, Nov 4 -5, 02 GWS, Armed HELO, CDF, CIWS BLK IB, SM-2 uild ing Transfer Targets to AFWTF, DDG 82 CSSQT/801 DT HSMST PMTC, Mar 02 Planned: GWS, Armed Helo, CDF, 5 Tgts Actual: GWS, Armed Helo, CDF, 4 Tgts DDG 81 CSSQT HSMST Event AFWTF, Feb 02 Planned: GWS, 3 Target Actual: Non-Firing/Ships RHIBS dine ss B Conduct Range Risk Reduction Dry Run Oct 10, 02 5 Targets DDG 83 CSSQT HSMST Event PMTC, Mar 02 Planned: GWS, CIWS BLK IB Actual: GWS Non-Firing, CIWS BLK IB Goal: Successful OT Verify Test Plan Can Be Executed, Range Can Support Rea Planned: Range Actual: Engineering Work C 2 & Tactics Range C 2 & Tactics Training Range DDG 84 801 OT HSMST Event AFWTF, Feb 03 GWS, Armed Helo, CDF, SM-2 and CIWS BLK IB Range Risk Reduction With Multiple Targets Provides Introduction of CIWS BLK IB Into Ship Defense Capabilities, Risk Reduction, Complex Scenario, Multiple Elements, Validate Procedure, Command/Control Provides Test Plan Proof of Concept, Complex Scenario, Multiple Elements, Validate Procedure, Command/Control Planned: Actual: Provide Range Risk Reduction Opportunity Identified HSMST Mode Issues 11

DDG 84: HSMST DEVELOPMENTAL TEST Armed Helicopter Engage • • • DT Events 11/4 -5 – Manned Raid – Unmanned Raid – Live Fire Event 11/16 Present 5 HSMST RHIBs – CDF Queue OTH – Armed Helo Search Ship Coordinates Layered Defense: – Helo w/ Hellfire Missiles – STANDARD missile – 5”/62 Gun – CIWS Blk 1 B GWS Hazard Area Standard Missile & GWS Engage CIWS Engage CD FQ ue ue T 1 T 2 T 3 T 4 T 5 12

SUMMARY • Conducting Small Boat Defense Testing Is More Complex Than Most Appreciate • Operationally Meaningful Testing, With Measurable Results, is Critical to ID of Deficiencies/Solutions • End-To-End Detect to Engage Evaluation Requires Holistic Organizational and Technical Approach • Collaboration With Fleet, SWDG and OPNAV Leveraged Limited Resources to Address Critical Capability Build a Little, Test a Little, Learn a Lot 13