Lesson 13 Evaluate Suitability LOG 211 Lesson 13

Lesson 13 Evaluate Suitability LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -1

Topic 1: Introduction What’s in it for me? Suitability is the degree to which a system can be satisfactorily placed in field use, with consideration given to Reliability, Availability Maintainability, safety, Supportability, technical manuals, manpower, training and other factors. Suitability is principally evaluated through the conduct of the Logistics Demonstration (Log Demo) as part of the Developmental Test and Evaluation (DT&E) and the Initial Operational Test & Evaluation (IOT&E) events. The Log Demo assesses the adequacy and accuracy of the Product Support Strategy and the Product Support Package and quantifies R&M metrics through the conduct of selected maintenance tasks performed by user personnel in the operational environment. Given it’s impact on Operational Effectiveness and Affordability, Suitability is a primary criterion in the Milestone Decision process. LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -2

Considering Suitability Operational Environment Performance Supportability Suitability “the measure of a system’s ability to be supported in its intended operational environment” DAG 9. 4. 5 LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -3

Life Cycle Management Framework Where are you? What Influence Do You Have? Technology Maturation & Risk Reduction Design a Reliable, Maintainable, and Sustainable System LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -4

What is Suitability Analysis? ASOE Model The ASOE Model has four distinct intersections representing program performance and Sustainment objectives. Balancing these priorities requires using TPMs in a series of trade-offs. Technical Performance Process Efficiency Supportability • Functions • Reliability • Production • Capabilities • Maintainability • Maintenance • Support Features • Logistics Life Cycle Cost Total Ownership Cost • Operations LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -5

Evaluate Suitability Lesson Approach This lesson will trace Suitability from the beginning of the Lifecycle Management Framework through Milestone C to the Full Rate Production Decision Review. Select the maintenance tasks and associated metrics, define the support environment, and gather Logistics Product Data Suitability analysis of manuals, training, human factors, MANPRINT, TMDE, tools, spares, repair parts, and task durations in LOG Demo Disseminate and clarify LOG Demo impacts on design and product support strategies through relevant mediums, artifacts and IPTs LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -6

Topics and Objectives TLO: Evaluate Suitability in terms of Supportability and adequacy of Product Support. Topic Objectives Topic 1: Introduction • • Welcome Where am I? What Influence Do I Have? Topic 2: The Path to Suitability • Relate Suitability to Supportability Analysis • Relate Test and Evaluation/Supportability Demonstration to system design, Supportability, and Product Support. • Evaluate Product Support Capability Outcomes from the Supportability Demonstration Assess the impact of Supportability Demonstration on Supportability and Supportability Analysis Topic 3: Measuring Suitability Topic 4: Demonstrate Product Support Capability • • Topic 5: Exercise Distinguish the impact and provide remediation measures when the Log Demo results are deemed not to meet Suitability requirements Topic 6: Summary LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -7

Topic 2: The Path to Suitability LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -8

Measures of Suitability MS A User Need MOE KPP KSA Metric MOP E. g. : • Speed • Payload MOS TPM E. g. : • MTBF • MTTR E. g. : • Reliability • Maintainability LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -9

Design a Maintainable and Sustainable System MSA TD EMD PD Upper Limit Planned Profile Objective Achieved to Date Variance Threshold mit i L r e Low TPMs E. g. : • MTBF • MTTR LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -10

Topic 3: Measuring Suitability LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -11

Traceability to User Needs Technology Maturation & Risk Reduction User Need ICD • • Ao Am Reliability O&S Cost MOP Traceability MOEs KPP/KSA Refine • Speed • Payload TPM MOS • Reliability • Maintainability • MTBF • MTTR TPMs T&E CDD CPD LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -12

Evaluation of Operational Suitability DAG 9. 4. 5 “Operational Suitability should be evaluated in a mission context in order to provide meaningful results. ” Army – Logistics Demonstration (Log Demo) Air Force – Operational Assessments (OAs) Older Term – Maintainability Demonstration (M-Demo) LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -13

Evaluate Suitability Inputs and Outputs Process Inputs Supportability Analyses MTA R&M Analysis FMECA LORA FTA RCM Logistics Demonstration Refine Outputs Recommendations LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -14

Topic 4: Demonstrate Product Support Capability LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -15

Evaluate Suitability: Set Up 1. 1 1. 2 Set Conditions Define Log Demo Data Inputs 1. 3 Define Support Environment Create Developmental Testing Outline Create Reports and Checklists Our planning considers three main areas: 1. 1 Conditions – Select Maintenance Tasks 1. 2 Data Management – Identify the metrics and Logistics Product Data associated with the Maintenance Tasks to be tested 1. 3 Support Environment – Define the environment in which the system will be supported, testing outline, and tools LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -16

Set Conditions Review Associated Analyses for Candidate Tasks LORA Ao $ LPD HFE MTA Maintenance Task Sampling Plan Transportability Analysis FMECA Select Maintenance Tasks to LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 Test 13 -17

Define Log Demo Data Inputs • • LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 All maintenance task MMETs contribute to that task’s MTTR. All maintenance task MTTRs contribute to an overall system level MTTR. Failing to achieve the MTTR threshold means the system cannot be restored to a serviceable condition in the time allocated. Differences in PMET and MMET may indicate trade-space. 18

Define Support Environment Apply maintenance concept constraints, policy and mission realism to Log Demo execution LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -19

Create Developmental Testing Outline Operational Environment Spares MTA Mission Personnel Equipment LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 Spares 13 -20

Create Checklists and Reports LOG Demo Reports System Part RPSTL Tools and Spares Data MMET Sample list from the Logistics/MANPRINT Demonstration Plan for the Single Net Solution – Remote Deployment Device (SNS-RDD) LOG 211 Lesson 13 Evaluate Supportability LCLs can leverage the features within the Logistics Product Database by entering Log Demo results directly and running reports that compare predicted with actual task completion times. After LCLs review the Log Demo Report results, more MTA iterations may result. FINAL v 1. 3 13 -21

Evaluate Suitability: Analysis Our analysis consists of 5 main areas : 2. 1 Analyze Documentation and Training – Inspect technical data consistency among manuals, training, and devices 2. 2 Analyze Human Factors Engineering (HFE) – Examine HFE aspects related to operator and maintainer tasks 2. 3 Analyze Test, Measurement, & Diagnostic Equipment –Trade-off on-board vs. external testing / diagnostics 2. 4 Validate Tools and Spares – for simplicity & effectiveness 2. 5 Assess Task Duration – Determine whether maintenance concept goals are met 2. 1 Analyze Documentation and Training 2. 2 Analyze Human Factors Engineering 2. 3 Analyze TMDE 2. 4 Validate Tools & Spares 2. 5 Assess Task Duration Iterative Analysis LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -22 22

Analyze Documentation and Training Technical Manual Training Plan Train the Troops LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -23 23

Analyze Human Factors Engineering MANPRINT Maintainability: Skill Level + Manpower Accessibility, Testability & Modularity • What skill level of technician is needed? • Is a part accessible? • Will it take more than one person to do the job? • Is the fault detectable? • Are there enough skilled technicians to do the job? • Can the testability determine the failed part without ambiguity? • Can the technician(s) do the job in the time allotted? • Can a part be removed and replaced efficiently and effectively? LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -24 24

Analyze TMDE – Trade-off Test, Measurement, and Diagnostic Equipment (TMDE) are: • Defined in the MTA at each repair level • Used by Organizational Level (less complex tasks) to assist in troubleshooting • Employed by Depots (more complex tasks) as test benches/advanced diagnostics Goals • Comply with Strike Talon Maintenance Concept • Eliminate TMDE where possible MDE External T s/BIT nostic g ia d d r a o On b $ Ao MTTR Logistics Footprint Maintenance Concept LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -25 25

Validate Tools & Spares RPSTL Tools, Spares, and Parts Equipment Standard Field Toolbox Special Tools/ Equipment LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -26 26

Assess Task Duration Log Demo is a timed test to evaluate the Suitability of system support that ensures the right resources, with the appropriate skill level, under the appropriate environment conditions, to achieve the maintenance task in the time allotted. LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -27 27

Iterate Analysis Iterative analysis may be required to: • Test under various environmental conditions • Validate Maintenance Strategy assumptions • Address Maintainability components (accessibility, modularity, testability) issues that may have caused failure to meet MTTR overall OPERATIONAL ENVIRONMENT MMET LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -28

Evaluate Suitability: Report Findings DT&E/Log Demo Our analysis findings fall into 3 categories: Report to the IPT – Report test results to the PSM IPT and make recommendations for design or procedure changes where necessary. Retest as needed. 3. 1 3. 2 Provide Log Demo Report LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 Inform IPTs 3. 3 Update MTA, if needed 29

Log Demo Analysis Report Findings LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -30 30

Report Findings (continued) 3. 1 Provide/Update Log Demo Report 3. 2 Inform IPTs 3. 3 Update MTA, if needed LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -31

Evaluate Suitability: Report Findings IOT&E Technology Maturation & Risk Reduction Report to Congress Recommendations Mitigations Our analysis findings fall into 1 category: Report to Congress– Report test results to Congress and make recommendations changes for design or procedure changes where necessary. Retest as needed. LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 32

Topic 5: Exercise LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -33

What are the implications and mitigation options if you do not achieve MTTR during Log Demo? Key Question Exercise Directions In Blackboard, select the Exercise link in the Lesson 14 folder. Answer the questions in Blackboard based on the two scenarios. Time: 30 mins Group LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -34

Topic 6: Summary LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -35

Takeaways Suitability not only addresses accomplishment of MTTR, etc. , but also determines the system's ability to meet the KPP/KSA requirements for Availability, Reliability, Operation & Support Cost and Mean Down Time. Evaluate Suitability through real-world testing of system support • DT&E/Log Demo o Perform Log Demo prior to Milestone C o Reports to the PSM IPT LOG 211 Lesson 13 Evaluate Supportability • FINAL v 1. 3 IOT&E o Conducted by outside OT&E agency o Reports to Congress 13 -36

Summary TLO: Evaluate Suitability in terms of Supportability and adequacy of Product Support. Topic Objectives Topic 1: Introduction • • Welcome Where am I? What Influence Do I Have? Topic 2: The Path to Suitability • Relate Suitability to Supportability Analysis • Relate Test and Evaluation/Supportability Demonstration to system design, Supportability, and Product Support. • Evaluate Product Support Capability Outcomes from the Supportability Demonstration Assess the impact of Supportability Demonstration on Supportability and Supportability Analysis Topic 3: Measuring of Suitability Topic 4: Demonstrate Product Support Capability • • Topic 5: Exercise Distinguish the impact and provide remediation measures when the Log Demo results are deemed not to meet Suitability requirements Topic 6: Summary LOG 211 Lesson 13 Evaluate Supportability FINAL v 1. 3 13 -37