CATE Overview Astro 2020 Large Mission Concepts September

CATE Overview Astro 2020 Large Mission Concepts September 20, 2016 © The Aerospace Corporation 2016

Aerospace Partnership with NAS and Space Studies Board • Provide independent advice to the nation on space exploration and science – NAS Decadal Surveys, CATE • • Astro 2010 Planetary Heliophysics Earth Science, in progress – NASA Space Technology Roadmaps – Human Exploration beyond LEO 2

Background: Astro 2020 Large Mission Concepts 3 • STDT Teams have been formed to develop four large mission concepts for the National Academies Astro 2020 Decadal Survey – Far-IR Surveyor, GSFC – Hab. Ex, JPL – LUVOIR, GSFC – X-ray Surveyor, MSFC • Each STDT team will decide the proposed budget target(s) and associated science value for consideration – Concepts are currently in various stages of development • Aerospace will offer independent guidance to the study teams for consideration in reducing the technical and cost risk of the engineering elements – All four teams will be treated equally and there is no preferred concept – Communicate cost and risk implications of mission architecture choices

What is a CATE? : Cost and Technical Evaluation • CATE developed by NAS/Aerospace for recent Decadal Surveys – Previous Decadal Surveys had no process to validate advocate mission costs – US Congress required NAS to use independently validated costs – CATE estimates needed to reflect historical project growth • CATE process differs from typical ICE and process for TMC evaluation – Begins with typical Independent Cost Estimate, ICE – Adds three types of cost threats, where appropriate: • Schedule, design (mass & power growth) and launch vehicle • CATE is used for future consideration with respect to NASA budgets – Used to evaluate science value versus budget availability • Sometimes used to re-assess Decadal recommended concept descopes – Incorporates typical growth based on the historical record and design maturity • It is more conservative than an ICE of a “specific” concept presented 4

General Limitations of Assessment • Technical risk assessment – Limited to top-level maturity and risk discussions • Not meant to be a Proposal Evaluation level of effort • Cost and schedule assessment – Meant for high-level budgetary estimates – It is understood that the CATE is likely to be higher than advocate estimate – Design growth threat is typically the biggest disconnect with project teams • • 5 Project often defends current specific concept being presented Advocate estimate may not adequately factor in “future” modifications and “growth”

Aerospace is the Custodian for the NAS CATE Process • Requires independent analysis – Reconciliation with Project teams is recommended, where appropriate – However, NAS committees are concerned with advocate team influence • Requires consistency across diverse concepts – CATE process is flexible to handle differences in design maturity • Stay true to the NAS process – Advocate teams and NASA HQ do have special requests – This often can be handled, but the CATE generated S-curve represents the cost risk assessment – There is a “T” in CATE and committees and decision makers need a consistent technical risk assessment 6

Lessons Learned from Exoplanet Probe Support • Get Involved Early – Listen and be educated on desired science and proposed implementations – Provide initial feedback on specific technical deep dives to evaluate risk • Offer Initial CATE Analysis Feedback – Top Technical Risks – Initial CATE full mission cost • Study Teams Should Establish Multiple Concepts for Evaluation – Evaluate range of concept implementation on science goals – Establishes low, middle and higher technical and cost risk range – Allows STDT to assess science value • Focus on Technology Risk Deep Dives and/or Consultations • Provide Intermediate and Final CATE Results 7

Consider Potential Available Funding* $400 M - $500 M could be made available annually in FY 25 and beyond Cumulative in Future Strategic Mission (~area under the curve) $3. 5 B by 2030 $7. 0 B by 2035 * Note: As taken from slide 37 of NASA Townhall Meeting, AAS 227 th Meeting, Jan 2016, as presented by Paul Hertz $5 B mission could take 10 years to fund given $400 M - $500 M available annually

Notional Prioritized Concepts within the Budget Wedge Available Budget Probe #2 CATE: $1, 000 M LRD: 2035 Concept #1 CATE: $2, 500 M LRD: 2033 Probe #1 CATE: $1, 000 M LRD: 2030 Note: Concept #1 profile is NOT optimized for execution 9 Concept #1 CATE: $4, 500 M LRD: 2035 Probe #1 CATE: $1, 000 M LRD: 2030

Technical Risk & Maturity Assessment Approach 10 • Identify key risks to achieving required performance – Highlight significant deviations from current state of the art performance – Trace performance risk to science mission impact – Evaluate potential of planned risk mitigation efforts • Assess technical maturity risk liens on cost and schedule – Assess claimed TRL level of key technologies – Apply mass and power growth contingencies consistent with maturity • Mass growth allowance could result in launch vehicle cost threat – Late technology maturation steps identified as schedule risks – Complex system integration issues identified as schedule risks – Check of relative system complexity with Complexity Based Risk Assessment (Co. BRA) capability

Technical Risk Color Rating Scale Low • Medium Low Medium High Technical risk rating, independent of cost – Blue: Minimal new development, high margins on tech resources, and minimal risk to achieve mission objectives as proposed – Green: Moderate new development, adequate margins on tech resources, and/or moderate risk to achieve major mission objectives as proposed – Yellow: Medium new development, adequate to aggressive margins, and/or medium risk of achieving major mission objectives as proposed – Orange: Significant new development, aggressive to negative margins, and/or significant risk of achieving major mission objectives – Red: High risk of achieving major mission objectives as proposed for one or more reasons

Project X Top Technical Risks and Concerns Project X Technical Risk Rating is Medium • • • 12 Ex am Medium new development, mostly in the engineering implementation – Increase in detector array size – Migration from FPGAs to ASICs – Modernization of heritage instrument control unit Mass margins and power margins are aggressive and launch mass margin is very sensitive to changes in dry mass – Concept design is closer than recommended to Atlas V 551 capacity limit and the system is very sensitive to changes in mass – Several mass liens against concept design Time critical mission operations contributes to medium operational risk – Fault management for autonomous mode requires further definition – Sampling operations and hardware need further definition ple On ly

Project X Mass versus Launch Vehicle Capability 14000 12000 System Mass (kg) 10000 8000 6000 4000 Delta IV H = 13000 kg Ex am ple 2000 0 • 13 Project X Atlas 551 = 8500 kg Atlas 541 = 7900 kg On 10% Launch Margin CATE MEV Launch Mass (kg) Project MEV Launch Mass (kg) ly Project MEV Dry Mass (kg) Project CBE Dry Mass (kg) Project X concept design has smaller launch margin than recommended when applying CATE growth contingency – Critical when on the borderline between LV classes

CATE Cost Estimating Approach Overview Estimate Instruments & Spacecraft Estimate Other Elements Based on historical data Based on probabilistic cost risk analysis Multiple analogies and models Estimate Design Growth Threat Re-run estimate with CATE contingencies 14 Estimate Cost Reserves Estimate Schedule Threat Based on ISE results and project burn rates Integrate Results & Level Across Concepts Cross-check with Co. BRA

The Cost Estimating Relationship (CER) • CERs use regression techniques to establish a relationship between variables that are representative of the design, and cost • CERs can be applied at the system level, subsystem level or component level: – e. g. spacecraft, instrument – e. g. attitude determination & control, optics – e. g. star tracker, CCD Spacecraft Cost ($) CER Spacecraft Mass, Power, Data Rate, Pointing Accuracy, etc. CERs are based on historical data 15 Instrument Cost ($) CER Instrument Mass, Power, Data Rate, # Pixels, etc.

Hardware Cost Estimates 16 • Instruments and spacecraft buses – Multiple analogies are used for each element • Historical flight hardware with known cost, schedule and technical parameters • Analogies chosen based on similarity to proposed item and by supplier – Multiple cost models are also used for each element, as appropriate • Instruments - MICM, SOSCM, NICM, PRICE, SEER • Spacecraft - PCEC, SSCM • Same general philosophy is applied to other hardware elements – Emphasize analogy-based estimates as much as possible – System-level cost models are generally not applicable, but subsystem or component-level models can often be used – Extrapolate from ground-based systems, testbeds, etc.

Example Hardware Estimate Results Notional Results $140 $120 Estimated Cost (FY 15$M) $100 $80 Adjusted Analogy Model $60 Aerospace $40 Project $20 4 A na lo gy lo na A gy 3 2 A na lo gy na lo od M A 17 gy 1 2 el 1 el od A er o M Pr oj ec t $0

Cost Risk Process Overview Used to estimate reserves Multiple Cost Estimates for Each WBS Element Triangular Distribution of Possible Element Cost H DMF: Design Maturity Factor added to high estimate to capture worst case DMF Probability M L L M Lower Most-Likely Limit Total Distribution is a Combination of Cost Distributions for WBS Elements + +. + WBS Element N Cost 70 th Percentile Initial Cost Reserves Estimate WBS Element 2 . . Upper Limit Example Total Cost Distribution Sum of Most-likely Costs WBS Element 1 H Total Cost Probability Distribution

Design Growth and Launch Vehicle Threats • All CATE estimates based on project team inputs – Wide range in the maturity of the designs – Some responses are essentially concept descriptions – Others already have had significant investment maturing the design and required technology • Need to ensure that immature projects didn’t have an unfair advantage – Apply higher mass and power contingencies for less mature projects • Mass and power drive cost estimates from both analogies and models – Use project-supplied contingencies for estimate without threats • • Aerospace-applied contingencies to develop “Design Growth” cost threat Add cost of moving to next larger launch vehicle as “Launch Vehicle” cost threat – If mass contingency results in less than 10% launch vehicle mass margin 19

Analogy Based Schedule Risk Process Overview Multiple Estimates for Each Schedule Phase Triangular Distribution of Phase Duration Probability H M L L M Optimistic Estimate Total Distribution is a Combination of Triangular Distributions for Milestone Durations H Most-Likely Estimate Duration Pessimistic Estimate Example Schedule Distribution Phase B start - PDR + Project Schedule PDR - CDR +. . . + PSR - Launch 20 Total Schedule Probability Distribution 70 th Percentile

Example Cost Risk S-Curve 100% Notional Results Project Estimate Cumulative Probability (%) 90% 80% CATE Estimate 70% 60% 50% CATE Estimate w/o Threats 40% CATE Distribution 30% 20% Project Estimate 10% 0% $ 500 21 $ 750 $ 1, 000 $ 1, 250 Estimated Cost (FY 15$M) $ 1, 500 $ 1, 750

Example Cost Bar Charts Notional Results $1. 4 B $1. 1 B 22

Example Cost Estimate Table Notional Results 23

Conclusions 24 • CATE is a consistent and robust process to evaluate missions for prioritization within a budget constrained profile – Analogies and parametric models are used – Instruments and spacecraft are key drivers and often are under estimated by the projects – Process uses a statistical approach to capture appropriate reserves • Concept maturity • Technology readiness – Process uses historical data to address likely cost threats • Design growth • Schedule • Potential change in launch vehicle • Results are suitable for prioritization and long-range planning