Engineering Elegant Systems Engineering at the System Level

Engineering Elegant Systems: Engineering at the System Level 11 July 2017 Michael D. Watson, Ph. D. www. nasa. gov/sls Consortium Team UAH George Washington University Iowa State Texas A&M University of Colorado at Colorado Springs (UCCS) Missouri University of S&T University of Michigan Doty Consulting Services AFRL Wright Patterson Space Launch System National Aeronautics and Space Administration

Outline u Understanding Systems Engineering • Postulates • Hypothesis • Principles u Systems Engineering Domain • System Integration ‒ System State Variables • Goal Function Tree • State Analysis Model ‒ System Value Model ‒ System Integrating Physics ‒ System Autonomy ‒ Multidisciplinary Design Optimization (MDO) ‒ Engineering Statistics ‒ Methods of System Integration • Discipline Integration ‒ Sociological Concepts in Systems Engineering ‒ Information Flow ‒ System Dynamics ‒ Cognitive Science ‒ Policy and Law in Application • Supporting Processes u Summary 2

Understanding Systems Engineering

Motivation u System Engineering of Complex Systems is not well understood u System Engineering of Complex Systems is Challenging • System Engineering can produce elegant solutions in some instances • System Engineering can produce embarrassing failures in some instances • Within NASA, System Engineering does is frequently unable to maintain complex system designs within budget, schedule, and performance constraints u “How do we Fix System Engineering? ” • Michael D. Griffin, 61 st International Astronautical Congress, Prague, Czech Republic, September 27 -October 1, 2010 • Successful practice in System Engineering is frequently based on the ability of the lead system engineer, rather than on the approach of system engineering in general • The rules and properties that govern complex systems are not well defined in order to define system elegance u 4 characteristics of system elegance proposed as: • System Effectiveness • System Efficiency • System Robustness • Minimizing Unintended Consequences 4

Consortium u Research Process • Multi-disciplinary research group that spans systems engineering areas • Selected researchers who are product rather than process focused u List of Consortium Members • Michael D. Griffin, Ph. D. • Air Force Research Laboratory – Wright Patterson, Multidisciplinary Science and Technology Center: Jose A. Camberos, Ph. D. , Kirk L. Yerkes, Ph. D. • Doty Consulting Services: John Doty, Ph. D. • George Washington University: Zoe Szajnfarber, Ph. D. • Iowa State University: Christina L. Bloebaum, Ph. D. , Michael C. Dorneich, Ph. D. • Missouri University of Science & Technology: David Riggins, Ph. D. • NASA Langley Research Center: Peter A. Parker, Ph. D. • Texas A&M University: Richard Malak, Ph. D. • Tri-Vector Corporation: Joey Shelton, Ph. D. , Robert S. Ryan, Kenny Mitchell • The University of Alabama in Huntsville: Phillip A. Farrington, Ph. D. , Dawn R. Utley, Ph. D. , Laird Burns, Ph. D. , Paul Collopy, Ph. D. , Bryan Mesmer, Ph. D. , P. J. Benfield, Ph. D. , Wes Colley, Ph. D. , George Nelson, Ph. D. • The University of Colorado – Colorado Springs: Stephen B. Johnson, Ph. D. • The University of Michigan: Panos Y. Papalambros, Ph. D. • The University of Texas, Arlington: Paul Componation, Ph. D. • The University of Bergen: Erika Palmer u Previous Consortium Members • • Massachusetts Institute of Technology: Maria C. Yang, Ph. D. Stevens Institute of Technology – Dinesh Verma Spaceworks – John Olds (Cost Modeling Statistics) Alabama A&M – Emeka Dunu (Supply Chain Management) George Mason – John Gero (Agent Based Modeling) Oregon State – Irem Tumer (Electrical Power Grid Robustness) Arkansas – David Jensen (Failure Categorization) ~50 graduate students and 15 undergraduate students supported to date 5

Understanding Systems Engineering u Definition – System Engineering is the engineering discipline which integrates the system functions, system environment, and the engineering disciplines necessary to produce and/or operate an elegant system. • Elegant System - A system that is robust in application, fully meeting specified and adumbrated intent, is well structured, and is graceful in operation. u Primary Focus • System Design and Integration ‒ Identify system couplings and interactions ‒ Identify system uncertainties and sensitivities ‒ Identify emergent properties ‒ Manage the effectiveness of the system • Engineering Discipline Integration ‒ Manage flow of information for system development and/or operations ‒ Maintain system activities within budget and schedule u Supporting Activities • Process application and execution 6

Systems Engineering Postulates u Postulate 1: Systems engineering is product specific. u Postulate 2: The Systems Engineering domain consists of subsystems, their interactions among themselves, and their interactions with the system environment u Postulate 3: The function of Systems Engineering is to integrate engineering disciplines in an elegant manner u Postulate 4: Systems engineering influences and is influenced by organizational structure and culture u Postulate 5: Systems engineering influences and is influenced by budget, schedule, policy, and law u Postulate 6: Systems engineering spans the entire system life-cycle u Postulate 7: Understanding of the system evolves as the system development or operation progresses 7

System Engineering Hypotheses u Hypothesis 1: If a solution exists for a specific context, then there exists at least one ideal Systems Engineering solution for that specific context u Hypothesis 2: System complexity is greater than or equal to the ideal system complexity necessary to fulfill all system outputs u Hypothesis 3: Key Stakeholders preferences can be accurately represented mathematically 8

Systems Engineering Principles u Principle 1: Systems engineering integrates the system and the disciplines considering the budget and schedule constraints u Principle 2: Complex Systems build Complex Systems u Principle 3: The focus of systems engineering during the development phase is a progressively deeper understanding of the interactions, sensitivities, and behaviors of the system • Sub-Principle 3(a): Requirements are specific, agreed to preferences by the developing organization • Sub-Principle 3(b): Requirements and design are progressively defined as the development progresses • Sub-Principle 3(c): Hierarchical structures are not sufficient to fully model system interactions and couplings • Sub-Principle 3(d): A Product Breakdown Structure (PBS) provides a structure to integrate cost and schedule with system functions u Principle 4: Systems engineering spans the entire system life-cycle • Sub-Principle 4(a): Systems engineering obtains an understanding of the system • Sub-Principle 4(b): Systems engineering models the system • Sub-Principle 4(c): Systems engineering designs and analyzes the system • Sub-Principle 4(d): Systems engineering tests the system • Sub-Principle 4(e): Systems engineering has an essential role in the assembly and manufacturing of the system • Sub-Principle 4(f): Systems engineering has an essential role during operations and decommissioning 9

Systems Engineering Principles u Principle 5: Systems engineering is based on a middle range set of theories • Sub-Principle 5(a): Systems engineering has a mathematical basis ‒ Systems Theory Basis ‒ Decision & Value Theory Basis (Decision Theory and Value Modeling Theory) ‒ Model Basis ‒ State Basis (System State Variables) ‒ Goal Basis (Value Modeling Theory) ‒ Control Basis (Control Theory) ‒ Knowledge Basis (Information Theory) ‒ Predictive Basis (Statistics and Probability) • Sub-Principle 5(b): Systems engineering has a physical/logical basis specific to the system • Sub-Principle 5(c): Systems engineering has a sociological basis specific to the organization u Principle 6: Systems engineering maps and manages the discipline interactions within the organization u Principle 7: Decision quality depends on the system knowledge represented in the decision-making process u Principle 8: Both Policy and Law must be properly understood to not overly constrain or under constrain the system implementation u Principle 9: Systems engineering decisions are made under uncertainty accounting for risk 10

Systems Engineering Principles u Principle 10: Verification is a demonstrated understanding of all the system functions and interactions in the operational environment • Ideally requirements are level and balanced in their representation of system functions and interactions • In practice requirements are not balanced in their representation of system functions and interactions u Principle 11: Validation is a demonstrated understanding of the system’s value to the system stakeholders u Principle 12: Systems engineering solutions are constrained based on the decision timeframe for the system need 11

Methods of System Integration Goal: Techniques to Enable Integrated System Design and Assessments by the Systems Engineer

System State Variables Goal: Utilize system state variables to understand the interactions of the system in relation to system goals and system execution

System State Models u System Stage Models represent the system as a whole in terms of the hardware and software states that the system transitions through during operation u Goal Function Tree (GFT) Model • “Middle Out” model of the system based on the system State Variables • Shows relationship between system state functions (hardware and software) and system goals • Does not contain system physical or logical relationships and is not executable u System State Machine Model • Models the integrated State Transitions of the system as a whole (i. e. , hardware states and software states) • Confirms system functions as expected ‒ Checks for system hazardous, system anomalies, inconsistent state progression, missing states, improper state paths (e. g. , short circuits in hardware and/or software design) ‒ Confirms that the system states progress as stated in the system design • Executable model of system 14

System Works Booster – CS Ascent GFT

System State Machine Model u The state analysis model is split into two main components: • Manager software model • System Plant u Modeled using MATLAB Stateflow • Allows the software model to look like the Sys. ML Activity Diagrams • Allows the Systemb. Plant to be modeled as State Machines • Allows those two models to interact with each other within the MATLAB environment ‒ Facilitates the ability to generate custom analysis tools u Reads in command sequence to execute model 16

State Analysis Model for SLS M&FM Commands From Launch Countdown Doc Control (Sys. ML to Stateflow) Sensor Values § 14% of R 12 modeled §Over 7, 200 Transitions in the Vehicle and Software §Over 3, 500 States in the Vehicle Plant (State Machines) Faults Physics Values

System Value Goal: Utilize system state variables to understand the interactions of the system in relation to system goals and system execution

System Value Model u A System Value Model is a mathematical representation of Stakeholders Preferences (Expectations) for the system • The basic structure is straight forward • The sociology/psychology of representing the Preferences can be a challenge u The System Value Model is the Basis of System Validation!!! • The Requirements and Design Models form the basis of System Verification • The System Value Model forms the basis of System Validation u Constructing an SLS Value Model to compare to System Validation results • Can expand to Integrated Stack with input from MPCV and GSDO u System Value model also provides basis for a measure of System Robustness • How many mission types are supported by the system? 19

Integration with the Capability Model Congressional Model Development Cost Capability Value Mission 1 Mission 2 SLS Attributes Mission 3

Production Function (�� − 1)/�� �� /(�� − 1) +Other u�� = �� ∗[�� ∗�� +(1−�� )∗�� ] �� �� Preferences • �� and �� ) �� �� �� = Total output (Best Value from substituting �� • �� = Elasticity of Substitution • �� = Distribution Parameter ‒Value between 0 and 1 • �� �� = Productivity Factor • �� �� = Value of capital/services from Program 1 (SLS) ‒Societal Benefits, Resource Value, and New Information from program 1 • �� �� = Value of capital/services from Program 2 (Other) ‒Societal Benefits, Resource Value, and New Information from program 2

Fundamental Objectives • Value model must quantitatively answer the question: “How does the SLS contribute to NASA’s top-level objectives? ” • Concrete contributions of the SLS are shown below as “sub-goals”. • Note that SLS contributes to the four positive objectives indirectly (for the most part) by enabling missions – the missions themselves are the actual engines of value generation. • Prior work on launch vehicle value models is in the context of specific missions – these value models, while useful, are really value models for the missions, not the launch vehicle itself. Top. Level Objs. Sub. Goals Generate National Prestige Impress other Nations Impress Public the Public Further Scientific Knowledge Enable Missions Increase Military Capability Support Economic Growth Create Jobs & Support Industry Minimize Negative Impacts Minimize Environmental Damage Minimize Loss of Life • A value model for the SLS should value the mission-enabling capability of the vehicle in the generic sense. Sli de -

Constructing the Value Model • A generic value model maps a design, described by an envelope of top-level attributes, to a scalar value. • This model can (and should) be capable of accepting uncertain attributes and propagating them through to output an uncertain value. • This uncertain value can then be adjusted for risk attitude if desired, or (if decision-maker is risk-neutral) the expected value can be used as the final “score” for the design. • A proposed valuation paradigm for the mission enabling capability of SLS should have two distinct components: • Capability model (from system behavioral models – “what can it do? ”) • Value model (from preference elicitation – “how does this benefit us? ”) Design variables, environmental variables, operational variables (includes uncertainties) Lower-level Attributes Value Model Capability Model (physics, software, etc. ) (preferences) Top-level Attributes System Value Sli de -

Constructing the Value Model • Modifications to basic value model framework for SLS: • Because the SLS does not create value directly (instead enabling missions that create value), the basic value model framework must be modified. • Instead of mapping capability directly to value, instead capture all relevant capabilities of the vehicle and interface them with a portfolio of individual mission value models. Design variables, environmental variables, operational variables (includes uncertainties) Lower-level Attributes Value Model Capability Model (physics, software, etc. ) (preferences) Top-level Attributes System Value Sli de -

The Capability Envelope • The first step in constructing the value model is to determine the appropriate abstractions for capturing the capabilities of any given launch vehicle. • What are the “top-level attributes” for a launch vehicle? • A top-level attribute is defined as any system attribute that directly impacts value delivery – or to put it another way, any system attribute that a potential “customer” of SLS might care about. • It is important to identify which system attributes are top-level attributes. • Top-level attributes (such as cost, payload mass to orbit, and weather envelope) are derived from other attributes (such as materials used, engine specific impulse, and controllability) via various behavioral models. • Any given system design is represented by an N-dimensional envelope in top-level attribute space. • This envelope is here called the “capability envelope”. • It is then mapped to value by interfacing with mission value models. Sli de -

The Capability Envelope • The capability envelope is the fundamental abstraction that allows for valuation of any given launch vehicle design. • Why an envelope specifically? • Systems are commonly represented with vectors in attribute space. • “Envelope” implies instead a bounded region of attribute space. • For many top-level attributes, a vector is sufficient to represent capability. • However, certain relationships should be expressed as continuous relationships or envelopes in order to accurately model value. • Definition of “capability envelope”: The Capability Envelope is the combination of all top-level attribute values and all relationships between top-level attributes that fully define any individual design concept. Sli de -

The Capability Envelope • One of the most important parts of the capability envelope is the answer to the question “how much can it carry, and how far? ” • This is expressed as an envelope bounded by the curve representing the relationship between max payload mass and total energy imparted. • Can be expressed as “C 3 vs payload mass” or “DV vs payload mass”. • Converting between the “C 3 vs payload mass” and “DV vs payload mass” curves is trivial as long as the required DV to some reference orbit can be assumed or estimated from aerodynamic and trajectory models. OR Sli de -

The Capability Envelope • Other top-level attributes included in the capability envelope: • Cost (including production, integration, launch, etc. ) • Reliability (probability of mission success for any given launch) • Payload services provided (power, thermal conditioning, etc. ) • Load factors imparted to payload (max Q and during dynamic events) • Shock loads imparted to payload (max Q and during dynamic events) • Controllability envelope (wind speeds vs fairing dims vs Co. G offset) • Allowable payload volumes (related to fairing dimensions) • Time between launches (including assembly time, pad time, etc. ) • Injection accuracy (altitude and inclination) • Ability to operate in loss of communication (autonomy) • Important to capture relationships that may restrict variables, even if the variables may affect different aspects of value. • Example: controllability envelope relates wind speeds (affects value by determining how often vehicle can launch) to fairing dimensions (affects value by determining which payloads can be carried). Sli de -

The Capability Envelope • Note that the capability envelope is quite specifically the inherent capabilities of any given single design (see slide 15). • The capability envelope is not a design trade-space envelope. • The former is an abstraction of a single design in the space of all possible designs, while the latter is a representation of all designs that are possible. • Example: the relationship “max wind speed vs fairing diameter” is part of the capability envelope (assuming multiple fairings exist for a single design), while the relationship “cost vs reliability” is not (unless you’ve designed an SLS that literally allows for trading those off on any given launch). • The capability envelope is not a mission trade-space envelope. • The former is the inherent capabilities of the vehicle, while the latter is related to specific mission planning. • Example: the relationship “max payload mass vs max transfer energy” is part of the capability envelope, while the relationship “flight duration vs transfer energy” (as shown on a porkchop plot) is not. Sli de -

Mapping Capability to Value • This is only the basic framework for the valuation model. • Implementation may vary, but these core elements should remain. • There is not necessarily a “correct” specific implementation of this framework – the implementation is limited in detail only by the designer’s knowledge of their preferences and/or ability to codify them. • The “mission portfolio” valuation paradigm is beneficial because it allows for the consideration of the entire gamut of launch vehicle applications throughout its lifetime. • The following slide presents a general illustration of how the mission portfolio maps the capability envelope to value. Mission A Mission C Mission B • 20, 000 m/s d. V required • 15, 000 m/s d. V required • 32, 000 m/s d. V required • Value = $50000 * m • Value = $30000 * m • Value = $80000 * m • Demand = 25% of total • Demand = 60% of total • Demand = 15% of total

Mapping System Capability to Value & Missions Attempted “Will it work? ” (Reliability) “What can it carry? ” • Load Factors • Shock Loads • Payload Volume • Payload Services • Injection Accuracy “How expensive is it? ” • Production cost • Launch cost • etc. Missions Succeeded Total Value Delivered by Launch Vehicle 31

Mapping Capability to Value • Possible variations on this valuation framework: • Mission Classifications: ‒ While location is used in the example as the sole classifier for missions, it is far from being the only potentially useful classifier. ‒ Missions could also be broken up by purpose (science/military/commercial), manned/unmanned, etc. – it is up to the designer to decide what is most useful. ‒ The important thing is that each unique mission archetype has a characteristic C 3/DV requirement, valuation function, and demand. • Value Curve: ‒ Instead of having a required C 3/DV for each mission and value being a function of mass, value for each mission could simply be a function of C 3/DV and mass. ‒ This is a more complicated formulation that would be much more cumbersome to construct, but it would allow for more realistic representations of certain situations (such as scientific mission to a specific location that can accomplish more with a smaller payload and greater DV than with a larger payload and smaller DV). ‒ Again, it is up to the designer to decide which formulation will best serve the project – in essence trading model fidelity for model effort. Sli de -

Mapping Capability to Value • Possible variations on this valuation framework: • Mission Demand: ‒ Various restrictions could potentially be placed on any given mission’s demand to more accurately reflect real-world limitations – “no more than X launches of Y mission type every Z years”, etc. ‒ These would allow for more realistic modeling of the change in value delivery granted by having a vehicle that can launch more frequently. ‒ Example: It could be the case that no matter how often the launch vehicle can launch, you do not want to (or cannot) launch more than 2 Jupiter missions every 10 years, but would instead do many more LEO and Luna missions). • General: ‒ Various restrictions could potentially be placed on individual missions – “this mission requires at least X kg of payload to be attempted”, that sort of thing. ‒ Any small tweaks which increase the fidelity of the model are welcome, however designers must remain conscious of the increased effort that will result. ‒ It is recommended to start with a low-fidelity model (similar to that which is shown in the example formulation) and refine as needed instead of attempting to construct a highly detailed model from the start. Sli de -

System Physics and System Integrating Physics Goal: Utilize the key system physics to produce an elegant system design

System Integrating Physics u Consortium is researching the significance of identifying and using the System Integrating Physics for Systems Engineering • First Postulate: Systems Engineering is Product Specific. • States that the Systems are different, and therefore, the Integrating Physics for the various Systems is different u Launch Vehicles • Thermodynamic System u Spacecraft • Robotic ‒ Integrated through the bus which is a thermodynamic system • Each Instrument may have a different integrating physics but integrates with the bus thermodynamically • Crew Modules ‒ Integrated by the habitable volume (i. e. , ECLSS) • A thermodynamic system • Entry, Descent, and Landing (EDL) ‒ Integrated by thermodynamics as spacecraft energy is reduced in EDL u Other Thermodynamic Systems • Fluid Systems • Electrical Systems • Power Plants • Automobiles • Aircraft • Ships u Not all systems are integrated by their Thermodynamics • Optical Systems • Logical Systems ‒ Data Systems ‒ Communication Systems • Biological Systems u System Integrating Physics provides the engineering basis for the System Model

Launch Vehicle System Exergy Efficiency S-II Center Engine Cut-Off S-1 C Stage Separation S-II Stage Separation S-1 C Center Engine Cut-Off S-IVB Burn 1 Cut-Off LEO Insertion S-II Engine Mixture Ratio Shift S-IVB Burn 2 Cut-Off Max Q S-!VB Separation 36

Spacecraft Integration Model u 37

Crew Module Exergy Balance: ISS ECLSS u 38

System Autonomy Goal: Establish system interfaces to provide autonomy algorithms with system information necessary and sufficient to manage system

System Engineering of Autonomous Systems u Elegant System Engineering requires • Understanding the Mission Context ‒ System Applications ‒ System Environments (operational, test, abort, etc. ) • Understanding the Physics of the System ‒ System Interactions with themselves and with their environments are governed by their physics ‒ Information Theory provides linkages between physical state representations and actual physical states • Managing the organizational influences on system design and the system context influences on the organization • Understanding Policy and Law Constraints ‒ National Space Policy ‒ International Space Treaties and agreements • Space Debris, Contamination, Property 40

Autonomy in Context: What and Why? u Spacecraft and Surface System Autonomy is the enabling capability for Human Exploration beyond Lunar Sortie Missions • Autonomy is necessary for complex system operations • Timely response to unplanned or unscheduled events u Propulsion, Structure, Thermal Conditioning, ECLSS, Electrical Power, Avionics, RCS, Communication are all understood sufficiently to allow engineered solutions to be reliably produced • Challenges do exist in terms of Space Environmental Effects, efficiency, compact size ‒ Radiation Hardened computer processors needed • Physics and demonstrated solutions are available from which to engineer a vehicle u Operations are sufficiently understood for terrestrial based execution, not onboard execution • Manual operations provide a rich knowledge base of planning and execution processes • Manual operations have a generic template (derived from Apollo/Saturn) applied uniquely to each spacecraft • Terrestrial based manual operations will not support operations beyond 5 light minutes from Earth u Autonomous Operations are essential to Human Exploration of the Solar System 41

Operations Concept Drivers u Small Crew Size (4 -6) • 1 crew member per shift available for vehicle operations • Limited systems experts u Complex Systems • • Nuclear Power and Propulsion Systems Life Support and Environmental Protection USN Attack Submarines are similar complexity systems but have 134 crew members ~525 high level functions to manage an interplanetary crewed spacecraft. u Abort Scenarios • • Unambiguous determination Extremely low latency Fully autonomous/automated (crew incapacitated conditions) Vehicle reconfiguration necessary u Long Communication Latency/Blockages • 15 minutes one way, 30 minutes round trip to Mars ‒ Ground based intelligence not responsive to maintain crew safety • 1 hour blockage by Moon each Lunar orbit u Harsh Environment • Solar flare radiation • Meteorites 42

25 min. 15 min. Moon 1. 28 sec. 2. 5 min. Mars Mercury Venus Earth 1 min. SUN . 387 . 723 1. 0 AU (Not to Scale) 1. 5236

Spacecraft Systems Overview u Beyond Earth Orbit (BEO) crew transport vehicle are comprised of several unique and intricately integrated subsystems • Propulsion • Structure • Electrical Power • Avionics • Thermal Management • Flight control system • Communication and Tracking • Vehicle Management (Guidance, Navigation and Control (GN&C) and Mission and Fault Management (M&FM)) • Environmental Control and Life Support Systems (ECLSS) u Each of these subsystems are driven by unique physics and information theory relationships u Control Theory governs the control of each subsystem both independently and at the vehicle level 44

State Variable Methodology u Goal/Function Tree • State Variable to define System Performance ‒ State variables are defined as inputs and outputs to functions: y=f(x) • x = inputs to the functions f • f transforms the inputs into the outputs y ‒ Goals = Requirements => define intended range of the output state variables y ‒ Failure = state (value) of output state variable y is out of intended range ‒ State variables enforce strong connection of the functional decomposition to the system’s physical laws and causation ‒ The state variables are the connection between function and design—exist in both function and design representations • Allows system to be analyzed in each mission phase and goals which can have different ranges and values for each state variable ‒ Allowed leak rates vary inversely with time from Earth Function f Return date 45

X, V, Θdot, A, Struc T, Cabin T, O 2 LV “Ready”, Struc T, Cabin T, O 2 X, V, Θdot, A, Struc T, Cabin T, O 2 Reach Mars Vicinity Achieve Earth Orbit f Complete Mars Science Mars Landin g X, V y = f (x) Mars Mission simplified GFT Example Functioning Crew Performs Mars Mission & Returns to Earth Deliver to Destination Achieve Mars Orbit Land on Earth A, Cabin T, O 2, Θdot y Keep crew alive x X, V Control Trajectory Θ Control Attitude A, LV Struc T Θdot Maintain Struc Integrity Control Crew Roll Rate provide desired Pos/Vel Meas X Meas V Measure actual Pos/Vel limit accelerations on crew keep capsule intact control structural temp Control Error Between Desired & Measured Pos/Vel Θ Err Control Attitude Error Struc Crack Size, Alloy Purity Control Structure Defects LV Struc A Control LV Struc Loads LV Struc T Control LV Struc Temps P, Cabin T, O 2 Crew A Capsule Struc T X Err, V Err Des X Des V Capsule A, Capsule Struc T provide life support Capsule A control structural loads • Crewed BEO Mission Goal Typ • Transportation • Crew health and safety • Scientific and Technical

Transportation Goals u Position, Velocity, Acceleration u Earth Departure, Mars Departure • Propulsion System • Flight Control System u Interplanetary Coast • Propulsion System • Flight Control System u Planetary Orbital Insertion • Propulsive • Aero Braking u Surface Descent • Propulsive • Aero Surfaces u Planetary Mobility • Drive force • Control System 47

Crew Health and Safety Goals u Provides link between human health and System Performance • Biological • Psychological u Biological State Variables are linked directly with System State Variables • Biological ‒ Heart rate ‒ Respiration rate ‒ Food intake ‒ Water intake ‒ Solid and Liquid waste production rate • Spacecraft Systems ‒ Breathable air (oxygen concentration, carbon dioxide concentration, atmospheric pressure) • Oxygen can be stored as LOX and converted to gas as needed ‒ Drinkable water (mass) ‒ Consumable food (mass) ‒ Solid and Liquid waste processing/disposal (mass) ‒ Vehicle acceleration rates (linear and rotational accelerations) ‒ Crew Cabin/Suit temperature (temperature and humidity) ‒ Activity (work and exercise) and sleep times (hours or minutes / crew day) ‒ Communication System (family communications (email, video, audio), entertainment, etc. ) u Ranges vary with mission phases 48

Science and Technology Goals u Information Return • Communication systems ‒ Transmission rates • radiated power • signal strength • beam width u Sample Return • Containment System (mass, pressure, leakage rate) • Samples (mass) 49

Autonomy Stack u Autonomy must operate consistent with the physical control laws of the vehicle systems u Multiple subsystems exist within the vehicle • Management algorithms must match subsystem physical control laws u Vehicle level integration is a unique set of relationships dependent on the subsystem types chosen • Type of Propulsion • Type of Flight Control System(s) • Type of ECLSS • Type of Electrical Power Generation • Etc. 50

Autonomy Stack u Vehicle Autonomy has 5 distinct functions • Control • Monitoring (sensing) • Performance Determination • Diagnostics • Prognostics Control System FDIR ISHM u Subsystems Autonomy has the same 5 distinct functions • Control • Monitoring (sensing) • Performance Determination • Diagnostics • Prognostics FDIR Control System ISHM 51

Subsystem Management Functions for System Control Performance Monitoring Diagnostics Prognostics Subsystem Control

Autonomy System Stack Mission Objectives & Constraint Data Mission Planning Mission Execution Vehicle Control Vehicle ISHM Prognostics Control Logic Diagnostics Actuation System Control ISHM Detection System

Candidate Autonomous Algorithms for Spacecraft Systems u Several classes of Autonomous Algorithms • Expert Systems • Neural Networks • Bayesian Belief Networks • Model Based Reasoning • Fuzzy Logic u Demonstrated in marine, space, industrial, and aviation applications u Verification and Validation (V&V) approaches will need to be defined for these algorithms, both individually and as an integrated set • Formal V&V Methods (e. g. , model checkers) need to be properly applied • Non-deterministic V&V methods need definition 54

Candidate Autonomous Algorithms for Spacecraft Systems u Expert Systems • Expert rules establish decision structure • Knowledge base contains rules and relationships • Serves well as a central authority where rules/relationships are clearly established • Can be processing intensive with high data storage requirements depending on rules and rule relationship complexities • Well suited for: ‒ Mission Planning, Crew and Mission Constraint Management ‒ Subsystems with clear cut physical equations and well understood interrelationships 55

Candidate Autonomous Algorithms for Spacecraft Systems u Neural Networks • Gradient Descent Methods ‒ Deterministic due to the underlying mathematics ‒ Ideal for nonlinear and interpolative applications/situation • Static Networks ‒ Learning during training operations only ‒ Quality of application based on quality of training cases • Dynamic Networks ‒ Learning during real time operation ‒ Validation and predictability • Implementation ‒ Hardware (fast) ‒ Software ‒ Complexity can be difficult to verify and may require specialized chips (e. g. , ASIC) • Ideal for ‒ Control of highly nonlinear subsystems • Propulsion, Flight Control System transients ‒ Interpolation • Good where there is limited knowledge of complex physical interactions • Real time adaptation in the event of spacecraft subsystem reconfiguration (failure response) 56

Candidate Autonomous Algorithms for Spacecraft Systems u Bayesian Belief Networks • Applies Bayes Rule to Determine System State ‒ Prior States ‒ Current Belief probability • Best employed as an information source for other subsystem or vehicle autonomous algorithms ‒ Helps clarify/validate uncertainty ‒ Aids inference and reasoning (e. g. , augments Expert Systems) • Well Suited for: ‒ Performance Determination • Vehicle • Subsystem 57

Candidate Autonomous Algorithms for Spacecraft Systems u Model Based Reasoning • Models based on extensive domain knowledge ‒ Can leverage design models ‒ Uncertainty based on fidelity of model implemented • Software architecture must address ‒ Efficient Programming Language ‒ Operating System capable of dealing with • Conflict resolution • Efficient processing • Embedded systems for mission critical applications (i. e. , software health management) • Well Suited for: ‒ Vehicle and Subsystem Diagnostics ‒ GN&C (Kalman Filter) 58

Candidate Autonomous Algorithms for Spacecraft Systems u Fuzzy Logic • Classical Mathematical Set Theory • Requires deep knowledge of subsystem physical rules and interactions to properly train • Provides support to Reasoning Systems (e. g. , Model Based Reasoning) • Well Suited for: ‒ Flight Control Systems 59

Autonomous Algorithm Integration u 3 Levels • Mission Execution and Planning • Vehicle Management ‒ Subsystem Integration Based ‒ Physics form basis of subsystem interactions • Form basis of normal or failed states • Subsystem Level ‒ Physics based 60

Autonomous Algorithm Integration u Subsystem Level Autonomy • Keys: ‒ Understanding the physics of the system ‒ Selecting an autonomous algorithm that can • effectively manage the system physics(take the necessary actions based on all interactions) • and responsively manage the system physics (take the necessary action in a timely manner) • System physics are driven by the internal system processes, interactions with other systems, and interactions with the environment, all of which must be managed by the algorithm • System-level algorithm matching involves knowledge of the system transfer functions which include external system and environment interactions ‒ Control Theory is important in implementation. • The physics will define the poles and zeros of the control system and the relative proximity of the system response to these locations. • System Transfer Functions must be defined and matched with the characteristics of the autonomous algorithms 61

Autonomous Algorithm Integration u Vehicle Level Autonomy • Keys: ‒ Integration of the systems autonomous algorithms into a cohesive and response management system ‒ Algorithms taking proper responses to planned and unplanned conditions • Managing the subsystem physics effects on the vehicle are essential ‒ Manage interactions between systems • Vehicle must manage cooperative vs. competitive subsystem responses such that subsystems do not counter each other’s actions leaving the vehicle in a failed state 62

Autonomous Algorithm Integration u Mission Execution and Planning • Keys: ‒ Mission Execution • Manages the total execution of the all mission aspects from a vehicle stand point ‒ Proper knowledge of the current vehicle states ‒ Progress toward specific mission objectives • Mitigates subsystem interaction effects through adjustment to system control parameters in response to specific physical events. ‒ Mission Planning • Based on ‒ Proper knowledge of the current vehicle states ‒ Progress toward specific mission objectives • Conducts Re-planning (with crew approval) to ensure future vehicle states will stay within mission objectives and constraints • Three Levels ‒ Strategic: Earth-based controls will also be involved ‒ Tactical: Crew input and approval ‒ Emergency: Automated to prevent loss of mission, crew, or compromise of crew safety 63

Summary u Human exploration outside of the Earth planetary system (beyond Earth orbit) requires autonomous operation of the vehicle • Communication Latencies • Crew size Limits • Vehicle Complexity u A fully autonomous vehicle of this complexity will require multiple autonomous algorithms working cooperatively within a set of mission objectives and system constraints • The understanding of the physics of the systems, system interactions, and environmental interactions is essential to the system engineering of this complex system • The Goal-Function Tree methodology provides a system engineering approach to define the vehicle state variables and their interactions. u Algorithms at the vehicle level will need to handle future projected states to enable safe mission execution and planning. u Verification and validation approaches will need to be defined for these algorithms, both individually and as an integrated set • V&V will also need to borrow from Formal Methods (e. g. , model checkers) u Applications looking at autonomous system cooperation will be essential to the development of human rated spacecraft operated away from the Earth planetary system 64

System Design and Optimization Goal: Apply system design and optimization tools to understand engineer system interactions

Multidisciplinary Design Optimization Martins, J. R R. A. , Lambe, A. B. , “Multidisciplinary Design Optimization: A Survey of Architectures”, AIAA Journal, Vol. 51, No. 9, September 2013, pp 2049 – 2075

Engineering Statistics Goal: Utilize statistical methods to understand system uncertainties and sensitivities Systems Engineering makes use of Frequentist Approaches, Bayesian Approaches, Information Theoretic Approaches as appropriate

Maximum Likelihood Estimate (MLE) u

Maximum Likelihood Estimate (MLE) u 69

Maximum Likelihood Estimate (MLE) u 70

Information u Information is viewed as the number of meaningful parameters • Parameters with sufficient measurements to be reasonable estimates ‒ There is an optimum fit for a given set of models used to evaluate the same phenomena • Too few parameters limits the information • Too many parameters (with respect to the data measurements made) leads to large uncertainties in the parameter value and therefore limited information is contained by the parameter u Fisher Information Matrix • Defines information as the matrix of partial second derivatives ‒ Information is the amount of parameters with non zero values (so provides an indication of structure) ‒ This value converges to a maximum as the number of parameters goes to infinity ‒ Does not contain an optimum, always increases with added parameters 71

Akaike Information Criteria u 72

Akaike Information Criteria u 73

Akaike Information Criteria u 74

Optimal Sensor Information Configuration u Applying Akaike Information Criteria (AIC) corrected (AICc) to assess sensor coverage for a system u Two Views of Information Content • AIC Information ‒ Information is viewed as the number of meaningful parameters • Parameters with sufficient measurements to be reasonable estimates • Fisher Information Matrix ‒ Defines information as the matrix of partial second derivatives • Information is the amount of parameters with non zero values (so provides an indication of structure) • This value converges to a maximum as the number of parameters goes to infinity • Does not contain an optimum, always increases with added parameters u AIC/AICc has an adjustment factor to penalize sensor arrangements where: number of sensors < 3 x(number of measurements) u Provides an optimization tool for use with System Models 75

Flat Plate FEA Analysis and Akaike Information Criterion (AIC) • Results: DOFs kept, Method 1 • Overlaid on mode shapes From Paper: From my FEA analysis and overlay of Method 1 DOFs: Red circle: Method 1 DOFs kept Black circle: DOFS removed due to clamped BC Exergy-Based Information for Systems Analysis, Doty Sli de -

Flat Plate FEA Analysis and Akaike Information Criterion (AIC) • Numerical results (decomposed) corrected Akaike Information Criterion (AICc) allocation of information Source Bias Correction Small Sample Correction Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 n dofs 32 30 76 36 28 116 K 33 31 77 37 29 117 SSerror 1428. 37 1461. 59 1428. 91 857. 73 1762. 84 1092. 14 Bias 606. 56 616. 70 606. 72 381. 65 699. 34 488. 20 2 K AIC 2 nd order correction AICc 66 62 154 74 58 234 672. 56 678. 70 760. 72 455. 65 757. 34 722. 20 5. 51 4. 85 33. 09 6. 98 4. 23 85. 49 678. 07 683. 55 793. 81 462. 63 761. 57 807. 68 Bias Exergy-Based Information for Systems Analysis, Doty Bias Correction Small Sample Correction Sli de -

Flat Plate FEA Analysis and Akaike Information Criterion (AIC) • Bias Correction Small Sample Correction Total Corrections Sli de -

Flat Plate FEA Analysis and Akaike Information Criterion (AIC) Sli de -

Optimal Sensor Information Configuration u Applying Akaike Information Criteria (AIC) corrected (AICc) to assess sensor coverage for a system u Two Views of Information Content • AIC Information ‒ Information is viewed as the number of meaningful parameters • Parameters with sufficient measurements to be reasonable estimates • Fisher Information Matrix ‒ Defines information as the matrix of partial second derivatives • Information is the amount of parameters with non zero values (so provides an indication of structure) • This value converges to a maximum as the number of parameters goes to infinity • Does not contain an optimum, always increases with added parameters u AIC/AICc has an adjustment factor to penalize sensor arrangements where: number of sensors < 3 x(number of measurements) u Provides an optimization tool for use with System Models 80

Flat Plate FEA Analysis and Akaike Information Criterion (AIC) • Bias Correction Small Sample Correction Total Corrections Sli de -

Verification Process Method 1: ‘Intelligent’ Guess • Final Solution: Overlaid on Peaks Projected to XY plane Note: 2 initial guesses ‘removed’ (red) NEW points added (blue) MOST initial guesses ‘survive’ (green) Exergy-Based Information for Systems Analysis, Doty Slide - 82

Sensor Location • Sensor Placement is determined by locations of highest residual error • Indicates lowest level of information about the system • System model allows determination of highest residual error location • Must properly model physics of the system to be measured and associated interactions • Placing the first sensor here changes the information available and biases all other locations • Provides keystone for locating sensors appropriately • Provides an objective method to determine proper sensor measurement locations 83

Methods of System Integration Goal: System Design and Analysis

System Models Contain an Understanding of the System Value Model • State Variables Engineering Statistics Goal Function Tree (GFT) System Functions & State Variables Goals System Functions & State Variables System State Transition Model Discipline Physics Models System Integrated Physics Model (System Exergy) Multidisciplinary Design Optimization (MDO) Allow systems engineers to: • Define system functions based on the system state variables • Understand stakeholders expectations on system value (i. e. , capabilities) • Integrate discipline engineering models into a system level physics based model (e. g. , system exergy) • Design and Analyze system responses and behaviors at the System level • Magic. Draw Enterprise (Sys. ML) • Matlab State. Flow • Microsoft Excell

System Design and Integration

Methods of Engineering Discipline Integration Goal: Understand How Organizational Structures influence Design and Operations Success of Complex Systems

Sociological Concepts in Systems Engineering u Specification of Ignorance is important in the advancement of the understanding of the system u Consistent use of Terminology is important for Communication within the Organization u Opportunity Structures • Provide opportunity to mature ideas ‒ Task teams, working groups, communities of practice, etc. u Socially Expected Durations will exist about the project u Both Manifest and Latent Social Functions exist in the organization u Social Role Sets • Individuals have a set of roles for their position u Cultural Subsets will form • i. e. , disciplines can be a subset within the organization • Insider and Outsider attitudes can form ‒ Be Aware of the Self-Fulfilling Prophecy, Social Polarization u Reconsiderations Process (i. e. , Reclama Process) • Provides ability to manage social ambivalence • Must be able to recognize social beliefs that may be contributing to the disagreement • Helps to avoid putting people in to social dysfunction or complete social anomie ‒ Conformity ‒ Innovation ‒ Ritualism ‒ Retreatism ‒ Rebellion 88

Information Flow u Information Flow through a program/project/activity is defined by Information Theory • Organizational communication paths • Board Structure u Decision Making follows the First Postulate • Decision Process is specific to the decision being made • Tracked 3 SLS CRs, with 3 separate task team processes, all had equally rated effectiveness u Margin is maintained by the Organization, not in the margin management tables • Biased Information Sharing • Margin Management is focused on Managing the Disciplines (informed by the System Integrating Physics) u SLS Organizational Structure was defined by the LSE as a recommendation to the Chief Engineer and the Program Manager 89

Decision Structure Information Flow u 90

SLS Organizational Structure Modeling u Interviewed 12 Marshall engineers/designers (w/J. Shelton) • Understand strategies used to integrate subsystems with each other u Common strategy across subsystems – margins • Keep some percentage of a parameter in “back pocket” as hedge for future negotiations • Biased Information Sharing • (Here, “margins” different from “safety margin”) u How does maintaining a margin affect optimality of the final design? • Model as simple 2 Player System with 3 design parameters • 15 problem test suite 91

Simulation Results u Descending margin, �� =1. 3−. 1∗�� until �� =1 Static margin, m= 1. 3 - No margin condition reaches optimality quickest Descending margin still reaches optimal, but requires more iterations Margins are an issue - Interviews highlight real-world consequences - Simulations quantify extent of the problem - Still possible to achieve optimal design with descending margin, but takes additional time to achieve Slide - 92

Discipline Integration Models Organizational Values Value Model Goal Function Tree (GFT) Goals • Organizational Structure & Mapping Value Attributes System Functions Agent Based Model (ABM) Discrete Event Simulation System Dynamics Model Allow systems engineers to: • Understand information flow through the development and/or operations organization • Integrate discipline information into a system level design • Analyze information flow, gaps, and blind spots at the System level • Magic. Draw Enterprise (Sys. ML) • Matlab State. Flow • JAVA • Anylogic • Extend

What is System Dynamics? • A methodology that has been used extensively in business and engineering environments to develop better understanding of the interactions that define the operation and behavior of complex systems • Based on the concept that complex systems can be subdivided into simpler, more manageable components • The high-level operation of an overall system can be defined by examining and defining the behavior and interactions of the individual components. “While it is often difficult for scientists to intuitively understand the overall operations of complex systems there is often a great deal of in-depth knowledge on the operations of the structural components that constitute the system. ”

Tools and Methodologies • Tools and techniques have been developed using the System Dynamics methodology that make it possible to efficiently decompose complex systems and to quickly set-up and test models of system operation. • Tools promote understanding through visual diagramming and modeling.

Roles of System Dynamics • Creates a Structured View of the Problem Space. • Allows Visualization and Testing of Assumptions and Postulated Relationships. • Dynamically Traces Potential Results of Modifications. • Communicates Operational Understanding to Others.

Strategic Analysis n n System Dynamics is frequently used to support long-term strategic decision-making Modeling of strategic decision space often requires at least some tactical level modeling Art is in decomposing model to a level that captures driving relationships but is not too complex to accurately or efficiently model n Stochasticity is often an important driver in system behavior Typical strategic models are run probabilistically, with major drivers represented as prob. distributions

Applications § Methodology has been applied across many disciplines which involve complex systems with limited high-level operational understanding: - Business and monetary systems Power generation and distribution Information technology networks Aircraft /spacecraft design and operation Nuclear reactor operation Military operations NASA STS/ISS Transportation/Operations Analysis • Develop a methodology to evaluate strategic level options for the re-supply and operation of the ISS-STS system • Evaluate various potential transportation systems • Analyze impact (financial and performance) on Exploration System

STS-ISS Transportation / Operation Analysis

ISS-STS Transportation / Operations Analysis

Cognitive Science u Research Goal: Identify some of the key cognitive and organizational challenges in engineering complex systems and the implications to Systems Engineering • University of Michigan, Design Science ‒ Topic: Cognitive Science Perspective of Systems Thinking • Mapping Engineering Terminology to Cognitive Science Terminology to provide a scientific basis for the engineering cognitive concepts • Investigating Mediated Learning as a method to teach system thinking Cognitive Competencies from Frank, 2012 Related Concepts from Cognitive Psychology Understand the whole system and see the big Sensemaking; information integration; mental model formation; picture generalization Understand interconnections Induction; classification; similarity; information integration Understand system synergy Understand the system from multiple perspectives Think creatively Understand systems without getting stuck on details Understand the implications of proposed change Understand a new system/concept immediately upon presentation Understand analogies and parallelism between systems Understand limits to growth Ask good (the right) questions (Are) innovators, originators, promoters, initiators, curious Are able to define boundaries Are able to take into consideration nonengineering factors Are able to “see” the future Are able to optimize Deductive inference Perspective taking (direct mapping) Creativity (direct mapping) Abstraction; subsumption Hypothetical thinking Categorization; conceptual learning; inductive learning/inference Analogical thinking (direct mapping) Information integration Critical thinking Inquisitive thinking Functional decomposition Conceptual combination Prospection Logical decision-making 101

Space Policy and Systems Engineering u Impact of Government Oversight Time Allocation Study • Motivation: Industry and government leaders agree that government oversight leads to cost growth, but there is less agreement on how much and through what mechanisms. • Research Plan: ‒ Build an empirical basis for measuring the extent and nature of the impact of oversight ‒ Non-invasive “Time Allocation Study: ” Statistically valid aggregated observations of how engineers actually spend their time throughout a product’s life cycle. • Part One: Collect time-recall diaries to develop a composite list of activities performed • Part Two: Survey Population over several months at random times per day to accurately observe amount of time spent on activities • Data collection is complete and analysis is in process ‒ Most non-value added oversight is internal company driven ‒ Government generated insight/oversight is a small % of work done (< 10%) ‒ Corporate Communication and Administrative work drive non-value added work from viewpoint of practicing systems engineers within the company 102

Percentage of total time spent on each oversight category Brainard, S. M. , Zsajnfarber, Z. , “Understanding the burden of government oversight on engineering work: adding empirical data to the debate”, submitted to Space Policy 103

System Engineering Supporting Activities Process Application and Execution for the Specific System

Processes u Well defined in NASA/SP-2007 -6105 Rev 1, NASA Systems Engineering Handbook u SEMP is essential to capture appropriate application of processes to the specific system • Process application is specific to the system being developed ‒ Tailoring is not a special exception, it is the norm

System Engineering Standards in Practice 106

UAH SE Consortium - Comparing the Relationship between Systems Engineering Process and Project Success in Commercial and Government Research and Development Efforts, 2012 – 2014. Agriculture Aerospace Defense and security Transportation Communications Electronics Energy Infrastructure Processes with > 3 Correlations ≥. 4 Processes with < 3 Correlations ≥. 4 Original Study Correlations 107

UAH SE Consortium - Comparing the Relationship between Systems Engineering Process and Project Success in Commercial and Government Research and Development Efforts, 2012 – 2014. Processes with > 3 Correlations ≥. 4 Processes with < 3 Correlations ≥. 4 Original Study Correlations 108

Engineering the System 1. 2. 3. 4. 5. 6. 7. 8. a. b. c. d. e. Understand the System Application Define System Architecture and Mission Requirements Design the System Model the System Understand the System Physics, Environments, and Interactions Understand System Sensitivities and Uncertainties Integrate Discipline Designs Maintain Technical Solution within Cost and Schedule Test the System Development Verification Validation Support System Fabrication Support System Operations Mission Context Technical Integration (Physics Basis Focus) System Maintenance Responses System Obsolescence Upgrades System Capability Upgrades for New Applications System Decommissioning Planning and Execution Manage Flow of Information Through Organization Configuration Management Technical Data Management Recommend Board Structure based on Information Flow Advocate Opportunity Structures Support reclama path 9. Manage System Engineering Processes as Defined in SEMP 10. Understand Policy and Law Impacts on Technical Solution Organizational Structure & Information Flow Policy & Law 109

Summary u Discussed approach to Engineering an Elegant System u Discussed Systems Engineering Framework and Principles • System Integration • Engineering Discipline Integration u Discussed several methods and tools for conducting integrated system design and analysis • System Integration ‒ System Integrating Physics ‒ System Design and Optimization ‒ Engineering Statistics ‒ State Variable Analysis ‒ System Value • Discipline Integration ‒ Sociological Principles and Cognitive Science ‒ Decision Making ‒ Policy and Law Application • Processes Application u Systems Engineering Approach defined in two documents • “Engineering Elegant Systems: Theory of Systems Engineering” • “Engineering Elegant Systems: The Practice of Systems Engineering” 110

Backup 111

System Engineering Processes 1. Stakeholder Expectations 2. Technical Requirements Definition a. 3. 4. 5. 6. Product Validation 7. Product Transition 8. Product Operation and Sustainment 9. Technical Planning a. b. c. Technical Risk Management Technical Assessment Decision Analysis 10. Configuration Management a. b. c. Policy & Law Logical Decomposition Design Solution Definition Product Implementation Product Integration Product Verification a. Mission Context Technical Data Management Requirements Management Interface Management Technical Integration (Physics Basis Focus) Organizational Structure & Information Flow Focus on the intent of the processes not the processes themselves 112