2014 SPEC Innovations and Teaching Science and Technology

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Model-Based Reviews The Next Step in Model-Based Systems Engineering Presented by Steve Dam, SPEC Innovations and Jerry Sellers, Teaching Science and Technology, Inc. (TSTI) April 2014 0

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Presentation Outline • What is Model-Based Systems Engineering (MBSE)? • What is a Model-Based Review (MBR)? • MBR Example • Discussion 1

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved What is Model-Based Systems Engineering (MBSE)? “The formalized application of modeling to support system requirements, design, analysis, verification and validation activities beginning in the conceptual design phase and continuing throughout development and later life cycle phases. (INCOSE 2007). ” From SEBo. K • Computer-based modeling and simulation capabilities have been around since the 1960 s • Application of modeling and simulation to systems engineering has been around almost as long (e. g. , SREM in 1969) • In 2007, INCOSE began the MBSE initiative to promote the movement from a document centric SE environment to a model centric one 2

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved MBSE Vision (2007) • Presented at INCOSE International Symposium in San Diego by Sandy Friedenthal, Regina Griego, and Mark Sampson • Goal: Make MBSE the norm by 2020 3

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved MBSE Must Support the Overall Program Lifecycle • Every lifecycle process has milestone or “gate” reviews • Traditionally, ever increasingly large amounts of documents must be produced to support these reviews 4

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved MBSE Implications for Reviews • To support the traditional process, MBSE tools have focused on providing this documentation as an output from the data captured • Benefit: as the data changes, new versions of the report can quickly be produced • Benefit: Data overlap between documents is handled automatically • But do these documents reflect all the information captured in the tool? 5

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved MBSE Implications for Reviews • The simple answer to the previous question is no, but many may consider it “good enough” • The problem with “good enough” is that errors in the model that propagate to the resulting subsystems/components, still may not be discovered until too late • Also, we spend a large amount of time and money preparing for, conducting, and post review “You can not inspect • Reviews are the quality into the product; equivalent of a it is already there. ” product inspection W. Edwards Deming 6

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Model-Based Reviews (MBR) • What would a MBR entail? – Ideally this would be accessing the model in it’s native environment (a tool) – Be able to “walk through” the model to understand how it works, what assumptions are made, what decisions were made and why, what risks these assumptions result – Have a place to provide comments for adjudication • Hence, this is both a process and supporting tool(s) 7

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Example of a MBR • For a Stevens Institute of Technology graduate course, Dr. Jerry Sellers had the students conduct MBRs for the project – Using Inspiration Mars as the system of interest • They conducted two reviews: a preliminary Mission Concept Review (MCR) and final MCR • Used NASA input/exit criteria to set up reviewer guide 8

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Background - SYS 638 • SYS 638: Human Spaceflight Design – Graduate course at Stevens Institute of Technology • The 6 students were given the top-level design concept for Inspiration Mars as our System of Interest – Proposed by Dennis Tito to send 2 humans on a 501 day free-return voyage around Mars and back and asked to analyze the technical merits of the proposed mission. – They were asked to reverse systems engineer the proposed mission and apply human spaceflight design and analysis techniques to validate the proposed architecture. • To support the challenge of conducting a design course 100% online with students spread across the country, we leveraged a new cloudbased model-based systems engineering tool, Innoslate. – Gave us the additional opportunity to research how to perform a model-based design review. – This topic is of keen interest to NASA and other organizations who are increasingly interested moving toward MBSE but who are uncertain as to how to transition from traditional paper-based reviews to model-based ones. 9

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Inspiration Mars Foundation • The i. Mars Foundation is on a mission to. . . • inspire "the next generation of explorers to pursue their destiny through STEM education. " • ". . . push the envelope of human experience, while reaching out to our youth to expand their views of their own futures in space exploration. " • ". . . provide a platform for unprecedented science, engineering and education opportunities, using state of the art technologies derived from NASA and the [ISS]. " • The i. Mars Foundation consists of: Dennis Tito - Foundation Chairman and former ISS visiting astronaut NASA Ames Research Center Paragon Space Development Corp Applied Defense Solutions Space. X • • • www. inspirationmars. org 10

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved i. Mars Mission • "In 2018, the planets will literally align, offering a unique orbit opportunity to travel to Mars and back to Earth in only 501 days. Inspiration Mars is committed to sending a two-person American crew – a man and a woman – on an historic journey to fly within 100 miles around the Red Planet and return to Earth safely. " "The mission’s target launch date is Jan. 5, 2018. This exceptionally quick, free-return orbit opportunity occurs twice every 15 years. After 2018, the next opportunity won’t occur again until 2031. . . It will be financed primarily through philanthropic donations, with some potential support from government sources. " www. inspirationmars. org 11

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Model-based Review Concepts Approach Reviewer Asset-Action Artifact Model Review Diagram Hyperlink 12 Description Reviewers are entered into the model as “assets” and each asset is assigned an “action” within the model. Each action describes the steps the reviewer needs to take to complete the review. Pros • • Review is contained completely within the model Model is directly critiqued Cons • • Model becomes very complex depending on the amount of reviewers Reviewers require good working knowledge of using the model Model elements are exported into document • files that are provided to the reviewers. • Requires no working knowledge of the model Uses a more familiar document-centric review approach • • • Review is held outside the model Model is not directly critiqued Loses the essence of a “modelbased review” A diagram is generated within the model that • can be followed by reviewers. Each object in • the diagram contains the items to be reviewed. The diagram can be either sequential or follow an entrance/exit criteria. Review is contained completely within the model Model is directly critiqued • • Model becomes more complex Reviewers require some working knowledge of using the model Reviewers are given a set of hyperlinks that • directs them to model elements. • • No additional items need to be • created in the model Model is directly critiqued • Allows the option for the reviewer to independently review the model depending on their comfort level Reviewers require little working knowledge of using the model Hyperlinks must be well managed

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Model-based Review Process 13

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Model-based Review Process Step Process Reviewers Receive Project Invite 1 Details • All reviewers were sent an e-mail invitation along with general instructions for the review and Innoslate account setup Reviewers Setup Innoslate Account • All reviewers setup an account with Innoslate O It’s a free subscription • Model owner assigned all reviewers read-only status 2 Reviewers Participate-In/Review Presentation Primer 3 • Team conducted a Webex presentation primer • Reviewers had opportunity to ask questions • Reviewers that could not attend viewed the recording Reviewers Examine MCR Document • MCR document contained the entrance & exit criteria along with all the relevant hyperlinks to the model and a self-assessment of how well the team performed against the criteria • Team created a shared spreadsheet MCR document using Google Drive (AKA Google Docs) and it was discussed during presentation primer • Reviewers examined MCR document 4 5 Reviewers Examine Model • Reviewers used the hyperlinks in the MCR document to navigate the model • Reviewers had the freedom to explore the model deeper once directed through the hyperlink 6 Reviewers Create Comment/Action/Approval • Reviewers left Comments/Actions/Approvals directly in an entity in the model • These were tracked using the ‘export comment report’ feature in Innoslate 7 Reviewers Complete Online Survey • Reviewers completed an online survey through www. surveymonkey. com • Most questions used Likert scaling with the ability to leave direct comments 8 Team Updates Model Based On Feedback • Team reviewed all feedback (model and survey comments) and allocated actions among team members to update the model as required 9 Team Peer Review • Team met in a Webex environment to discuss actions, updates and lessons learned Team Documents Lessons-Learned • Team documented all lessons-learned in MBR paper 10 Team Baselines Model 11 14 • Team baselined model using the ‘export project’ feature in Innoslate • Baseline model become more of an artifact than a means of directly evaluating future updates O Innoslate is working on baseline feature

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Review Navigation Process Road Map Spreadsheet Expanded view Read instructions and click link Instructions Road Map Spreadsheet with hyperlinks to model Link takes to artifact in model Review artifact and leave comments 15

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Artifact Example - WBS 16

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Artifact Example - Con. Ops 17

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Artifact Example - Mission Context 18

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Artifact Example - Capabilities Allocation Action (System Capability) Description included by (Function) generated by (characteristic) C. 0 Capabilities These capabilities span the range of those needed for the i. Mars mission. F. 0 Perform i. Mars Mission C. 1 Mission Duration includes total mission duration, medium and short durations for the i. Mars mission. F. 0 Perform i. Mars Mission C. 1. 1 Short Mission Durations Short durations include single artifacts for the i. Mars mission (i. e. staging, impulses, etc. ). C. 1. 2 Medium Mission Durations Medium durations include all mission segments greater than impulsive (short) durations. C. 1. 3 Total Mission Duration Total mission duration is the entire i. Mars mission duration. C. 2 Propulsion Provide thrust, delta-V, course corrections, breaking burns. Includes propellant loading storage and pressurizations functions. F. 0 Perform i. Mars Mission C. 2. 1 Fueling and Conditioning Includes propellant loading storage and pressurizations capabilities for the i. Mars mission. F. 4 Perform Pad & Launch Operations C. 2. 2 Major Maneuver Delta V Provide thrust and major maneuver delta-V and breaking burns for the i. Mars mission. F. 7 Perform TMI Operations F. 9 Perform Mars Fly-By Operations C. 2. 3 Course Correction Maneuvers Provide course correction maneuvers during transit operations for the i. Mars mission. F. 8 Perform Trans Mars Coast Operations F. 10 Perform Trans Earth Coast Operations F. 11 Perform Earth Arrival Operations CH. 2. 3 TMI Delta V C. 2. 4 Gross Lift-Off and Payload Insertion Mass Captures the Gross Lift-off Mass (GLOM) and Payload Insertion Mass for each system supporting the i. Mars mission. F. 5 Perform Ascent Operations CH. 2. 1 Payload CH. 2. 2 Launch Trajectory C. 2. 5 In-Flight Propellant Storage Provides for in-flight propellant storage capabilities - boil-off and zero boil-off capabilities for the i. Mars mission. F. 7 Perform TMI Operations F. 8 Perform Trans Mars Coast Operations F. 9 Perform Mars Fly-By Operations F. 10 Perform Trans Earth Coast Operations F. 11 Perform Earth Arrival Operations C. 3 Structural Dynamics Methods or mechanisms for separation and jettisons, providing structural integrity, F. 0 Perform i. Mars Mission and withstanding loads exerted during all activities and events for the i. Mars mission. 19 CH. 1. 1 Mission Duration

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Artifact Example - Risk Matrix/Table Risk Description RS. 1. 0 Insufficient Funding Given the very high cost of building and flying a crewed space mission to Mars and the nation's difficult economic challenges, there is a possibility that insufficient funds will be raised to design, build and fly the current mission. This could result in reduced mission scope, lower vehicle performance, and potentially program cancellation. RS. 1. 6 Inability to Use Data Rights Given the lack of legal precedence in obtaining data rights for private human space missions, there is a possibility that the i. Mars program will be unable to legally utilize program (system, scientific, and crewmember) data for public entertainment and education resulting in reduced public engagement and support. This could lead to insufficient public, private, government, and international support and limit the impact of the i. Mars program need to inspire future trips to Mars. RS. 2. 4 Government Collaboration Given the inconsistency of US government support for private space exploration, there is a possibility that the i. Mars program may not be able to reach agreements with NASA and other government agencies interested in space utilization, which could result in the ineffective use of government resources and knowledge of space flight development and operations. This could lead to schedule delays in accessing needed government facilities, resources, and knowledge sources for designing, building, integrating, testing, and operating the launch vehicle element, in-space element, ground and mission operations element, technology development, and communications network. RS. 3. 0 Lack of Communication Coverage Given the long distance between space element and control center, there is a possibility that Throughout the Mission communication will get interfered/lost resulting in delay or loss of data to be transferred between that two. This performance impact could result in delay in space operations or in worst case loss of crew. RS. 3. 1 Unavailability of ECLSS Architecture for Integration by Launch Date Given the lack of proven ECLSS technologies for long-duration space missions, there is a possibility that a mature ECLSS architecture will not be available for integration into the design of the in-space element in time to support the In-Space Element PDR resulting in schedule delays. This schedule impact could result in missing the launch window in January 2018. RS. 3. 1 Long Lead Procurement & Given that tight launch window, there is a possibility of missing it due to long lead procurement and Integration of Launch Vehicle & In-space integration of available heavy launch vehicle. This schedule impact could result in delay in mission Element operations. RS. 3. 5 Inadequate Performance of In. Space TPS During Earth Re-entry Given the high re-entry velocity of the in-space element upon re-entering the Earth's atmosphere during the return leg of the mission, there is a possibility that the in-space element thermal protection system is not capable of withstanding the extreme heat loads which will adversely affect the structural integrity of the in-space element. This risk could lead to break up of the in-space element and crew death during re-entry. An analysis was performed to show that PICA was the right material, but the thickness required can only be calculated with an analysis beyond the scope of a pre-phase A study. There is a risk that the thickness required is greater than 10 inches which is the current manufacturing limit of PICA TPS shields. RS. 3. 5 Crew Safety During Earth Reentry Given the high re-entry velocity experienced during Earth re-entry, there is a possibility that the inspace element re-entry and landing systems may not adequately dissipate/absorb the energy of the inspace element resulting in higher than desired velocity upon water/landing. This could result in crew injury or death upon landing. RS. 4. 5 MMOD Threat to Crew Survival Given the lack of data on MMOD distributions encountered along the way to Mars, there is a possibility that the in-space element may be struck by MMOD with higher than expected momentum resulting in breaches of and damage to the in-space element structure and internal systems. This could lead to permanent damage to critical systems (power, thermal, ECLSS, data, GN&C, etc. ) and potentially risk to crew health and survival. RS. 5. 2 Uncertainty of Crew Psychological Given the extremely confined environment in which the crew must live during the entire 1. 4 -year Effects mission, there is a possibility that the confined working and living environment could create unhealthy conditions for the crew. This could lead to the crew's increased susceptibility to illness and psychological stress, which may ultimately affect crew operational performance and mission success. RS. 5. 2 Effects From Extended Exposure Given the uncertainty in the knowledge of the space radiation environment between Earth and Mars, to Radiation Environment there is a possibility that the in-space element may not provide adequate radiation shielding leading to insufficient shielding from space radiation. This can result in overexposure of the crew to high levels of radiation and impact crew health and safety. 20

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Artifact Example - Analysis Tool 21

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Commenting Tool 22

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Review Board and Feedback Collection • Survey Questions – 1. Have you participated in a traditional, document-based design review before (e. g. , SRR, PDR, CDR, etc. )? (Response: Yes/No) – 2. If you answered "yes" to question 2, please give a brief description of the review process (e. g. , power point charts, formal documents, RIDs, etc. ). (Response: Reviewer Text Input) – 3. Have you ever participated in a model-based design review such as this one? (Response: Yes/No) – 4. If you answered "yes" to question 4, please rate your experience participating in the review. (Response: 1 - 5 Rating) – 5. For this i. Mars MCR, how well did the kickoff presentation address the following? (Response: 1 - 5 Rating) – 6. Any particular thoughts you'd like to share with us regarding question 6? – 7. How well did the SYS 638 design team achieve the MCR success criteria? (Response: 1 - 5 Rating) – 8. So how did this model-based review (MBR) compare to traditional document-based reviews (DBR)? (Response: 1 - 5 Rating) – 9. We want to make model-based systems engineering an effective means of doing design and reviews. What can we do to make future model-based systems engineering reviews more effective and easier to become involved? (Response: Reviewer Text Input) 23

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Lessons Learned • Tool Access – Tool access is a pre-condition for a model-based review – Use of a cloud-based tool GREATLY simplifies the problem of universal access (platform independent, no software to install). – Trying to do our research using a platform-based tool would have been infeasible given the diverse reviewer group. • Tool Knowledge – Most people are generally uncomfortable navigating a new tool (such as Innoslate) for the first time while accomplishing the review process. » Some reviewers wanting to see outputs from the model that they didn’t have to navigate through (even though the majority of the essential review elements were one page hyperlinked diagrams from the MCR walkthrough spreadsheet). » If Model-Based systems engineering reviews are going to thrive, standard tools, standard interfaces, standard products (e. g. Do. DAF) may help. » Absent the above: Reviewers will need some training to be comfortable navigating the tool to allow them to find and interpret information. • Context – Tools tend to excel at capturing detail, but can miss the big picture. – Reviewers (especially those from outside the project) need background and context to interpret review data and products – Providing an overview briefing/video greatly aided reviewers in learning this context. • Design Data – An integrated design spreadsheet was used to capture detailed mission analysis data. This tool was also available for online review. However, again, this provided detail at the expense of context. – A more clear and concise way to present the overall effect of each trade against the baseline architecture would have been a very useful addition to the spreadsheet approach. 24

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Lessons Learned (cont’d) • “Review View” – MBSE tools ofter many views of the system (requirements, assets, functions, etc. ). – A “design review” view would be a helpful addition. – Could follow standard review entrance and exit criteria - tailorable by users - that would allow model artifacts to be linked to it • “Humanizing” the Review Process – A completely remote, model-based review where engineers and reviewers never meet makes the project more abstract. – The real-time presentation given to the reviewers prior to the allowing them access to the model for the review process was generally regarded as a good thing. Some reviewers that were not in attendance and could watch a playback video. – This helped to “humanize” the review and provided a remote introduction between engineers and reviewers. • Real-time Interaction – Some type of real-time forum is helpful to allow the reviewers to ask the design team questions about the model and review products directly in real or near-real time, in addition to simply leaving comments. This system would allow other reviewers to view the question as well, along with the answer, would prevent the same questions from being asked by multiple reviewers. – The i. Mars project team utilized an email distribution system through Google to allow the reviewers to email one email address that would alert the entire design team of their question. It was thought that this would be enough to satisfy any concerns or questions that the reviewers had. Apparently this was not the case per the survey results. 25

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved Observations and Conclusions • Observations – Scalability » Given the novelty of powerful MBSE tools, there is little collective experience on conducting model-centric design reviews for large projects » Additional research to characterize MBSE tool features and model-centric design review processes that enable successfully scaling the approach documented in this study to much larger programs would be helpful – Organizational Adoption » While NASA/ESA and other organizations do use MBSE tools and methods for some projects, the norm is the traditional document-centric approach. • For example, NASA’s Jet Propulsion Lab (JPL) uses MBSE methods for some of its planetary exploration missions. Other NASA centers use MBSE methods either in limited capacity or not at all for larger programs. » Identify a strategy for a particular agency to adopt MBSE methods and tools as the norm for future programs accounting for initial capital and human resource investment, integration with other design tools and processes, and overcoming cultural barriers to adoption. – Organizational Inertia » We like our documents. Some organizations exist to produce a specific document. Model-based Reviews (MBRs) can threaten sacred cows. » Moves to MBRs requires organizations to change their current paradigms for reviews • Conclusions – Model-based Reviews can work! – Model-based Reviews save a HUGE duplication of effort copy/pasting artifacts into Power Point slides – Many issues identified with the MBR are also present in traditional, document-based reviews (e. g. how to navigate through the data) – As organizations migrate to MBSE, the move to MBR becomes a logical imperative. » Failure to do so GREATLY reduces the project-level savings of investing in MBSE in the first place 26