Flightlike Flight Software Testing Methods Celina A Garcia

Flight-like Flight Software Testing Methods Celina A Garcia Jet Propulsion Laboratory California Institute of Technology © 2009 California Institute of Technology. Government sponsorship acknowledged.

Introduction § Flight Software (FSW) testing methods used by Mars Science Laboratory (MSL) – Not a new concept, many contributors § What is “flight-like” testing? – Same means & methods used in Mission Operations – Use flight scenarios similar to those defined by Systems to test FSW 11/5/2009 Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 2

Previous FSW Test Methods § Past experience with FSW test methods – FSW Developer: white box testing – FSW Build & Integration: minimal gray box testing – Systems Engineering: black box testing § Normal S/W development, the schedule can absorb any changes due to design flaws § Tight S/W schedules can’t afford to catch design flaws during Systems Eng. & ATLO (Assembly Test & Launch Ops) testing. 11/5/2009 Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 3

Issue § FSW schedule is impacted – Very tight schedule – Unscheduled point releases due to problems found during systems tests § Causes: – FSW team testing did not necessarily include sending commands and verifying telemetry – Dynamic design and/or poor interpretation of requirements & functionality 11/5/2009 Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 4

Solution § Formation of MSL FSW internal test (FIT) team – Objective: Use gray box testing approach to discover FSW problems before Systems Eng. testing does – Employ same interfaces to S/C as Systems Eng. – Use FSW knowledge to test & stress FSW in ways Systems Eng. may not • FSW Event Reporting throttling • Stressing uplink & downlink § Results in fewer point builds & ensures a more stable FSW product 11/5/2009 Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 5

FIT Approach § Gray box testing – “Test as you fly and fly as you test” – Mix of black & white box testing – Going beyond traditional unit testing – Testing against FSW requirements & functionality described in Functional Description Documents – Ensure that FSW works how Systems Eng. expects it to work – Make use of Ground S/W tools • scripting interface 11/5/2009 Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 6

FIT Approach (cont. ) – Test using same venues as Systems Eng. • Workstation Testset (WSTS) – simulated avionics with closed loop simulation for Attitude Control Sys. & Entry, Descent, & Landing • Testbed – combination of simulated and real avionics with closed loop simulation as well 11/5/2009 Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 7

Flight-like FSW Testing Process § FIT testers create functional tests by: – Reviewing Functional Description Documents (FDDs) and FSW design docs – Consulting FSW developer and Systems Engineer – Test script design & test results are reviewed & approved – Testing done using Workstation Simulation Testset and/or Testbed 11/5/2009 Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 8

Initial Results § In progress § Short term benefits: – FSW integration is an easier task near code freeze and release – FSW has access to the FIT test suite – FIT can act as a liaison between Systems, Ground S/W and FSW developer – Provide early feedback to Ground & Simulation S/W – Functional tests become means for regression testing 11/5/2009 Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 9

Contact Info § E-mail: Celina. A. Garcia@jpl. nasa. gov § MSL FSW Build & Integration Lead & FIT team member 11/5/2009 Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 10