Validation of Guidance Control Software Requirements Specification for

Validation of Guidance Control Software Requirements Specification for Reliability and Fault-Tolerance Annual Reliability & Maintainability Symposium January 30, 2002 Frederick T. Sheldon and Hye Yeon Kim Software Engineering for Dependable Systems (SEDS) Research Laboratory School of Electrical Engineering and Computer Science Washington State University SEDS Research Group School of EECS, Washington State University

Overview v Goal: Show the feasibility of this analysis approach using a industrial strength SRS to ensure: v Completeness and Consistency v Fault-tolerance v Specification Under Study v A NASA provided Guidance and Control Software (GCS) development specification for the Viking Mars Lander. v Analysis Approach v Using Zed to specify the data v Using Statecharts : Statemate for dynamical analysis v Summary and Future study SEDS Research Laboratory School of EECS, Washington State University

Introduction v Why Requirements Specification? v. Cost, Time, and Effort Defect detected phase Typical cost of correction Requirements Specification Coding/Unit Testing System Testing Acceptance Testing After Implementation $100 - $1, 000 or more $7, 000 - $8, 000 $1, 000 - $100, 000 Up to millions of dollars SEDS Research Laboratory School of EECS, Washington State University

Reliable Specification v. Is Correct v. Complete, consistent and robust v. Can the specification be trusted while minimizing the risk of costly errors? v. How to analyze the specification to prevent the propagation of errors into the downstream activities? SEDS Research Laboratory School of EECS, Washington State University

Consistency and Completeness v Completeness: The lack of ambiguity v. Incomplete if … v… the system behavior is not specified precisely because the required behavior for some events or conditions is omitted or is subject to more than one interpretation. v Consistency v. The Specification is free from conflicting requirements and undesired nondeterminism. SEDS Research Laboratory School of EECS, Washington State University

Fault Tolerance v. Faults v. A fault is a feature of a system that precludes it from operating according to its specification – H. Ammar, B. Cukic, C. Fuhrman, and A. Mili, A comparative Analysis of HW and SW fault tolerance: Impact on software reliability engineering, 1999 v. Fault Tolerance v. The ability to respond to unexpected system failure (detection and mask/recover) SEDS Research Laboratory School of EECS, Washington State University

Guidance and Control Software v Software Requirements – GCS Dev. Spec. v. The system was designed to provide software control of the embedded sensors and actuators of the Viking Mars Lander during the terminal decent (landing) phase. v. ARSP (Altimeter Radar Sensor Processing) v. The ARSP module reads data provided by the altimeter radar sensor to determine the lander’s altitude from the Mars surface. SEDS Research Laboratory School of EECS, Washington State University

Mars Lander trajectories SEDS Research Laboratory School of EECS, Washington State University

Velocity – Altitude Contour SEDS Research Laboratory School of EECS, Washington State University

SEDS Research Laboratory School of EECS, Washington State University

SEDS Research Laboratory School of EECS, Washington State University

Zed Overview v. Clarifying ambiguities v. Identify assumptions v. Correctness checking v. Mathematical proofs v. Giving an in-depth understanding of the SRS SEDS Research Laboratory School of EECS, Washington State University

Statecharts v Visual formalism: Diagrammatic in nature v Testability is provided through symbolic simulation v Predevelopment evaluation through v. Fault Injection v Statemate consists of … v. Activity chart v. Statecharts SEDS Research Laboratory School of EECS, Washington State University

Natural Language based SRS v Inherently ambiguous risking the possibility of multiple interpretations SEDS Research Laboratory School of EECS, Washington State University

Zed Schema SEDS Research Laboratory School of EECS, Washington State University

From NL to Zed v. Discovered Ambiguities v. The confusing definition of variable “Rotation”, and direction of the rotation. v. The type assigned to the AR_COUNTER variable was unclear. v. An undefined 3 rd order polynomial. v. Where the AR_COUNTER should be modified? SEDS Research Laboratory School of EECS, Washington State University

Statecharts Model: Activity chart SEDS Research Laboratory School of EECS, Washington State University

Statecharts Model: Statechart 1 SEDS Research Laboratory School of EECS, Washington State University

Statecharts Model: Statechart 2 SEDS Research Laboratory School of EECS, Washington State University

Some Theory … Set of Inputs ( ) Unknowns ( ) Known Set of Outputs Unsafe Sources: Normal Operation Hardware Failures Human Intervention Models/Simulators SEDS Research Laboratory Known Safe Assumed Safe School of EECS, Washington State University

SEDS Research Laboratory School of EECS, Washington State University

Paradigms … v. Software Failures: “Software does not fail - it just does not perform as intended” Professor Nancy Leveson, MIT v. Design and test for functionality: Also specify what the system should not do. . . then test it! SEDS Research Laboratory School of EECS, Washington State University

Some Theory… lets take a second look Set of Inputs ( ) Unknowns ( ) Fault Injection (added known) Known Set of Outputs Unsafe Sources: Normal Operation Hardware Failures Human Intervention Models/Simulators SEDS Research Laboratory Known Safe Assumed Safe School of EECS, Washington State University

Testing and Fault Injection v. By using symbolic simulation in Statemate v. Boundary Testing v. Input that is inside of the variable range v. Input that is outside of the variable range v. Fault Injection v. State variable alternation v. State transition redirection SEDS Research Laboratory School of EECS, Washington State University

Testing Results ARSP Specification Test Input and Output Input Variable Case 1 Case 2 Case 3 Case 4 Case 5 FRAME_COUNTER 2 2 1 1 3 AR_STATUS - - [0, 0, 0, 0] - [0, 1, 0, 0] AR_COUNTER -1 19900 -1 20000 -1 AR_STATUS KP KP K_ALT KP KP AR_ALTITUDE KP KP Output SEDS Research Laboratory [1, 0, 0, 0] [1, 1, 1, 1] [*, -, -, -] [0, -, -, -] [1, -, -, -] [2000, -, -, -] [1, 0, 1, 0] [0, 1, -, 1] KP School of EECS, Washington State University

Detailed Testing Results - Case 1 Variable Case 1 v Initial values FRAME_COUNTER 2 v Final values AR_STATUS - v Initial variable AR_COUNTER -1 Input AR_STATUS Output K_ALT AR_ALTITUDE SEDS Research Laboratory [1, 0, 0, [1, 1, 0, 0] 0] [1, 1, 1, 1] 1] [2000, -, -, [2000, -] values are being calculated based on the given equations. School of EECS, Washington State University

Fault Injection Results SEDS Research Laboratory School of EECS, Washington State University

Detailed Fault Injection Results Input Variable Case 1 FRAME_COUNTER 2 AR_STATUS - AR_COUNTER -1 AR_STATUS Output K_ALT AR_ALTITUDE SEDS Research Laboratory [1, 0, 0, [1/0, 1, 0, 0] 0] [1, 1, 1, 1] 1] [2000, -, -, [*, 2000, -, -] -] v Change FRAME_COUNTER at CURRENT_STATE from 2 to 3 School of EECS, Washington State University

Summary v Interpretation from NL to Zed v. Clarifying ambiguities v Interpretation from Zed to Statecharts v. Clarifying misinterpreted Zed specification v. Iterative process v Boundary Testing, Fault Injection analysis v. Reveals weak point(s) in the system v. Fault Tolerance validation SEDS Research Laboratory School of EECS, Washington State University

Conclusion v Using this combination of FMs we could: v Clarify ambiguities v Validate Correctness, Completeness, and Consistency v Validate Fault tolerance features of the SRS v This approach enabled us to: v Avoid the problems that result when incorrectly specified artifacts force corrective rework. v Minimize the risk of costly errors being propagated into downstream activities SEDS Research Laboratory School of EECS, Washington State University

Future Study v Build concrete translation rules between the methods v Find an effective algorithm to automate the process v Validate the algorithm for the different (domain/ application specific) critical software requirements v In depth comparative study with other approaches SEDS Research Laboratory School of EECS, Washington State University