NASA OSMA SAS 03 Software Requirements Analysis As

NASA OSMA SAS ‘ 03 Software Requirements Analysis As Fault Predictor Dolores R. Wallace SRS Technologies Software Assurance Technology Center http: //satc. gsfc. nasa. gov dwallac@pop 300. nasa. gov William M. Wilson SRS Technologies/SATC wilsonw@pop 300. nasa. gov SAS 03/Fault. Prediction

Software Requirements Analysis As Fault Predictor Presentation Outline 1. 2. 3. 4. 5. 6. 7. Research Rationale & Objective Requested Project Data Planned Approach Data Acquisition & Analysis Obstacles Results & Findings Conclusions Recommendations SAS 03/Fault. Prediction

Research Rationale & Objective Rationale • The earlier in the lifecycle that software faults can be found the less expensive it is to correct them. • The earliest opportunity to preclude software faults is during the development of the system's requirements. Objective • To determine if software requirements analysis results can be used as a predictor of software faults. SAS 03/Fault. Prediction

Requested Project Data • Requirements Specification – – – Accessible in electronic media in MS Word format for use with the ARM tool Conforming to NASA-STD-2100 Specification statements identified by hierarchical numbers SAS 03/Fault. Prediction

Requested Project Data – Cont. • Verification Test Matrix – Requirements to Test or vice versa - OR • Traceability Matrix – Requirements to Design Elements – Design Element To Software Component - AND - • Test Plans – Tests vs. Software Components. SAS 03/Fault. Prediction

Requested Project Data – Cont. • Failure Reports/Data - Preferably in DDTS format – Including: test id, module id, CSCI id, release/build id, suspected cause of failure and resolution (e. g. , actual cause of failure - what was fixed) • Configuration Control Reports – Baseline item changed -e. g. , Requirement, Design, Code – Reason/Source of change – e. g. , RID, Failure Report, PR, etc. SAS 03/Fault. Prediction

Planned Approach 1. Use ARM tool to analyze projects’ software requirements. 2. Collected projects’ failure data in a DDTS database. 3. Organize the data to represent same components for each project. 4. Conduct correlation analysis. 5. Study results of step 4 to prove or disprove hypothesis. SAS 03/Fault. Prediction

ARM TOOL Reports Identifies and Counts: • Size of Requirements Document – – Lines of Text Numbered Paragraphs Number of Specification Statements Number of Unique Specification Subjects • Number of Specifications Containing – Weak, Optional and/or Incomplete Phrases. – Compound and/or Complex Statements. • Requirements Document Structure – Depth and Distribution of Specifications SAS 03/Fault. Prediction

Data Acquisition & Analysis Obstacles • Common Problems – No commonality or standardization of data or formats across project. – Current projects are reluctant to provide data to outside organizations. – Data from completed projects is incomplete, not in a usable form or is not accessible. SAS 03/Fault. Prediction

Data Acquisition & Analysis Obstacles Cont. • Specific Problems – No project staff available to guide through the mass of documents in the collection – Documents locked on a WEB site – Documents in “. pdf” format – Variation of format and media within project – Complete set of data for a specific Build/Release not available – Data held by support contractor and released only on a “Project Need-To-Know” basis SAS 03/Fault. Prediction

Results & Findings • Data provided by four projects – Projects A and B • Major subsystems of operational information processing system – Projects C and D • Flight instrumentation packages SAS 03/Fault. Prediction

Results & Findings Cont SAS 03/Fault. Prediction

Conclusions • Data from many more projects is required to improve the possibility of finding a number of data sets with sufficient commonality to support analysis. • Analysis approach needs to be expanded to include determining if failures attributed to documentation and design flaws are in reality related to deficiencies in requirements. SAS 03/Fault. Prediction

Recommendations Advocate and support: • The use of NASA-STD-2100 documentation standard • The use of a format to report problems and failures that is compatible with DDTS • The creation and use of centralized Center repositories for data from completed projects. SAS 03/Fault. Prediction

Summary • Lessons – Getting data requires strong, interested Civil servant advocate – Analyzing data requires tedious effort, even with automated tools • Future – Analyze latest set of data – Attempt correlations for the 4 data sets SAS 03/Fault. Prediction