Goddard Space Flight Center Greenbelt MD Charlie Robinson

Goddard Space Flight Center - Greenbelt, MD Charlie Robinson / NCAS Quality Assurance and Risk Management Services, Inc. 28 -30 October 2008

AGENDA PROCESS Ø Preparing for an Assessment RESULTS Ø Using Historical Data LESSONS-LEARNED Ø Document Control Ø Contracts Ø Software Ø Internal Audits SUMMARY

Preparing for an Assessment AS 9100 Assignments E-mails to Team & NASA Rep. Team Conference Call General Information Assessment Blank Plan Assessment Card Evaluation Form In-Briefing Assessors Input Template Attendance Sheet The Assessment Team Takes Pre-Assessment Planning Seriously

Using Historical Data 100% NCAS Data - Percent of Suppliers with one or more Issues in the Category 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% DC C CA T CM AI SI ED ES P MC Q NC R CP O GF RM GP CR DA WI SS SP SR AT E DP FA NCAS – 55% (18 out of 33) have Software problems (NOV 06 – SEP 08) SAC – 89% (32 out of 36) had Software problems (FEB 02 - JUN 05) 10 Suppliers have been assessed 2 to 4 times since FEB 2002

Using Historical Data (continued) HIGH Identify Requirements R I S K Ø Items produced & delivered before identifying the Environment & Contamination Requirements Ø Contractually required Stress Relieve Procedure not developed. Ø Unaware of DPAS rating LOW Average Small Medium Large University Ø Wiring ‘Crimped” – Contract requires “Welding”

Using Historical Data (continued) HIGH Software R I S K Ø No Configuration Management Process Ø Amendment to Instrument Coding Standard made without Peer-Review Ø Life-cycle metrics not distributed as required by contract Ø No Software Safety Program LOW Average Small Medium Large University Ø Coding Standards in SDP not used. Ø A “Draft” Satellite Simulator Software is being used.

Lessons-Learned Document Control 47 Non-compliances & 37 Observations Internal Conflicts between procedures Ø Matrix of procedure inter-relationships is a way to avoid these conflicts Industry Specifications not current Ø A consolidated listing of all industry specifications that are maintained Procedures not maintained to reflect current practices Ø Supervisor responsibility & Internal Audits should provide direction and assurance that procedures are current

Lessons-Learned Contracts 46 Non-compliances & 45 Observations There is not a reliable internal flow-down of requirements Ø Clean-room, DPAS, Calibration, QMS, Record Control, etc. The review is flawed as suppliers agree to requirements they do not meet. Ø Suppliers often do not know their basic system and therefore are not able to identify those contractual requirements that they do not satisfy. CDRL Items are either late or revised without updating the customer. Ø Assure that a single person is responsible for tracking the status of all CDRLs

Lessons-Learned Software 38 Non-compliances & 24 Observations The software controls are not all encompassing Ø Control of software includes deliverable, design, test, and operational software. There is not a compliant Software Safety Program (SSP) Ø Evaluate the SSP against NASA-Std-2100 The individual who developed the software integration are developing the integration test. Ø The need for objective independence is compromised by having the individual who developed the software integration also develop the integration test.

Lessons-Learned Internal Audits 18 Non-compliances & 14 Observations Contractual compliance and procedural support are not always part of the internal audit process. Ø Without auditing the contractual requirements the management will believe the Quality Management System is appropriate when in fact it does not support contractual requirements. Internal Audits are behind schedule. Ø This is a discipline that is necessary to avoid requirement creep or slippage. As Dwight D Eisenhower said “The uninspected inevitably deteriorate. ” The internal auditors do not audit or are not qualified to audit special processes. Ø Assure that subject mater experts are used when auditing software, soldering, heat treat, plating, etc.

Summary Individual commitment to a group effort - that is what makes a Ø team work, Ø company work, Ø society work, Ø civilization work. Vince Lombardi