SEASAT Lessons Learned And Not Learned Rick Obenschain
SEASAT Lessons Learned. . . And Not Learned Rick Obenschain Acting Director of Flight Programs and Projects NASA Goddard Space Flight Center Rick Obenschain Arthur. F. Obenschain@nasa. gov
NASA Goddard Space Flight Center • • COLUMBIA Physical cause of the loss of Shuttle Columbia: “a breach in the Thermal Protection System on the leading edge of the left wing caused by a piece of insulating foam which separated from the left bipod ramp section of the external tank at 81. 7 seconds after launch, and struck the wing in the vicinity of the lower half of Reinforced Carbon panel number 8. ” Organizational Contributions to Loss: – Original design/implementation shortfalls required to stay within budgetary limitation that enabled the Shuttle program – Continuing schedule/funding constraints – find ways to do more with less • Workforce downsized • Outsourced various shuttle program responsibilities including safety oversight – Reliance on past success as a substitute for sound engineering practices – Organizational barriers that prevented effective communication of critical safety information – Unwillingness to listen to alternate view/concerns – Evolution of an informal chain of command decision making processes that operated outside of organization’s rules 9/12/2021 2
NASA Goddard Space Flight Center • • WIRE Physical Cause: FPGA Transient on startup clock oscillator start time Organization Contributions to loss: – Pyro Box Simple, not focused on, never reviewed, fell through cracks – NASA attempts to penetrate design review blocked – Failures in I&T: Didn’t go through schematics – blamed on test equipment by similarity • Didn’t contact designer or design organization • Didn’t write malfunction report which prevented failures from being reviewed • Result: immediate failure on orbit 9/12/2021 3
NASA Goddard Space Flight Center • • CHALLENGER Physical cause of the loss of Shuttle: “a failure of the joint and seal between the lower segments of the right Solid Rocket Booster. Hot gases blew past a rubber O-ring in the joint, leading to structural failure and explosive burning of the shuttle’s Hydrogen fuel. ” A number of significant NASA management failures highlighted – Communication failures and incomplete/misleading information • Key shuttle managers unaware of flight safety program • Contractors required to prove it was not safe to launch, rather than proving it was safe • Multiple missed warning signs – seal and joint degradations accepted as deviations • Safety Management displayed “a lack of problem reporting requirements, inadequate trend analysis, misrepresentation of criticality and lack of involvement in critical decisions” • NASA Human Space Flight Culture – Despite many outward management changes, the culture remained largely intact – By the winter of 2003 institutional practices that were in effect at the time of Challenger – inadequate concern over deviations from expected performance, a silent safety program and schedule pressure - had returned 9/12/2021 4
NASA Goddard Space Flight Center • • SEASAT Spacecraft launched June 26, 1978, Spacecraft failed October 9, 1978; Mission Lifetime 1503 revolutions/105 days Fully Redundant Spacecraft Bus – a single redundant system failed and caused mission loss 9/12/2021 5
NASA Goddard Space Flight Center • SEASAT Spacecraft failure caused by loss of electrical power resulting from a massive, progressive short in one of the ship ring assemblies used to connect the rotating solar arrays into the power system 9/12/2021 6
NASA Goddard Space Flight Center • SEASAT Arc between two adjacent ship ring brush assemblies – most adjacent brush assemblies were of opposite electrical polarity – Wire-to-brush assembly contact – Brush-to-brush contact – Momentary short caused by a contaminant that bridged internal components of opposite electrical polarity 9/12/2021 7
NASA Goddard Space Flight Center • SEASAT Slip ring failure possibilities well known within prime contractor facility; failures occurred on other programs. No communication within company of failures 9/12/2021 8
NASA Goddard Space Flight Center • SEASAT Contd. Feeling that existing spacecraft bus design “was standard”, although three of the major subsystems were substantially modified – Even when it became evident that significant changes were being made, belief in qualification by similarity persisted – Program policy to minimize testing and documentation – Program direction to minimize penetration into “standard bus” by government – Important component failures were not reported to project management, tests were waived without proper approval and compliance with specification was weak – Failure modes and effects analyses incomplete: did not even consider shorting failure mode; did not provide a basis for development of a full complement of safing command sequences that could be used by the flight controllers in responding to in-flight anomalies – Proper FMECA would have demonstrated risk areas and permitted simple design changes to be implemented – Controllers not sufficiently knowledgeable of systems being controlled; post failure analysis demonstrated that it would have been possible to separate bus into two sections with associated reduction in capabilities 9/12/2021 9
NASA Goddard Space Flight Center What Caused the Failure • Environment – Seasat conceived/initiated during post Apollo era • Apollo characterized by extensive test programs, large formal documentation systems and comprehensive/frequent technical and management reviews • NASA Low Cost Systems Office established to promote use of standardized hardware • Emphasis on shifting work out-of-house to reduce NASA workforce base • Design-to-cost techniques, cost benefits of heritage through use of hardware and software developed for other programs emphasized in the approval cycle – Management Philosophy • Design to cost fundamental tenet of Seasat Project definition; overruns to be offset by descoping mission content • To satisfy small funding contingency for the spacecraft bus, only government role was “monitoring” contractor’s activity; maximum reliance placed on existing contractor management systems and procedures 9/12/2021 10
NASA Goddard Space Flight Center What Caused the Failure Contd. – Treat Launch Readiness date as a “Planetary Launch Opportunity” • Program initiation delayed 8 months; launch date shipped 6 weeks – As cost escalations were experienced on both the instrumentation and spacecraft platforms, HQ pressure to cut back/eliminate penetration of spacecraft bus • Increasing reliance on tenet that spacecraft bus had extensive, flight proven history despite fact many changes were creeping in • In power system alone, solar arrays were first application of a rotating array on the aft end of the spacecraft bus, the slip ring assembly had no applicable flight experience and the solar array drive electronics had undergone extensive redesign 9/12/2021 11
NASA Goddard Space Flight Center What Caused the Failure Contd. • Hardware – Slip ring assembly design, development and parts qualification completed for earlier program cancelled prior to flight – Although not a direct match for requirements, decision made to use existing hardware • Unnecessarily crowded mechanical design • Subcontractor request to lengthen assembly denied due to “programmatic reasons” that did not apply to SEASAT • Decision made to alternate positive/negative communication to brushes to “reduce magnetic moments” requirement that did not exist • Significant slip ring problems noted a contractor facility on other program – never conveyed to SEASAT • Slip ring assembly on another program at contractor’s plant modified wiring to eliminate alternating plus/minus power configuration; SEASAT decided not necessary as slip rings not powered during launch vibration environment (Prelaunch operational change did apply power to slip rings during launch) 9/12/2021 12
NASA Goddard Space Flight Center 9/12/2021 Slip Ring Assembly 13
NASA Goddard Space Flight Center What Caused the Failure Contd. • Quality Assurance and Flight Readiness – Compliance with requirements weak; requirement that all electronic assemblies undergo at least eight thermal/thermal vacuum cycles not contained in slip ring component assembly specification – No “closed loop” compliance system to validate contractual requirements met – Qualification by similarity very loosely interpreted – Failure modes effects criticality analysis showed no power system single point failure 9/12/2021 14
NASA Goddard Space Flight Center What Caused the Failure Contd. • Mission Operations – Nature of low earth orbit operation requires different philosophy than deep space mission • Spacecraft not in continuous communication with ground station • “Snapshot” pictures of spacecraft/instrument operation • Extreme emphasis on ability to quickly analyze operations situation and “Safe” the Spacecraft before an anomaly cascades into a total failure • All credible single point failure modes should be removed in redundant bus applications and recovery procedures put in place and practiced • Major deficiencies in flight controller training and in development of mission roles and procedures – Spacecraft training at very high level • Insufficient to ensure capability of real-time anomaly assessment • Total of two Spacecraft anomalies practiced • No preplanned emergency safing sequence; when failure observed, no actions undertaken 9/12/2021 15
NASA Goddard Space Flight Center Concluding Thoughts • To stay within tight fiscal constraints, fundamental decisions made early on that resulted in fatal design/implementation shortfalls • Continuing schedule and funding pressure – reduce insight, testing, documentation • Over reliance on “Standard Flight Proven Bus” resulted in belief that past history justifies elimination of sound engineering practices • Lack of penetration into hardware developments precluded knowledge gaining communication • Alternate views/opinions stifled – contractor/government team attempts to convey magnitude of concerns resulting from overly constrained resources ignored by management • Lack of vigor in chain of command review/approval of documentation, testing modifications, performance waivers and training set stage for failure 9/12/2021 16
NASA Pinout - RJDA 1 -to-Thruster Harness (+Y Thruster) Connector Goddard Space Flight Center Functions: Fuel & Ox solenoid command power, heater power Location: Aft Body/Doghouse Reference Designator: 50 P 9967 Type: 24 -61 plug, (61) 20 ga. Contacts Observations: Most thruster coil pins are separated from 28 V pins. Bent pins can only short or disable thruster except for noted violation. Fuel & Ox solenoid command power, ten twisted-shielded pairs, 20 AWG, 7 A fuse. Thruster, Keel, & OME heater power, 14 twisted-shielded pairs, 20 AWG, 3 A protection from 50 P 254 Not connected Low current, indicators, etc. Twisted-shielded pairs, 20 AWG L OMS CONT V 2 PWR, FUSE 3 A Return (ground) Foreign object debris (FOD) min. distance ~0. 4 in. Violation of design standard to separate command power pins 9/12/2021 9/3/04 17 16
NASA Goddard Space Flight Center • • Credible Failure Modes Not Considered by Program Connectors – Pin-to-pin short – RJD output command pin to 28 VDC resulting in inadvertent thruster firing. Wire Shorts – Low resistance shorts between RJD control wiring and any voltage sources capable of 12. 5 V or more and 1 A or more (circuit analysis and WSTF data determined this threshold for fuel & oxidizer valve actuation) – Valve coil command wire short to 28 volt conductor due to wire insulation flaws caused by aging – Valve coil command wire short to 28 volt conductor due to conductive liquid between wire and cracked insulation causing low resistance short – Valve coil command wire short to 28 volt conductor due to shield braid wire foreign object debris (~36 AWG strand) bridging between 28 V and command line through ring-cracks in insulation. Note: Braid foreign object debris would need to "float" over from nearby LRU that uses tag-ring back shell with shielded wire Note: These 14 of 38 failure modes identified were not considered by Program 9/12/2021 18
- Slides: 18