2 2 Software Myths Myth 1 The cost

2. 2 Software Myths • Myth 1. The cost of computers is lower than that of analog or electromechanical devices. – Hardware is cheap compared to other electromechanical devices – However cost of software, with reliability and maintenance, is enormous • e. g. Space-Shuttle software has 400, 000 words (relatively small) but costs NASA approximately $100, 000 a year to maintain. – Software Costs can become exorbitant over time.

Myth 2. Software is easy to change. – Yes changes are easy to make -- but hard to make without introducing errors. – Every change must be verified and rectified – Becomes more “brittle” with changes – We become hesitant to change software over time -- recognizing

Myth 3 Cont. – Little available data on software reliability vs. non-computer systems – British Royal Signals and Radar Establishment analyzed software for highly safety critical purposes. – 10% of modules or functions deviated from original design. – Deviations found even in tested software. – 1 in 200 new modules had errors with observable effects on performance. – Integer overflow errors. – Complete error elimination is a hard and lofty goal to achieve.

Myth 3 Cont. – These are not just “teething problems” but chronic ones over tens or hundreds of hours of use – e. g. (1) Therac-25 worked correctly thousands of times before first know overdose occurred. – (2) Space Shuttle -- NASA invested enormous effort and resources since 1980 » yet 16 severity-level 1 software errors have been discovered (errors that would result in loss of shuttle or crew » 8 errors remained in code used in flights, though not encountered

Myth 3/ Cont. • 12 errors with lower severity triggered during flight; 3 threatened mission, 9 had to be worked around • ALL DESPITE THE SOPHISTICATION OF NASA’s software development and verification program.

Myth 3/ Cont. • Redundancy is not a solution as in the case of hardware wearout. • “Zero-Defect” software is false claim. • Usually not enough time to perfect software; costs also severe. • Computers may be more reliable == but not necessarily safer.

Myth 4: Increasing software reliability will increase safety – Software errors may not be related to safety at all – Compliance with requirements specification may not remove errors – Safety-critical software errors can often be traced to Requirements – That is, software is doing exactly what it is supposed to do. – Software may be correct and 100% reliable -- yet responsible for serious accidents. – RELIABILITY DOES NOT EQUAL SAFETY

Myth 5 Testing software or “proving” (using formal verification techniques) software correct can remove all the errors. – Software limitations well known – Exhaustive testing is impossible – Only a relatively small part of the state space can be covered – Despite improved testing techniques, no breakthroughs – Mathematical proofs advanced - but even arguments for impossibility of complete proof of correctness – Mathematical verification of software may be possible in the future.

Myth 5: /cont. • Correct behavior of software must be specified in a formal mathematical language. May be as difficult and error-prone as the code. • Software errors often involve overload -- outside the realm of specification • Intricate software interactions complicate the issue. • In summary, most safety-related software errors can be traced to the requirements

Myth 6: Reusing Software increases safety – Reuse of proven software may increase reliability, but has little or no effect on safety – May even decrease safety because of the complacency it engenders – Specific hazards of new implementation may not have been considered – Examples include:

Therac-20 parts reused for Therac-25 with same error, but causing two deaths » Error did not have serious consequences in Therac-20. » Resulted in occasional blown fuse -- not massive overdose » Never detected or fixed in Therac-20

Air Traffic Control Software » Successful in US for many years but not in Great Britain » Was not developed for zero degrees longitude along the Greenwich Meridian » Manchester plopped on top of Warwick

Aviation Software written for Northern Hemisphere has problems in Southern Hemisphere • Software written for US F-16 s caused problems when reused in aircraft flown over the Dead Sea where altitude is below sea level. • Safety is not a property of software alone, but of the software design, and environment where software is used. • Application, environment, and system-specific.

Myth 7: Computers reduce risk over mechanical systems b Argument 1 -- can check parameters through finer control more often b Counter -- Finer control allows operation under smaller safety margins b No way to test adequately

Myth 7: /cont. Argument 2 -- automated system allow operators to work farther away from hazardous areas. More accidents due to operators’ entry into 7 - 7 environment. Humans enter unsafe hazardous environments Became unsafe to enforce robot shutdown protocols

Myth 7/cont. • Argument 3 -- eliminating operators eliminates human errors • Argument 4 -- Computers have the potential to provide better information to operators and thus improve decision making • Argument 5 -- Software does not fail.