History of Software Testing Philippe CHARMAN charmanfr ibm

History of Software Testing Philippe CHARMAN charman@fr. ibm. com http: //users. polytech. unice. fr/~charman/ Last update: 21 -04 -2014

Summary • Testing periods and evolution of testing models • Software testing timeline

Testing Periods • Until 1956: Debugging-oriented period • 1957– 1978: Demonstration-oriented period • 1979– 1982: Destruction-oriented period • 1983– 1987: Evaluation-oriented period • 1988– …: Prevention-oriented period

Debugging-oriented Period ( … - 1956) • No consensus between testing and debugging: – For some people, testing is a sub-activity of debugging – For others, debugging and testing are the same activity – For still others, debugging and testing are two distinguished activities • The main focus is on the hardware not on the software

Demonstration-oriented Period (1957 - 1978) • People realize bugs can have a negative impact • Charles Baker makes a clear distinction from debugging and testing: – Debugging: Make sure the program runs – Testing: Make sure the program solves the problem • Testers should write tests having in mind they have to be successful

Destruction-oriented Period (1979 – 1982) • In 1979, G. J. Myers writes his famous book:

Destruction-oriented Period (1979 – 1982) • Myers defined testing as “the process of executing a program with the intent of finding errors” • Tester should write tests having in mind they have to fail • Myers: Showing a program has bugs is more efficient than showing a program has no bugs. • Google Testing Blog: “If it ain't broke, you're not trying hard enough”

Evaluation-oriented Period (1983 – 1987) • Guideline for Lifecycle Validation, Verification and Testing of Computer Software • Methodology that integrates analysis, review and test activities to provide product evaluation during the software life cycle • A tester can start working on the software life cycle from the very beginning • A tester can review and gives his feedback on the specifications of the features to be implemented • For instance a tester can detect that: – User needs won’t be fully satisfied with the feature – A feature has some holes and thus cannot be correctly tested in some particular cases – etc.

Prevention-oriented Period (1988 – …) • Quality metrics: • Code coverage • Complexity • etc. • Rule coding • Comments in the code • etc. • Continuous Inspection

Major Testing Models Primary Goal Phase Models Demonstration To make sure the program solves the problem Destruction To detect implementation faults Evaluation To detect requirements, design and implementation faults Life Cycle Prevention Models To prevent requirements, design and implementation faults

Software Testing Timeline

Some Companies and Organizations • • 1896 1958 1973 1981 1989 1991 2002 IBM founded First software test team Compuware founded Rational Software founded Mercury Interactive founded Pure Software founded Sticky. Minds. com launched ITSQB founded

Some Tools and Applications • • 1988 1989 1991 1995 1998 2002 2005 First defect tracking tool (DDTS) First version of Load. Runner Start of the newsgroup comp. software. testing First release of Winrunner Bugzilla released IBM Rational Functional Tester JIRA 1. 0 released Hudson/Jenkins released

Some Models and Standards • • 1970 1983 1986 1993 1996 2001 2002 Waterfall model IEEE 829 V-model SCRUM Extreme Programming Agile Manifesto Test Driven Development

References • • Baker C Review of D. D. Mc. Cracken’s “Digital Computer Programming”” Mathematical Tables and Other Aids to Computation 1 I, 60. (Oct. 1957). 298 -305 Gelperin, D. ; B. Hetzel (1988). "The Growth of Software Testing". CACM 31 (6) Guideline for Lifecycle Validation, Verification and Testing of Computer Software. Processing Standards, National Bureau of Standards, June, 1983. Myers, Glenford J. (1979). The Art of Software Testing. John Wiley and Sons.

Some interesting links • • http: //www. stickyminds. com/ http: //www. testingreferences. com/ http: //googletesting. blogspot. com/ http: //idtus. com/img/Useful. Automated. Testing. Metrics. pdf