1 Chapter 9 1 Software Testing Strategies Galin

1 Chapter 9. 1 Software Testing Strategies Galin, SQA from theory to implementation © Pearson Education Limited 2004

2 Chapter 9. 1 – Software Testing Strategies • • • Definitions and objectives Software testing strategies Software test classifications White box testing • • • Data processing and calculation correctness tests Correctness tests and path coverage Correctness tests and line coverage Mc. Cabe’s cyclomatic complexity metrics Software qualification and reusability testing Advantages and disadvantages of white box testing • • • Equivalence classes for output correctness tests Other operation factor testing classes Revision factor testing classes Transition factor testing classes Advantages and disadvantages of black box testing Black box testing Galin, SQA from theory to implementation © Pearson Education Limited 2004

3 Introduction • In the past, testing (other than some unit testing) was undertaken as the last part of development after the entire app was developed and documented. • Later, when discovered problems were discovered too late, unit testing and integration testing became in vogue. • Later, the notion of testing has been expanded significantly to include documentation, and hosts of other related software quality assurance activities. • We will concentrate on testing as accommodated by running the code. Galin, SQA from theory to implementation © Pearson Education Limited 2004

4 Introduction • As of 1994, some 24% of project development budget allocated to testing; 32% of project management budget slated for testing. • Major emphases on increased testing has met resistance and project managers had to reduce. – We cannot test everything!!! • We often ‘test’ documentation via standard SQA. But this does not test the functionality of the code. • Will concentrate on testing classifications and strategies and classification of requirements types Galin, SQA from theory to implementation © Pearson Education Limited 2004

5 Introduction • Need to be able to: – Explain testing objectives – Discuss the differences between the various testing strategies: advantages and disadvantages – Describe the concepts of both black box and white box testing and their advantages and disadvantages – Discuss path coverage versus line coverage – Describe the various types of black box tests. – Testing has become a huge business and major emphasis in software engineering! • Testing has been described as the process of executing a program with the intention of finding errors. Galin, SQA from theory to implementation © Pearson Education Limited 2004

6 Software Tests - Definition Software testing is a formal process carried out by a specialized testing team in which a software unit, several integrated software units or an entire software package are examined by running the programs on a computer. All the associated tests are performed according to approved test procedures on approved test cases. Galin, SQA from theory to implementation © Pearson Education Limited 2004

7 • Formal: scheduled in advance; central in the development process; not ad hoc. • Specialized Testing Team: independent test team or external consultancy. Unit tests by developers – okay, but that is it. • Running the program – a must; nothing static. • Approved Test Procedures: use a plan approved. • Approved Test Cases: All planned; all tests undertaken. Galin, SQA from theory to implementation © Pearson Education Limited 2004

8 Software Testing Objectives Direct objectives a. To identify and reveal as many errors as possible in the tested software b. To bring the tested software, after correction of the identified errors and retesting, to an acceptable level of quality. c. To perform the required tests efficiently and effectively, within the limits budgetary and scheduling limitation. Indirect objectives a. To compile a record of software errors for use in error prevention (by corrective and preventive actions) Galin, SQA from theory to implementation © Pearson Education Limited 2004

9 Software Testing Strategies • Incremental testing strategies: - Test incrementally: Unit testing; Integration testing; System testing • Bottom-up testing • Top-down testing • Big bang testing – Test entire software at one time. Galin, SQA from theory to implementation © Pearson Education Limited 2004

10 Bottom-up testing Explain sequence M 11 Stage 4 Integration B Integration c M 9 Stage 3 M 10 Integration A M 8 Stage 2 Stage 1 M 2 Galin, SQA from theory to implementation M 3 M 4 M 5 M 6 M 7 © Pearson Education Limited 2004

Top-down testing Explain: 11 Integration D Integration C Integration B Stage 1 Integration A M 11 M 9 Stage 2 M 10 M 8 Stage 3 M 6 Stage 4 Stage 5 M 1 M 7 M 2 Stage 6 Galin, SQA from theory to implementation M 3 M 4 M 5 © Pearson Education Limited 2004

12 • There are many variations to these approaches too. • One can do a depth-first approach (vertically sequenced, according to your book) or a horizontallysequenced approach. • This depends on many factors • Can also do a sandwich approach too: partially bottom-up and then top-down. • When to do this? Perhaps when existing modules are reusable and already exist. Can do both! Galin, SQA from theory to implementation © Pearson Education Limited 2004

13 Use of Stubs and Drivers for Incremental Testing Top-down testing of module M 8 M 9 M 8 Stub of M 1 Bottom-up testing of module M 8 Module tested in an earlier stage Drive of M 9 Module on test Stub of M 2 Galin, SQA from theory to implementation M 8 M 1 Module on test M 2 Modules tested in an earlier stage © Pearson Education Limited 2004

Tree Class – Top Down: Examples of Drivers and Stubs class Tree { private Node root; // only data field in Tree; but key! public void find (int key) { // stub: not showing details of this method here }// end find() public void insert (int id, double dd) { // stub; placeholder }// end insert() pubic void delete (int id) { // stub }// end delete() } // end class Tree. 14

Top Down: Drivers: In main somewhere: • Tree the. Tree = new Tree(); • the. Tree. find(); // main hunks of functionality • the. Tree. insert(); • the. Tree. display(); • // program runs in its entirety each time. 15

Tree Class – Example of Stubs – can do “Displays!” class Tree { private Node root; // only data field in Tree; but key! public void find (int key) { // stub: not showing details of this method here System. out. println ( “Got into find()” ); }// end find() public void insert (int id, double dd) { // stub; placeholder System. out. println ( “Got into insert()” ); }// end insert() pubic void delete (int id) { // stub System. out. println (“ Got into delete routine () “ ); }// end delete() } // end class Tree. 16

Fill in code incrementally…. Develop method… public Node find (int key) { Node current = root; // assumes non-empty tree // start at root while (current. i. Data != key) // if no match { if (key < current. i. Data) current = current. left. Child; // recall: current = current. next? ? else current = current. right. Child; If (current == null) return null; // not found; boundary condition } // end while return current; // returns reference to node } // end find() 17

Insert code into placeholder as developed… • class Tree • { • • • private Node root; public void find (int key) { • // only data field in Tree; but key! Node current = root; // start at root while (current. i. Data != key) // if no match { if (key < current. i. Data) current = current. left. Child; else current = current. right. Child; If (current == null) return null; // not found; boundary condition } // end while return current; // returns reference to node } // end find() public void insert (int id, double dd) { // stub; placeholder System. out. println ( “Got into insert()” ); }// end insert() pubic void delete (int id) { // stub System. out. println (“ Got into delete routine () “ ); }// end delete() } // end class Tree. 18

19 Tree Class – Bottom Up: • For bottom up, you must simulate data passed to a lower level module from above. • Use dummy data…and parameters and returns… Galin, SQA from theory to implementation © Pearson Education Limited 2004

20 Comparison: Bottom-Up versus Top Down • Bottom Up – – Main advantage is the relative ease of its performance. – Main disadvantage is the lateness at which the program as a whole can be observed. – Sometimes the pieces may not fit too. Structure may be off. – Sometimes awkward to pass dummy data and to accept returned data… Galin, SQA from theory to implementation © Pearson Education Limited 2004

21 Comparison: Bottom-Up versus Top Down • Top Down: - – Main advantage is the possibilities if offers to demonstrate the entire program functions shortly after activation of the upper level modules are completed. – Can show early analysis and design flaws – Main disadvantage – often requires complicated programming and relative difficulty of analyzing the results of tests. – Supports top down programming, top down testing, and more. – Easy to add functionality via stubs and drivers. Galin, SQA from theory to implementation © Pearson Education Limited 2004

22 Comparison: Bottom-Up versus Top Down • Lots of debates on preferable strategy. • Choice is usually dependent upon the developers choice of a strategy – top down or bottom up. • The testing strategy needs to follow the development strategy. Galin, SQA from theory to implementation © Pearson Education Limited 2004

Big Bang Approach vs Incremental 23 • Big Bang: In general, not a good approach, unless program is very small and not terribly complicated. – Difficult to identify errors and where they are located. – Simply way too much code / functionality to evaluate at one time. • Incremental testing provides a number of advantages – – – – Test on usually small hunks of code, like unit or integration tests. Easier to identify errors than with whole project Correction is much simpler and requires far fewer resources too. Find errors much earlier in process. Prevents migration of errors into later, more complex stages. But you do have overhead of developing drivers and stubs and drivers for integration testing. Also, you carry out many testing operations on the same program vice only a single testing operation. • Best: generally incremental approach is preferred despite its disadvantages. Galin, SQA from theory to implementation © Pearson Education Limited 2004

24 Software Test Classifications • While quite different (each has strong proponents), there are two accepted classification schemes: – 1. Black Box testing – 2. While Box testing. • Black box Testing: (Functional Testing) – identifies bugs only according to software malfunctioning as they are revealed in its erroneous outputs. In cases that the outputs are found to be correct, black box testing disregards the internal path of calculations and processing performed. • White Box Testing: (Structural) – Examines internal calculation paths in order to identify bugs. Although the term ‘white’ is meant to emphasize the contrast between this method and black box testing, the method’s other name – ‘glass box testing’ better expresses its basic characteristic, that of investigating the correctness of code structure. Galin, SQA from theory to implementation © Pearson Education Limited 2004

25 Black Box and White Box - IEEE Definitions Black box testing – IEEE definition 1. Testing that ignores the internal mechanism of the system or component and focuses solely on outputs in response to selected inputs and execution conditions 2. Testing conducted to evaluate the compliance of a system or component with specified functional requirements White box testing – IEEE definition Testing that takes into account the internal mechanism of a system or component Galin, SQA from theory to implementation © Pearson Education Limited 2004

26 Classification According to Requirements • White box testing (to me) is more complicated than black box testing. There are many kinds of white box testing: – Path testing, branch testing, equivalence class testing, static analysis, dynamic analysis, etc. Complexity testing, flow graphs, call graphs, etc. – These can be very complicated. – Good for verification. • Black box testing – oversimplified – checks to see if the correct outputs are produced for specific inputs. – There is no internal checking. – Simple outputs given inputs – Great for validation testing. Galin, SQA from theory to implementation © Pearson Education Limited 2004