Integration Testing CS 4311 I Burnstein Practical Software

Integration Testing CS 4311 I. Burnstein. Practical Software Testing, Springer-Verlag, 2003. 1

Hierarchy of Testing Ad Hoc Program Testing Unit Testing Integration Testing Black Box Top Down Equivalence Boundary Decision Table State Transition Bottom Up Big Bang Sandwich System Testing Acceptance Testing Function Properties Benchmark Performance Pilot Reliability Alpha Availability Beta Security Use Case Usability Domain Analysis Documentation Portability White Box Control Flow Data Flow Capacity 2

Outline n n n Big bang Top-down Bottom-up Sandwich Regression 3

In Class n n Suppose you have constructed and tested all of the units. What are the possible things you can do next? 4

Review: Test Procedures Driver Module to be tested Stub Results Test cases Stub 5

Integration Testing n n Assumes the units have been tested individually Tests units working together Identifies errors in the interfaces between units Approaches ¨ Big bang ¨ Top-down ¨ Bottom-up ¨ Sandwich 6

Big-bang Integration n Test all components in isolation, then mix them all together and see how it works. As all components are merged at once, it is difficult to identify the cause of a failure. Q: Need test drivers and stubs for big-bang testing? 7

Top-Down Integration Testing n n Modules are integrated by moving downward through control hierarchy. Modules subordinate to main control module are incorporated Depth-first ¨ Breadth-first M 1 M 2 M 5 M 6 M 3 M 4 M 7 ¨ Pairs: What get’s tested first in breadth first? M 8 8

Top-Down Integration Testing n In Top Down, Breadth First M 1 tested with stubs for M 2, M 3 and M 4 ¨ Then M 1 -M 2 tested with stubs for M 3, M 4, M 5, and M 6 ¨ Then M 1 -M 2 -M 3 tested with stubs for M 4, M 5, M 6, and M 7 ¨ … M 1 ¨ M 2 M 5 M 6 M 3 M 4 M 7 M 8 9

Steps in Top-Down Integration Testing n n The main module is used as a test driver Stubs are substituted for all components directly subordinate to the main control module Stubs are replaced one at a time with actual components Tests are conducted as each component is integrated. 10

Evaluation of Top-Down Strategy n n Verifies major control or decision early in the test process Allows early recognition of major problems Depth first strategy allows a complete function of the software to be implemented Stubs replace lower-level modules so no significant data can flow upward ¨ Delay some tests until have actual modules ¨ Add code to simulate module 11

Bottom-Up Integration n n Low-level components are combined into clusters A driver is written to coordinate test case input and output Cluster is tested Drivers are removed and clusters are combined moving upward in program structure. 12

Bottom Up Integration Testing In Bottom Up: M 1 M 2 M 5 M 6 M 3 M 4 M 7 M 8 Pairs: What get’s tested first? 13

Bottom Up Integration Testing In Bottom Up: tested with drivers ¨ M 5 -M 8 is tested with drivers for M 5 ¨ M 5 -M 6 -M 8 is tested with drivers for M 5 M 6 ¨… M 1 ¨ M 8 M 2 M 5 M 6 M 3 M 4 M 7 M 8 14

Evaluation of Bottom-Up Strategy n n n Need for stubs is eliminated Operational modules tested thoroughly Begins construction and testing with atomic modules 15

Sandwich Integration n Combination of bottom-up and top-down integrations System is viewed as layers Approach 1: ¨ Top-down approach is used for the top layer ¨ A bottom-up approach is used for the bottom layer ¨ Allows integration to begin early in the testing phase ¨ Does not test individual components thoroughly before integration 16

Sandwich integration (Cont. ) n Approach 2: ¨ Start with a layer in the middle ¨ Use drivers to and stubs to check ¨ Work out from middle ¨ Allows integration to begin early in the testing phase ¨ Does not test individual components thoroughly before integration 17

Regression Testing n Adding new or changing module impacts the system ¨ New data flow paths established ¨ New I/O may occur ¨ New control logic invoked n Regression testing is re-execution of subset of tests that have already been conducted ¨ Ensures changes have not propagated unintended side effects 18

Regression Test (Cont. ) n Approaches ¨ Manual testing ¨ Capture/Playback tools: capture test cases and results for subsequent playback and comparison n Test suite contains following classes of test cases: ¨ Representative sample of tests that exercises all software functions ¨ Focus on functions likely affected by change ¨ Focus on components that have been changed 19