ECE 453 CS 447 SE 465 Software Testing

ECE 453 – CS 447 – SE 465 Software Testing & Quality Assurance Instructor Kostas Kontogiannis 1

Overview èIntegration Testing èDecomposition Based Integration èCall Graph Based Integration èPath Based Integration èDiscussion Ref: “Software Testing A Craftsman's Approach” 2 nd edition, Paul C. Jorgensen 2

Path Based Integration Testing • The basic motivation is to combine structural and behavioral type of testing for integration testing as we did for unit testing • The basic idea is to focus on interactions among system units rather than merely to test interfaces among separately developed and tested units • In this respect, interface-based testing is structural while interaction-based is behavioral • Overall we want to express integration testing in terms behavioral threads 3

Extended Concepts (1) • Source node: – a program statement fragment at which program execution begins or resumes. – for example the first “begin” statement in a program. – also, immediately after nodes that transfer control to other units. • Sink node: – a statement fragment at which program execution terminates. – the final “end” in a program as well as statements that transfer control to other units. 4

Extended Concepts (2) • Module execution path: – a sequence of statements that begins with a source node and ends with a sink node with no intervening sink nodes. • Message: – a programming language mechanism by which one unit transfers control to another unit. – can be interpreted as subroutine invocations, procedure calls and function references. – convention: the unit which receives the message always eventually returns control to the message source. – messages can pass data to other units. 5

MM-Paths • MM-Path: – an interleaved sequence of module execution paths and messages. – we can describe sequences of module execution paths that include transfers of control among separate units. – MM-paths always represent feasible execution paths, and these paths cross unit boundaries. 6

MM-Path Example 4 1 A 1 2 3 C 1 B Figure 1 3 2 4 5 3 4 4 5 6 The Figure 1 above illustrates an MM-Path (the heavy line) for three modules. • For example, in module A nodes 1 and 5 are source nodes while nodes 4 and 6 are sink nodes. 7 Ref: “Software Testing A Craftsman's Approach” 2 nd edition, Paul C. Jorgensen

Execution Paths Example • In addition, the following are module execution paths: MEP(A, 1) = <1, 2, 3, 6> MEP(A, 2) = <1, 2, 4> MEP(A, 3) = <5, 6> MEP(B, 1) = <1, 2> MEP(B, 2) = <3, 4> MEP(C, 1) = <1, 2, 4, 5> MEP(C, 2) = <1, 3, 4, 5> 8

MM-Path Graph • Given a set of units, – their MM-Path graph is the directed graph in which nodes are module execution paths and, – edges correspond to messages and returns from one unit to another • The definition is with respect to a set of units. – It directly supports composition of units and composition based integration testing. – It is possible to compose down to level of individual module execution paths • but it would be more detailed than necessary. 9

MM-Path Graph Example MEP(A, 2) MEP(B, 1) MEP(A, 1) MEP(C, 1) Figure 2 MEP(B, 2) MEP(A, 3) MEP(C, 2) The Figure 2 above illustrates the MM-Path graph for the example in Figure 1. The solid arrows indicate messages and the corresponding returns are indicated by dotted arrows. Ref: “Software Testing A Craftsman's Approach” 2 nd edition, Paul C. Jorgensen 10

MM-Path Analogy • MM-Paths implement a function that transcends unit boundaries, thus we have the following relationship: – Consider the “intersection” of an MM-Path with a unit. • The module execution paths in this intersection are an analog of a slice with respect to the (MM-Path) function. – Hence the module execution paths in such an intersection are the restriction of the function to the unit in which they occur. 11

MM-Path Endpoints • There are three observable behavioral criteria that put endpoints on MM-Paths: – event quiescence: occurs when a system is nearly idle, waiting for a port input event to trigger further processing. • This is a system level property with an analog at the integration level: message quiescence. – message quiescence: occurs when a unit that sends no messages is reached (i. e. module C in Figure 1). – data quiescence: occurs when a sequence of processing culminates in the creation of stored data that is not immediately used. 12

Atomic System Function • Data quiescence occurs when a sequence of processing culminates in the creation of stored data that is not immediately used. • These criteria are “natural” endpoints for MM-Paths. • A second guideline for MM-Paths serves to distinguish integration from system testing: – atomic system function (ASF): is an action that is observable at the system level in terms of port input and output events. • It begins with a port input event, • traverses one or more MM-Paths, • and terminates with a port output event. 13

Atomic System Function – When viewed from the system level, there is no compelling reason to decompose an ASF into lower levels of detail (hence the atomicity). – For example in the ATM case, • an example of an ASF is card entry, cash dispensing, or session closing. • While PIN entry would probably be too big since it might entail a molecular system function 14

Atomic System Function • ASFs are an upper limit for MM-Paths: – MM-Paths should not cross ASF boundaries. – ASFs represent the seam between integration and system testing: • they are the largest item to be tested during integration testing, • and the smallest item for system testing. 15

ASF Example A B C MM-path: Interleaved sequence of module exec path and messages Module exec path: entry-exit path in the same module Atomic System Function: port input, … {MM-paths}, … port output Test cases: exercise ASFs 16

Call Graph as a Model for Module Interaction 1. program. Sums 2. read(n); 3. i: = 1; 4. while i <= n 5. sum: =0; 6. Acc(sum, i); 7. write(sum, i); 8. i : = i + l ; 9. endwhile 10. end. 11. procedure Acc(x, y) refx, y 12. j: = 1: 13. whilejsy 14. Add(x, j); 15. Inc(j); 16. endwhile 17. return 18. procedure Inc(z) ref z 19. Add(z. 1): 20. return 21. procedure Add(a, b) ref a; value b 22. a: =a+b; 23. return Sums sum, i Acc x, j j Inc Add z, 1 Call Graph 17

Program Summary Graph 1. program. Sums 2. read(n); 3. i: = 1; 4. while i <= n 5. sum: =0; 6. Acc(sum, i); 7. write(sum, i); 8. i : = i + l ; 9. endwhile 10. end. 11. procedure Acc(x, y) refx, y 12. j: = 1: 13. whilejsy 14. Add(x, j); 15. Inc(j); 16. endwhile 17. return 18. procedure Inc(z) ref z 19. Add(z. 1): 21. procedure Add(a, b) ref a; value b 22. a: =a+b; 23. return 20. return Sums Acc Inc Add x sum 5 z 1 a 13 7 i 17 y 3 z j 15 9 x 11 x 12 j 4 2 sum y 1 i x 1 18 16 10 z 14 z a 18

Interprocedural Program Summary Graph 1. program. Sums 2. read(n); 3. i: = 1; 4. while i <= n 5. sum: =0; 6. Acc(sum, i); 7. write(sum, i); 8. i : = i + l ; 9. endwhile 10. end. 11. procedure Acc(x, y) refx, y 12. j: = 1: 13. whilejsy 14. Add(x, j); 15. Inc(j); 16. endwhile 17. return 18. procedure Inc(z) ref z 19. Add(z. 1): 21. procedure Add(a, b) ref a; value b 22. a: =a+b; 23. return 20. return Sums Acc Inc Add {U 6} sum 5 x z 1 3 a 13 7 i 17 y {U 2, U 3, U 5}} {U 6} 9 j {U 6} z 15 x {U 6} 11 U 1: sum in line 7 U 2: i in line 7 U 3: i in line 8 U 4: j in line 13 U 5: y in line 13 U 6: a in line 22 x 12 j 4 2 sum y 1 {U 2. U 3} i {U 1} x 1 {U 1, U 4, U 6} 16 10 z 14 z {U 4, U 6} a 18 19

Discussion on Behavioral Integration Testing • MM-Paths and ASFs are a hybrid of functional and structural testing. • They are functional in sense that each represents an action with inputs and outputs, – all functional testing techniques are potentially applicable. – The structural side is how they are identified : • MM-Path graph. 20

Discussion on Behavioral Integration Testing • We avoid the pitfalls of structural testing, – and at same time, integration testing gains a fairly seamless junction with system testing. • Advantages come at a price: – more effort to identify MM-Paths and ASFs. – Effort may be offset by elimination of stub/driver development. – Also may be an overkill for applications which are not event driven. 21