Chapter 8 Path Testing Path testing Path testing

  • Slides: 44
Download presentation
Chapter 8 Path Testing

Chapter 8 Path Testing

Path testing • Path testing is a “design structural testing” in that it is

Path testing • Path testing is a “design structural testing” in that it is based on detailed design & the source code of the program to be tested. • The methodology uses the graphical representation of the source code – Thus it is very much “control flow” or “path” oriented • This methodology has been available since the mid-1970’s and is an important White Box testing technique

Path Testing • Paths derived from some graph construct. • When a test case

Path Testing • Paths derived from some graph construct. • When a test case executes, it traverses a path. • Huge number of paths implies some simplification is needed. • Big Problem: infeasible paths. • By itself, path testing can lead to a false sense of security.

Program Graphs Definition: Given a program written in an imperative programming language, its program

Program Graphs Definition: Given a program written in an imperative programming language, its program graph is a directed graph in which nodes are statement fragments, and edges represent flow of control. (A complete statement is a “default” statement fragment. )

Program Graphs of Structured Programming Constructs

Program Graphs of Structured Programming Constructs

Sample Program Graph

Sample Program Graph

Common Objection to Path-Based Testing first (Trillions of Paths) A B C D F

Common Objection to Path-Based Testing first (Trillions of Paths) A B C D F E If the loop executes up to 18 times, there are 4. 77 Trillion paths. Impossible, or at least, infeasible, to test them all. [Schach] 50 + 51 + 52 + … + 518 = 4, 768, 371, 582, 030 G last Stephen R. Schach, Software Engineering, , (2 nd edition) Richard D. Irwin, Inc. and Aksen Associates, Inc. 1993

Schach’s flowchart

Schach’s flowchart

Test Cases for Schach’s “Program” first 1. 2. 3. 4. 5. A B C

Test Cases for Schach’s “Program” first 1. 2. 3. 4. 5. A B C D F G last E First-A-B-C-F-G-Last First-A-B-C-D-F-G-Last First-A-B-D-F-G-Last First-A-B-E-F-G-Last First-A-B-E-D-F-G-Last These test cases cover • Every node • Every edge • Normal repeat of the loop • Exiting the loop

Transforming Code to Diagram • In early chapter (chapter 4) on graph theory, we

Transforming Code to Diagram • In early chapter (chapter 4) on graph theory, we mentioned condensation graph as a graph that was derived by “condensing” a component (or set of code) into a node as a way to simplify the graphical diagram of source code. • Here we will : – assume that we have a condensed graph and discuss path analysis

Path Analysis • Why path analysis for test case design? – Provides a systematic

Path Analysis • Why path analysis for test case design? – Provides a systematic methodology of White Box testing. • Reproducible • Traceable • Countable Reasons similar to why document test cases • What is path analysis? – Analyzes the number of paths that exist in the system – Facilitates the decision process of how many paths to include in the test Path, as previously defined in Graph Theory, is a sequence of connected nodes

Linearly Independent Path • A path through the system is Linearly Independent** from other

Linearly Independent Path • A path through the system is Linearly Independent** from other paths only if it includes some segment or edge that is not covered in the other paths. - The statements are represented by the rectangular and diamond blocks (nodes). - The segments between the blocks are the edges, labeled with numbered circles. S 1 1 2 C 1 4 S 2 3 S 3 Path 1 : S 1 – C 1 – S 3 Path 2 : S 1 – C 1 – S 2 – S 3 OR Path 1: edges (1, 4) Path 2: edges (1, 2, 3) Path 1 and Path 2 are linearly independent because each includes some edge that is not included in the others. (note: not necessarily nodes) ** This definition will require more explanation later.

Another Example of Linearly Independent Paths S 1 1 C 1 2 S 2

Another Example of Linearly Independent Paths S 1 1 C 1 2 S 2 8 9 Path 1: Path 2: Path 3: Path 4: 10 Note that these are all linearly independent 5 C 2 3 S 3 6 C 3 7 S 5 4 S 4 edges (1, 2, 8) edges (1, 5, 3, 9) edges (1, 5, 6, 4, 10) edges (1, 5, 6, 7)

Statement Coverage Method • Count all the linearly independent paths • Pick the minimum

Statement Coverage Method • Count all the linearly independent paths • Pick the minimum number of linearly independent paths that will include all the statements (nodes) (S’s and C’s in the diagram) S 1 Path 1 : S 1 – C 1 – S 3 : edges (1, 4) Path 2 : S 1 – C 1 – S 2 – S 3 : edges (1, 2. 3 ) 1 2 C 1 4 S 2 3 S 3 Are both Path 1 and Path 2 needed to cover all the statements: (S 1, C 1, S 2, S 3) ? What would you say?

Another Example of Statement Coverage S 1 1 C 1 2 S 2 8

Another Example of Statement Coverage S 1 1 C 1 2 S 2 8 The 4 Linearly Independent Paths Covers: 5 C 2 3 S 3 9 6 C 3 7 4 S 4 10 Path 1: includes S 1 -C 1 -S 2 -S 5 : edges (1, 2, 8) Path 2: includes S 1 -C 2 -S 3 -S 5 : edges ( 1, 5, 3, 9) Path 3: includes S 1 -C 2 -C 3 -S 4 -S 5 : edges (1, 5, 6, 4, 10) Path 4: includes S 1 -C 2 -C 3 -S 5 : edges (1, 5, 6, 7) S 5 For 100% Statement Coverage, all we need are 3 paths : Path 1, Path 2, and Path 3 to cover all the statements (S 1, C 1, S 2, C 2, S 3, C 3, S 4, S 5) - - - no need for Path 4 - - !!

Statement Coverage Now - - What do you think of a Software Company which

Statement Coverage Now - - What do you think of a Software Company which boasts that they run every statement in the software before release? Try this on some students or other professors and see their reaction.

Branch Coverage Method (Also DD-Paths of our text book) 1. Identify all the decisions

Branch Coverage Method (Also DD-Paths of our text book) 1. Identify all the decisions 2. Count all the branches from each of the decisions ( “out-degree” of the decision node) 3. Pick the minimum number of paths that will cover all the branches from the decisions.

Branch Coverage Method One decision C 1 : B 1 : Path 1 :

Branch Coverage Method One decision C 1 : B 1 : Path 1 : C 1 – S 3 B 2 : Path 2 : C 1 – S 2 S 1 1 Branch 1 C 1 Branch 2 2 linearly independent paths cover : 2 4 S 2 3 B 1 : Path 1 : S 1 - C 1 – S 3 B 2 : Path 2 : S 1 - C 1 – S 2 – S 3 Are both Path 1 and Path 2 needed to cover Branch 1 and Branch 2 from C 1?

Another Example of Branch Coverage The 3 Decisions and the Branches: S 1 C

Another Example of Branch Coverage The 3 Decisions and the Branches: S 1 C 1: 1 C 1 2 S 2 - B 1 : C 1 - S 2 - B 2 : C 1 - C 2 8 C 2: - B 3 : C 2 – S 3 - B 4 : C 2 – C 3 5 C 2 3 S 3 9 - B 5 : C 3 – S 4 - B 6 ; C 3 – S 5 6 C 3 7 S 5 4 S 4 C 3: 10 The 4 Linearly Independent Paths Cover: Path 1: Path 2: Path 3: Path 4: includes S 1 -C 1 -S 2 -S 5 includes S 1 -C 2 -S 3 -S 5 includes S 1 -C 2 -C 3 -S 4 -S 5 includes S 1 -C 2 -C 3 -S 5 We need: Path 1 to cover B 1, Path 2 to cover B 2 and B 3, Path 3 to cover B 4 and B 5, Path 4 to cover B 6

Branch Coverage How do you feel about a Software Company who states that they

Branch Coverage How do you feel about a Software Company who states that they test every branch in the Software before release? Does All Branch Coverage subsume All Statements Coverage?

DD-Paths A DD-Path (decision-to-decision) is a chain in a program graph such that Case

DD-Paths A DD-Path (decision-to-decision) is a chain in a program graph such that Case 1: it consists of a single node with indeg = 0, Case 2: it consists of a single node with outdeg = 0, Case 3: it consists of a single node with indeg ≥ 2 or outdeg ≥ 2, Case 4: it consists of a single node with indeg = 1 and outdeg = 1, Case 5: it is a maximal chain of length ≥ 1.

DD-Path Graph Given a program written in an imperative language, its DD-Path graph is

DD-Path Graph Given a program written in an imperative language, its DD-Path graph is the directed graph in which nodes are DD-Paths of its program graph, and edges represent control flow between successor DD-Paths. • • • a form of condensation graph 2 -connected components are collapsed into an individual node single node DD-Paths (corresponding to Cases 1 - 4 ) preserve the convention that a statement fragment is in exactly one DD-Path

Condensation of Code to Table then to Graph Psuedo-code 1. Program Triangle 2. Dim

Condensation of Code to Table then to Graph Psuedo-code 1. Program Triangle 2. Dim a, b, c As Integer 3. Dim Is. Triangle As Boolean 4. Output ( “enter a, b, and c integers”) 5. Input (a, b, c) 6. Output (“side 1 is”, a) 7. Output (“side 2 is”, b) 8. Output (”side 3 is”, c) Def of 9. If (a<b+c) AND (b<a+c) And (c<b+a) DD-paths on 10. then Is. Triangle = True Page 140 11. else Is. Triangle = False 12. endif 13. If Is. Triangle 14. then if (a=b) AND (b=c) 15. then Output (“equilateral”) 16. else if (a NE b) AND (b NE c) 17. then Output ( “Scalene”) 18. else Output (“Isosceles”) 19. endif 20. endif 21. else Output (“not a triangle”) 22. endif 23. end Triangle 2 Path/node name DD-path Def. Skip 1 - 3 (or w/4) 4 5– 8 first A 1 5 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 B C D E F H I J K L M N G O last 3 4 4 3 3 4 3 2 code statement

Condensation Graph from Table - Statements coverage - Branch (DD-path) coverage - Cyclomatic #

Condensation Graph from Table - Statements coverage - Branch (DD-path) coverage - Cyclomatic # = 4+1 = 5 - All combinations first A B C D E F G H I J K L M N O Last - 4 paths - 5 lin. Ind paths - 8 paths

Closer Look into Path Testing • Look at 2 paths from A to E.

Closer Look into Path Testing • Look at 2 paths from A to E. The paths of either it is a triangle or not a triangle. – But there are many combinations to get “not triangle” ; so we still need to consider utilizing boundary values and equivalence class. Just one test case to cover the path may not be enough • Look at the path that leads from D to L (D= not triangle and L= Isosceles triangle). Is that path possible - - - can you generate a test case for that path? • Look at the path from C to G (C = Is a triangle and G = output “not triangle”) - - - is this a possible path? – There may be logical dependencies that prevent us from generating test case to traverse certain paths. Thus we also need to consider employing decision tables from Black Box testing technique.

Condensation of Code to Table then to Graph Psuedo-code 1. Program Triangle 2. Dim

Condensation of Code to Table then to Graph Psuedo-code 1. Program Triangle 2. Dim a, b, c As Integer 3. Dim Is. Triangle As Boolean 4. Output ( “enter a, b, and c integers”) 5. Input (a, b, c) 6. Output (“side 1 is”, a) 7. Output (“side 2 is”, b) 8. Output (”side 3 is”, c) Def of 9. If (a<b+c) AND (b<a+c) And (c<b+a) DD-paths on 10. then Is. Triangle = True Page 140 11. else Is. Triangle = False 12. endif 13. If Is. Triangle 14. then if (a=b) AND (b=c) 15. then Output (“equilateral”) 16. else if (a NE b) AND (b NE c) 17. then Output ( “Scalene”) 18. else Output (“Isosceles”) 19. endif 20. endif 21. else Output (“not a triangle”) 22. endif 23. end Triangle 2 Path/node name DD-path Def. Skip 1 - 3 (or w/4) 4 5– 8 first A 1 5 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 B C D E F H I J K L M N G O last 3 4 4 3 3 4 3 2 code statement

Mc. Cabe’s Cyclomatic Number • Is there a way to know how many linearly

Mc. Cabe’s Cyclomatic Number • Is there a way to know how many linearly independent paths exist? – Mc. Cabe’s Cyclomatic number which we used to study program complexity may be applied. There are 3 ways to get the Cyclomatic Complexity number from a flow diagram. • # of binary decisions + 1 • # of edges - # of nodes + 2 • # of closed regions + 1

Mc. Cabe’s Cyclomatic Complexity Number Earlier Example We know there are 2 linearly independent

Mc. Cabe’s Cyclomatic Complexity Number Earlier Example We know there are 2 linearly independent paths from before: Path 1 : C 1 – S 3 Path 2 : C 1 – S 2 – S 3 S 1 1 C 1 4 Mc. Cabe’s Cyclomatic Number: 2 Closed region S 2 3 S 3 a) # of binary decisions +1 = 1 +1 = 2 b) # of edges - # of nodes +2 = 4 -4+2 = 2 c) # of closed regions + 1 = 1 + 1 = 2

Mc. Cabe’s Cyclomatic Complexity Number Another Example Mc. Cabe’s Cyclomatic Number: a) # of

Mc. Cabe’s Cyclomatic Complexity Number Another Example Mc. Cabe’s Cyclomatic Number: a) # of binary decisions +1 = 2 +1 = 3 S 1 b) # of edges - # of nodes +2 = 7 -6+2 = 3 1 4 C 1 2 5 C 2 7 c) # of closed regions + 1 = 2 + 1 = 3 S 2 Closed Region S 4 6 3 S 3 There are 3 Linearly Independent Paths

An example of 2 n total path Since for each binary decision, there are

An example of 2 n total path Since for each binary decision, there are 2 paths and there are 3 in sequence, there are 23 = 8 total “logical” paths S 1 1 2 C 1 3 S 2 path 5 : S 1 -C 2 -C 3 -S 4 path 6 : S 1 -C 2 -C 3 -S 5 path 7 : S 1 -C 2 -S 3 -C 3 -S 4 path 8 : S 1 -C 2 -S 3 -C 3 -S 5 4 C 2 5 S 3 6 How many Linearly Independent paths are there? Using Cyclomatic number = 3 decisions +1 = 4 7 C 3 One set would be: 9 8 S 5 path 1 : S 1 -C 1 -S 2 -C 3 -S 4 path 2 : S 1 -C 1 -S 2 -C 3 -S 5 path 3 : S 1 -C 1 -S 2 -C 2 -S 3 -C 3 -S 4 path 4 : S 1 -C 1 -S 2 -C 2 -S 3 -C 3 -S 5 S 4 path 1 : includes edges (1, 2, 4, 6, 9) path 2 : includes edges (1, 2, 4, 6, 8) path 3 : includes edges (1, 2, 4, 5, 7, 9) path 5 : includes edges (1, 3, 6, 9) Note 1: with just 2 paths ( Path 1 and Path 8) all the statements are covered. Note 2: with just 2 paths ( Path 1 and Path 8) all the branches are covered.

Example with a Loop Total number of ‘logical’ paths may be “infinite” (very large)

Example with a Loop Total number of ‘logical’ paths may be “infinite” (very large) because of the loop ---- so we will only look at this “statically” S 1 1 C 1 Linearly Independent Paths = 1 decision +1 = 2 4 S 3 path 1 : S 1 -C 1 -S 3 (segments 1, 4 ) path 2 : S 1 -C 1 -S 2 -C 1 -S 3 (segments 1, 2, 3, 4 ) 2 S 2 “One path” will cover all statements: path 2 : S 1 -C 1 -S 2 -C 1 -S 3 (S 1, C 1, S 2, S 3) 3 “One path” will cover all branches: path 2 : S 1 -C 1 -S 2 -C 1 -S 3 branch 1 (C 1 -S 2) and branch 2 (C 1 -S 3) *** Note***: To get one path --- S 2 must somehow change C 1

Example with a Loop (cont. ) Total number of ‘logical’ paths may be “infinite”

Example with a Loop (cont. ) Total number of ‘logical’ paths may be “infinite” (very large) because of the loop S 1 1 C 1 Linearly Independent Paths = 1 decision +1 = 2 4 S 3 path 1 : S 1 -C 1 -S 3 (segments 1, 4 ) path 2 : S 1 -C 1 -S 2 -C 1 -S 3 (segments 1, 2, 3, 4 ) 2 S 2 3 “One path” will cover all branches: path 2 : S 1 -C 1 -S 2 -C 1 -S 3 branch 1 (C 1 -S 2) and branch 2 (C 1 -S 3) Should we consider the loop boundaries in C 1? & also bring in our boundary value testing thoughts?

Mc. Cabe Basis Paths in the Triangle Program There are 8 topologically possible paths.

Mc. Cabe Basis Paths in the Triangle Program There are 8 topologically possible paths. 4 are feasible, and 4 are infeasible. Exercise: Is every basis path feasible? V(G) = 23 - 20 + 2(1) = 5 Basis Path Set B 1 p 1: 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 18, 19, 20, 22, 23 p 2: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 16, 18, 19, 20, 22, 23 p 3: 4, 5, 6, 7, 8, 9, 11, 12, 13, 21, 22, 23 p 4: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 20, 22, 23 p 5: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 16, 17, 19, 20, 22, 23 (mainline) (flipped at 9) (flipped at 13) (flipped at 14) (flipped at 16)

Condensation with Structured Programming Constructs

Condensation with Structured Programming Constructs

Violations of Structured Programming Precepts

Violations of Structured Programming Precepts

More on Linearly Independent Paths • In discussing dimensionality, we talks about orthogonal vectors

More on Linearly Independent Paths • In discussing dimensionality, we talks about orthogonal vectors (or “basis” of vector space). – (e. g. ) Two dimensional space has two orthogonal vector from which all the other vectors in two dimension can be obtained via “linear combination” of these 2 vectors: • [1, 0] • [0, 1] [2, 4] [1, 0] [0, 1] e. g. [2, 4] = 2[1, 0] + 4[0, 1]

More on Linearly Independent Paths • A set of paths is considered to be

More on Linearly Independent Paths • A set of paths is considered to be a Linearly Independent Set if every path may be constructed as a “linear combination” of paths from the linearly independent set. For example: We already know: a) there a total of 22=4 logical paths. b) 1 path (path 4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths. C 1 2 1 S 1 1 3 C 2 5 4 S 2 6 path 1 1 path 2 1 2 3 4 5 6 1 1 1 path 3 1 1 path 4 1 1 1 We pick: path 1, path 2 and path 3 as the Linearly Independent Set path 4 = path 3 + path 1 – path 2 = (0, 1, 1, 1, 0, 0)+(1, 0, 0, 0, 1, 1)- (1, 0, 0, 1, 0, 0) = (1, 1, 1, 1) - (1, 0, 0, 1, 0, 0) = (0, 1, 1, 0, 1, 1)

Other Sets of Linearly Independent Paths 1 path 1 1 path 2 1 2

Other Sets of Linearly Independent Paths 1 path 1 1 path 2 1 2 3 4 5 6 1 1 1 path 3 1 1 path 4 1 1 1 -Consider the set of linearly independent paths: 1, 2 & 4 instead. - Can we get Path 3 = (0, 1, 1, 1, 0, 0)? -Consider path 3 = path 2 + path 4 – path 1 = (1, 0, 0, )+(0, 1, 1, 0, 1, 1) – (1, 0, 0, 0, 1, 1) = (0, 1, 1, 1, 0, 0) -Consider another set of linearly independent paths: 2, 3 & 4 instead. - Can we get Path 1 = (1, 0, 0, 0, 1, 1))? -Consider path 1 = path 2 + path 4 – path 3 = (1, 0, 0, )+(0, 1, 1, 0, 1, 1) – (0, 1, 1, 1, 0, 0) = (1, 0, 0, 0, 1, 1)

More on Linearly Independent Paths We already know: a) there a total of 22=4

More on Linearly Independent Paths We already know: a) there a total of 22=4 logical paths. b) 1 path (path 4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths. C 1 2 1 1 S 1 3 C 2 5 4 path 1 1 path 2 1 2 3 4 5 6 1 1 1 path 3 1 1 path 4 1 1 1 S 2 6 Note : Although path 1 and path 3 are linearly independent, they do NOT form a Linearly Independent Set because no linear combination of path 1 and path 3 can get , say, path 4.

More on Linearly Independent Paths • Because the Linearly Independent Set of paths display

More on Linearly Independent Paths • Because the Linearly Independent Set of paths display the same characteristics as the mathematical concept of basis in n-dimensional vector space, the testing using the Linearly Independent Set of paths is sometimes called the “basis” testing. • The main notion is that since the “linear independent set of paths” as a set can span all the paths for the design/code construct, then basis testing covers the “essence” of the whole structure. Then ------ is there a way to find a Linearly Independent Set ?

An Algorithm to Find the Basis Set 1. Select a baseline path, an arbitrary,

An Algorithm to Find the Basis Set 1. Select a baseline path, an arbitrary, normal execution path that contains as many decisions as possible. Retrace the baseline path and “flip” each of the decision encountered; each flip creates a new path. Continue until all the decisions are flipped The basis set is composed of all the paths generated from steps 1 and 2 above 2. 3. Cyclomatic # = 2+ 1 = 3; So there are 3 linearly independent paths C 1 2 1 S 1 3 C 2 5 4 S 2 6 1. pick baseline path P 1: C 1 –S 1 - C 2 – S 2: <0, 1, 1, 0, 1, 1> 2. flip C 1 P 2 : C 1 – C 2 – S 2 : <1, 0, 0, 0. 1, 1> 3. flip C 2 P 3: C 1 - C 2 : <1, 0, 0, 1, 0, 0> Can we get the 4 th path : C 1 – S 1 – C 2 : <0, 1, 1, 1, 0, 0> from the above basis set? How about : (P 1 + P 3) – P 2 ? (P 1 + P 3) – P 2 = (<0, 1, 1, 0, 1, 1> + <1, 0, 0, 1, 0, 0>) - <1, 0, 0, 0, 1, 1> = <1, 1, 1, 1> - <1, 0, 0, 0, 1, 1> = <0, 1, 1, 1, 0, 0> = P 4

Total “Possible” Logical Paths can be Big! s 1 There are 5 choices each

Total “Possible” Logical Paths can be Big! s 1 There are 5 choices each time we process through this loop. For passing through the loop n times we have 5 n possibilities of logical paths. c 1 1 c 2 2 s 2 4 c 3 5 3 If we go through the loop just 3 times, we have (5)3 = 125 possible paths! s 3 c 4 s 4 You also note that for n loops there may be 3 n decisions. e. g. : We have 3*3 = 9 decisions for looping 3 times.

Paths Analysis • Interested in total number of all possible combinations of “logical” paths

Paths Analysis • Interested in total number of all possible combinations of “logical” paths • Interested in Linearly Independent paths • Interest in Branch coverage or DD-path • Interested in Statement coverage