COSC 4506ITEC 3506 Software Engineering Dataflow Testing DataFlow

COSC 4506/ITEC 3506 Software Engineering Data-flow Testing

Data-Flow Testing 4 Data-flow testing uses the control flowgraph to explore the unreasonable things that can happen to data (i. e. , anomalies). 4 E. g. , Pick enough paths to assure that: 4 Every data object has been initialized prior to its use. 4 All defined objects have been used at least once. 4 Consideration of data-flow anomalies leads to test path selection strategies that fill the gaps between complete path testing and branch or statement testing. 2

Data Object (variable) Categories 4 (d) Defined, Created, Initialized 4 (k) Killed, Undefined, Released 4 (u) Used: 4 (c) Used in a calculation 4 (p) Used in a predicate 4 ~x : all prior actions are not of interest to x 4 x~: all post actions are not of interest to x 3

(d) Defined Objects 4 An object (e. g. , variable) is defined when it: 4 appears in a data declaration 4 is assigned a new value 4 is a file that has been opened 4 is dynamically allocated 4. . . 4

(u) Used Objects 5 4 An object is used when it is part of a computation or a predicate. 4 A variable is used for a computation (c) when it appears on the RHS (sometimes even the LHS in case of array indices) of an assignment statement or as an output value 4 A variable is used in a predicate (p) when it appears directly in that predicate. (used to determine the immediate execution path)

Example: Definition and Uses What are the definitions and uses for the program below? (definition occurrence, or use occurrence) 1. 2. 3. 4 5. 6. 6 read (x, y); z = x + 2; if (z < y) w = x + 1; else y = y + 1; print (x, y, w, z);

Example: Definition and Uses 1. 2. 3. 4 read (x, y); z = x + 2; if (z < y) w = x + 1; else 5. y = y + 1; 6. print (x, y, w, z); 7 Def C-use P-use x, y z x z, y w x y y x, y, w, z

Data Flow Testing Monitors the lifecycle of a piece of data and looks out for appropriate usages of data during definition, use in predicates, uses in computation and termination (killing). It identifies potential bugs by examining the patterns in that piece of data is used. 8

Data Object (variable) Categoriesagain 4 (d) Defined, Created, Initialized 4 (k) Killed, Undefined, Released 4 (u) Used: 4 (c) Used in a calculation 4 (p) Used in a predicate 4 ~x : all prior actions are not of interest to x 4 x~: all post actions are not of interest to x 9

Testing Anomalies Table of possible anomalies Anomaly ~d First define du Defineuse Explanation Allowed dk Potential bug. . Date is killed without use after definition ~u ud 10 uk ~k Definekill Allowed. Normal case.

Example: Bill Application A billing application calculates the bill of cell customer based on his usage: If his bill is more than $100, 10% discount is given and more: Usage (min) Bill (Dollar) <100 $40 101 -200 50 cents for every additional minute. 10 cents for every additional minute. >200 11

Example: Bill Application Usage (min) public static double calculate. Bill (int Usage) { <100 double Bill = 0; if(Usage > 0) 101 -200 { Bill = 40; } >200 if(Usage > 100) { if(Usage <= 200) { Bill = Bill + (Usage - 100) * 0. 5; } else { Bill = Bill + 50 + (Usage - 200) * 0. 1; if(Bill >= 100) { Bill = Bill * 0. 9; } 12 } } } return Bill; Bill (Dollar) $40 50 cents for every additional minute. 10 cents for every additional minute.

Example: Bill Application Control-flow Graph Annotated control-flow graph for “Bill” Annotated control-flow graph for “Usage” 13

Example: Bill Application For variable ‘Usage’, the define-use-kill patterns are: • ~ define : normal case • define-use : normal case • use-kill : normal case 14

Example: Bill Application For variable ‘Bill’, the define-use-kill patterns are • ~ define : normal case • define-define: suspicious • define-use : normal case • use-define : acceptable • use-use : normal case • use-kill : normal case The static data-flow testing for the given application discovered the following anomaly: Bill: define – define 15

Example: Bill Application Static analysis for variable “Bill” Anomaly ~d 0 -1 dd 0 -1 -2 -3 du 3 -4 -5 -6 Explanation Allowed. Normal case Potential bug. Double definition. Allowed. Normal case. ud 6 uk dd uu k~ 10 -11 1 -2 -3 7 -8 11 Allowed. Data is used and then redefined. Allowed. Potential bug. Double definition. Allowed. Normal case. 16 Referring to the Table, we observe that static data-flow testing for variable ‘Bill’ discovered the following usage 0 -1 -2 -3 as a potential bug.

Example: Bill Application Static analysis for variable “usage” Anomaly ~d 0 du 0 -1 -2 uk 9 -10 -11 Explanation Allowed. Normal case. Allowed. uu k~ Allowed. Normal case. 5 -6 11 17 Referring to the Table, we observe that static data-flow testing for variable ‘Usage’ did not discover any bugs.

Static vs Dynamic Anomaly Detection 4 Static Analysis is analysis done on source code without actually executing it. 4 E. g. , Syntax errors are caught by static analysis. 18

Static vs Dynamic Anomaly Detection (Cont’d) 4 Dynamic Analysis is analysis done as a program is executing and is based on intermediate values that result from the program’s execution. 4 E. g. , A division by 0 error is caught by dynamic analysis. 4 If a data-flow anomaly can be detected by static analysis then the anomaly does not concern testing. (Should be handled by the compiler. ) 19

Anomaly Detection Using Compilers 4 Compilers are able to detect several data-flow anomalies using static analysis. 4 E. g. , By forcing declaration before use, a compiler can detect anomalies such as: 4~u 4 ku 4 Optimizing compilers are able to detect some dead variables. 20

Is Static Analysis Sufficient? 4 Questions: 4 Why isn’t static analysis enough? 4 Why is testing required? 4 Could a good compiler detect all data-flow anomalies? 4 Answer: 21

Static Analysis Deficiencies 4 Current static analysis methods are inadequate for: 4 Dead Variables: Detecting unreachable variables is unsolvable in the general case. 4 Arrays: Dynamically allocated arrays contain garbage unless they are initialized explicitly. (-u anomalies are possible) 22

Static Analysis Deficiencies (Cont’d) 4 Pointers: Impossible to verify pointer values at compile time. 4 False Anomalies: Even an obvious bug (e. g. , ku) may not be a bug if the path along which the anomaly exists is unachievable. (Determining whether a path is or is not achievable is unsolvable. ) 23

Data-Flow Modeling 4 Data-flow modeling is based on the control flow graph. 4 Each link is annotated with: 4 symbols (e. g. , d, k, u, c, p) 4 sequences of symbols (e. g. , dd, du, ddd) 4 that denote the sequence of data operations on that link with respect to the variable of interest. 24

du Path Segments 4 A du Path is a path segment such that if the last link has a use of X, then the path is simple and definition clear. 25

def-use Associations 4 A def-use association is a triple (x, d, u, ), where: x is a variable, d is a node containing a definition of x, u is either a statement or predicate node containing a use of x, and there is a sub-path in the flow graph from d to u with no other definition of x between d and u. 26

Example: Def-Use Associations 1 read (x, y) 2 z=x+2 3 5 4 6 27 (x, 1, 2), (x, 1, 4), … (y, 1, (3, t)), (y, 1, (3, f)), (y, 1, 5), … (z, 2, (3, t)), . . . F T w=x+1 z<y Some Def-Use Associations: y=y+1 print (x, y, w, z)

Example: Def-Use Associations What are all the def-use associations for the program below? read (z) x=0 y=0 if (z 0) { x = sqrt (z) if (0 x && x 5) y = f (x) else y = h (z) } y = g (x, y) print (y) 28

Example: Def-Use Associations def-use associations for variable z. 29 read (z) x=0 y=0 if (z 0) { x = sqrt (z) if (0 x && x 5) y = f (x) else y = h (z) } y = g (x, y) print (y)

Example: Def-Use Associations read (z) x=0 def-use associations for variable y=0 x. if (z 0) { x = sqrt (z) if (0 x && x 5) else } y = g (x, y) print (y) 30 y = f (x) y = h (z)

Example: Def-Use Associations read (z) def-use associations for variable y. x=0 y=0 if (z 0) { x = sqrt (z) if (0 x && x 5) y = f (x) else y = h (z) } y=g (x, y) print (y) 31

Definition-Clear Paths 4 A path (i, n 1, . . . , nm, j) is called a definition-clear path with respect to x from node i to node j if it contains no definitions of variable x in nodes (n 1, . . . , nm , j). 4 The family of data flow criteria requires that the test data execute definition-clear paths from each node containing a definition of a variable to specified nodes containing c-use and edges containing p-use of that variable. 32

Data-Flow Testing Strategies 4 All du Paths (ADUP) 4 All Uses (AU) 4 Others not covered in this course … 33

All du Paths Strategy (ADUP) 4 ADUP is one of the strongest data-flow testing strategies. 4 ADUP requires that every du path from every definition of every variable to every use of that definition be exercised under some test All du Paths Strategy (ADUP). 34

An example: All-du-paths What are all the du-paths in the following program ? read (x, y); for (i = 1; i <= 2; i++) print (“hello”); Sa ; if (y < 0) S b; else print (x); 35

An example: All-du-paths 1 2 3 T print(“hello”) 4 6 print x Sb i <= 2 7 8 F 5 Sa 9 i=i+1 6 36 F T read (x, y) i=1 y<o

Example: pow(x, y) b 1 a 5 8 c 37 g d 9 f e 14 16 h i 17

Example: pow(x, y) du-Path for Variable x /* pow(x, y) This program computes x to the power of y, where x and y are integers. INPUT: The x and y values. OUTPUT: x raised to the power of y is printed to stdout. */ 1 void pow (int x, y) 2 { 3 float z; 4 int p; b 5 if (y < 0) 6 p = 0 – y; a f d 7 else p = y; 1 5 8 9 8 z = 1. 0; 9 while (p != 0) c 10 { e 11 z = z * x; 12 p = p – 1; 13 } 14 if (y < 0) 15 z = 1. 0 / z; 16 printf(z); 17 } 38 g 14 16 h i 17

Example: pow(x, y) du-Path for Variable x /* pow(x, y) This program computes x to the power of y, where x and y are integers. INPUT: The x and y values. OUTPUT: x raised to the power of y is printed to stdout. */ 1 void pow (int x, y) 2 { 3 float z; 4 int p; b 5 if (y < 0) 6 p = 0 – y; a f d 7 else p = y; 1 5 8 9 8 z = 1. 0; 9 while (p != 0) c 10 { e 11 z = z * x; 12 p = p – 1; 13 } 14 if (y < 0) 15 z = 1. 0 / z; 16 printf(z); 17 } 39 g 14 16 h i 17

Example: pow(x, y) du-Path for Variable y /* pow(x, y) This program computes x to the power of y, where x and y are integers. INPUT: The x and y values. OUTPUT: x raised to the power of y is printed to stdout. */ 1 void pow (int x, y) 2 { 3 float z; 4 int p; b 5 if (y < 0) 6 p = 0 – y; a f d 7 else p = y; 1 5 8 9 8 z = 1. 0; 9 while (p != 0) c 10 { e 11 z = z * x; 12 p = p – 1; 13 } 14 if (y < 0) 15 z = 1. 0 / z; 16 printf(z); 17 } 40 g 14 16 h i 17

Example: pow(x, y) du-Path for Variable y /* pow(x, y) This program computes x to the power of y, where x and y are integers. INPUT: The x and y values. OUTPUT: x raised to the power of y is printed to stdout. */ 1 void pow (int x, y) 2 { 3 float z; 4 int p; b 5 if (y < 0) 6 p = 0 – y; a f d 7 else p = y; 1 5 8 9 8 z = 1. 0; 9 while (p != 0) c 10 { e 11 z = z * x; 12 p = p – 1; 13 } 14 if (y < 0) 15 z = 1. 0 / z; 16 printf(z); 17 } 41 g 14 16 h i 17

Example: pow(x, y) du-Path for Variable y /* pow(x, y) This program computes x to the power of y, where x and y are integers. INPUT: The x and y values. OUTPUT: x raised to the power of y is printed to stdout. */ 1 void pow (int x, y) 2 { 3 float z; 4 int p; b 5 if (y < 0) 6 p = 0 – y; a f d 7 else p = y; 1 5 8 9 8 z = 1. 0; 9 while (p != 0) c 10 { e 11 z = z * x; 12 p = p – 1; 13 } 14 if (y < 0) 15 z = 1. 0 / z; 16 printf(z); 17 } 42 g 14 16 h i 17

All Uses Strategy (AU) 4 AU requires that at least one path from every definition of every variable to every use of that definition be exercised under some test. 4 Hence, at least one definition-clear path from every definition of every variable to every use of that definition be exercised under some test. 4 Clearly, AU < ADUP. 43

Example: Bill Application A billing application calculates the bill of cell customer based on his usage: If his bill is more than $100, 10% discount is given and more: Usage (min) Bill (Dollar) <100 $40 101 -200 50 cents for every additional minute. 10 cents for every additional minute. >200 44

Example: Bill Application Usage (min) public static double calculate. Bill (int Usage) { <100 double Bill = 0; if(Usage > 0) 101 -200 { Bill = 40; } >200 if(Usage > 100) { if(Usage <= 200) { Bill = Bill + (Usage - 100) * 0. 5; } else { Bill = Bill + 50 + (Usage - 200) * 0. 1; if(Bill >= 100) { Bill = Bill * 0. 9; } 45 } } } return Bill; Bill (Dollar) $40 50 cents for every additional minute. 10 cents for every additional minute.

Example: Bill Application Control Flow Graph Annotated CFG for variable “Bill” Annotated CFG for variable “usage” Data Flow Strategy for each variable Test suites for each variable 46

Effectiveness of Strategies 4 Ntafos compared Random, Branch, and All uses testing strategies on 14 Kernighan and Plauger programs are a set of mathematical programs with known bugs that are often used to evaluate test strategies. 4 Ntafos conducted two experiments: 47

Results of 2 of the 14 Ntafos Experiments 48 Strategy Mean Number of Test Cases Percentage of Bugs Found Random 35 93. 7 Branch 3. 8 91. 6 All Uses 11. 3 96. 3 Strategy Mean Number of Test Cases Percentage of Bugs Found Random 100 79. 5 Branch 34 85. 5 All Uses 84 90. 0

Data-Flow Testing Tips 4 Resolve all data-flow anomalies. 4 Try to do all data-flow operations on a variable within the same routine (i. e. , avoid integration problems). 4 Use strong typing and user defined types when possible. 4 Use explicit (rather than implicit) declarations of data when possible. 4 Put data declarations at the top of the routine and return data objects at the bottom of the routine. 49

Summary 4 Data are as important as code. 4 Define what you consider to be a data-flow anomaly. 4 Data-flow testing strategies span the gap between all paths and branch testing. 4 AU has the best payoff for the money. It seems to be no worse than twice the number of required test cases for branch testing, but the results are much better. 4 Path testing with Branch Coverage and Data-flow testing with AU is a very good combination. 50