Unit 4 Domain Testing Compiled with reference from

Unit 4 – Domain Testing Compiled with reference from: Software Testing Techniques: Boris Beizer Craft of Software Testing: Brain Marrick

Domain Testing U 3 We will see in this part of Unit 3: • Domains & Paths • Nice & Ugly Domains • Domain Testing • Domains & Interfaces Testing • Domains & Testability ref boris beizer 2

Transaction-Flow Testing U 3 Contents of the unit • Domains • Model, as a set, paths • Predicates • closure, dimensionality, assumptions • Nice & ugly domains • specified & implemented domains • Nice domains – properties, testing implications • Ugly domains, simplification by programmers & testers • Domain testing • Bugs & testing for those bugs • Testing procedure • Variations, tools, effectiveness • Domains & Interface Testing • Range, closure, span • Interface range, domain compatibility • Finding the values • Domains & Testability • Linearizing, coordinate transformations. Canonical program form ref boris beizer 3

Domain Testing - Domains & Paths U 3 Domain Testing Model D 1 D 2 (x, y, z, …) CLASSIFY D 3 Dn DO CASE 1 DO CASE 2 (1, 2, 3, 4, 5, …) DO CASE 3 OUTPUT { Outcome } ref boris beizer INPUT DO CASE n 4

Domain Testing U 3 Domain Testing Verifies if the classification is correct ref boris beizer Views Programs as input data classifiers 5

Domain Testing - Domains & Paths U 3 Domain Testing Model Two Views • Functional Testing • Based on Implementation information • Structural Technique ref boris beizer • Based on Specs 6

Domain Testing - Domains & Paths U 3 • Variables • Simple combinations of two variables • Numbers from - to + • Input variables as numbers • Loop-free Programs ref boris beizer Domain Testing Model 7

Domain Testing - Domains & Paths U 3 Domain Testing Model • For each Input Case, • A Hypothetical path for Functional testing ref boris beizer • Structural Knowledge is not needed. Only Specs. • An Actual path for Structural testing 8

Domain Testing - Domains & Paths (simplified) • A Single Connected Set of numbers • No arbitrary discrete sets • Defined by the boundaries • One or more boundaries • Specified by predicates • Bugs likely at the boundaries • One or more variables D 1 ref boris beizer Domain U 3 9

Domain Testing - Domains & Paths U 3 Predicates • Interpretation ref boris beizer • Structural Testing - CFG • Functional Testing - DFG • Specifies the Domain Boundary • n Predicates in Sequence 2 n domains • Or, just two domains A. AND. B. AND. C 10

Domain Testing - Domains & Paths U 3 Paths • More paths for disconnected domains ref boris beizer • At least One PATH thru the Program 11

Domain Testing - Domains & Paths U 3 Summary: • For every domain, there is at least one path thru the routine, along which that domain’s processing is done ref boris beizer • Domain for a loop-free program corresponds to a set of numbers defined over the input vector • The set of interpreted predicates traversed on that path (ie. , the path’s predicate expression) defines the domain’s 12 boundaries.

Domain Testing - Domains & Paths - Domain Closure U 3 Boundary: Closed / Open X <= MAX X >= MIN MAX MIN D 2 D 1 D 3 X >= MIN D 1 MIN Closed D 2 ref boris beizer Closed X < MAX Open D 3 MAX 13

Domain Testing - Domains & Paths - Domain Closure Boundary: Closed / Open MAX D 2 D 1 X > MIN Open D 3 X < MAX ref boris beizer MIN 14

Domain Testing - Domains & Paths - Domain Dimensionality U 3 • One dimension per variable • Slicing Boundary ref boris beizer • At least one predicate • N-spaces are cut by Hyperplanes 15 • Slices thru previously defined Domain

Domain Testing - Domains & Paths – Bug Assumptions • Processing is OK. U 3 Domain definition may be wrong. Predicates are wrong. ref boris beizer Boundaries are wrong. • Once input vector is set on the right path, it’s correctly processed. • More bugs causing domain errors … 16

Domain Testing - Domains & Paths – Bug Assumptions. . U 3 • Double-zero representation ref boris beizer • Floating point zero check • Contradictory domains defined • Ambiguous domains undefined 17

Domain Testing - Domains & Paths – Bug Assumptions. . U 3 • Over-specified domains • Boundary errors Boundary closure Shifted ref boris beizer X > 3. AND. X < 2. AND. Y > 3 18

Domain Testing - Domains & Paths - Bug Assumptions U 3 • Boundary errors…. Missing ref boris beizer Tilted 19 Extra boundary

Domain Testing - Domains & Paths - Bug Assumptions U 3 X >= 3 ref boris beizer • Closure Reversal • Faulty Logic Simplification of compound predicates 20

ref boris beizer Domain Testing - Domains & Paths - Bug Assumptions U 3 Strategy for domain testing in 2 -dim 21

Domain Testing - Domains & Paths - Restrictions U 3 In General… DT cannot detect Example F 1(x) : +1 D 1 • Representative outcome Partition testing D 2 F 2(x) : -1 Input Equivalence ref boris beizer 1. Coincidental Correctness 22

Domain Testing - Domains & Paths U 3 Domain Testing Model INPUT (x, y, z, …) CLASSIFY DO CASE 1 OUTPUT D 2 (1, 2, 3, 4, 5, …) { Outcome } DO CASE 2 f 2 DO CASE 3 f 3 Dn ref boris beizer D 1 Function f 1 f 4 DO CASE n 23

Domain Testing - Domains & Paths - - Restrictions U 3 3. Simple boundaries & Compound predicates D 1 X X >=0. AND. X <= 16 16 D 1 Y Y = 1. AND. X >=0. AND. X <= 16 0 16 X ref boris beizer 0 24

Domain Testing - Domains & Paths - - Restrictions U 3 Compound predicates…. • Impact of . OR. • Adjacent domains with same function • Example ref boris beizer • Concave, Disconnected ABC + DEF 25 • Eliminate compound predicates

Domain Testing - Domains & Paths - Restrictions U 3 4. Functional Homogeneity of Bugs a x + b y >= c • Bugs are only in a, b, c ref boris beizer • Functional form still Retained 26

Domain Testing - Domains & Paths - - Restrictions U 3 5. Linear Vector Space • Conversion to linear vector space • 2 -d Polar co-ordinates ref boris beizer • Linear boundary predicate, Interpreted • Simple relational operators • Polynomials • Problems with Non-linear Boundaries 27

Domain Testing - Domains & Paths - Restrictions U 3 Predicate for each iteration Loop over the entire transaction Definite loop ref boris beizer 6. Loop-free Software 28

Domain Testing - Domains & Paths – Nice & Ugly domains U 3 Nice Domains Requirements Before DT ref boris beizer • Bugs ill-defined domains • Analyze specs • Make the Boundary Specs Consistent & Complete 29

Domain Testing - Domains & Paths – Nice domains U 3 Implemented Domains Specified Domains ref boris beizer • Complete, Consistent & Process all inputs • Incomplete, Inconsistent Programmer’s / Designer’s Effort 30

Domain Testing - Domains & Paths – Nice domains U 3 Nice Domains U 1 U 2 V 1 D 12 V 2 D 21 D 22 ref boris beizer • Linear, Complete, Systematic, Orthogonal, Consistently Closed, & Convex 31

Domain Testing - Domains & Paths – Nice domains U 3 Nice Domains • Ease of DT • Fewer Bug Occurrences ref boris beizer Advantages 32

Domain Testing - Domains & Paths – Nice domains U 3 Nice Domains 1. Linear 1. Complete 5. Closure consistency 6. Convex 2. Systematic 1. Orthogonal 7. Simply connected ref boris beizer Boundaries are 33

Domain Testing - Domains & Paths – Nice domains U 3 1. Linear Boundaries Interpreted linear inequalities n+1 Points n+1 + 1 Test Cases ref boris beizer n-dim Hyperplane: Non-Linear • Transform 34

Domain Testing - Domains & Paths – Nice domains U 3 2. Complete Boundaries • Span the total number space (- , + ) Incomplete… ref boris beizer • One set of Tests • Reasons • Tests 35

Domain Testing - Domains & Paths - Nice domains U 3 3. Systematic Boundaries • Linear Inequalities differing by a constant or, fj (X) ≥ g (j, c) = j + k * c • Parallel lines • Identical Sectors in a Circle ref boris beizer fj (X) ≥ kj • DT • Test a domain 36 Tests for other Domains

Domain Testing - Domains & Paths - Nice domains U 3 4. Orthogonal Boundaries or, boundary sets • Parallel to axes Uj Vj • DT ref boris beizer • Two boundaries • Each Set Independently • # Tests O (n) 37

Domain Testing - Domains & Paths - Nice domains U 3 Orthogonal Boundaries • Concentric circles with radial lines • Rectangular coordinates • Polar r ≥ aj. AND. r < aj+1. AND. ≥ j. AND. < j+1 ref boris beizer • Tilted sets • transformation • Test cases: O(n) 38

Domain Testing - Domains & Paths - Nice domains U 3 Non-Orthogonal Boundaries O ( n 2 ) + O (n) ref boris beizer • Test Intersections 39

Domain Testing - Domains & Paths - Nice domains U 3 5. Closure Consistency • Example • Same relational operator for systematic boundaries D 3 D 1 D 2 D 4 ref boris beizer • A Simple pattern in all boundary closures 40

Domain Testing - Domains & Paths - Nice domains U 3 6. Convex Domain • DT • n on-points & 1 off-pt Concave • “ But, However, Except, Or … “ in Specs • Handle with special care ref boris beizer • Line joining any two points lies with in the domain 41

Domain Testing - Domains & Paths - Nice domains U 3 7. Simply Connected Domain D 2 D 1 • 2 complementary domains • D 1: Convex • Programmer / Designer • Convex part First D 3 D 1 D 2: Concave, not-Connected D 2 ref boris beizer • In a single piece 42

Domain Testing - Domains & Paths – Ugly Domains U 3 Generally, Programmer / Designer Simplifies => Ugly to Good possibility of Introduction of more bugs ? !? ref boris beizer From Bad Specs 43

Domain Testing - Domains & Paths – Ugly Domains U 3 Causes 2. Ambiguities & Contradictions 3. Simplifying the topology 4. Rectifying boundary closures ref boris beizer 1. Non-linear Boundaries 44

Domain Testing - Domains & Paths – Ugly Domains U 3 1. Non-linear Boundaries ref boris beizer • Transform 45

Domain Testing - Domains & Paths – Ugly Domains U 3 2. Ambiguities • Holes in input vector space. • Detected by Specification languages & tools. ref boris beizer • Missing boundary 46

U 3 Domain Testing - Domains & Paths – Ugly Domains 2. Contradictions. . • Overlapping of • Closure Specs D 3 D 1 ref boris beizer • Domain Specs D 2 47

Domain Testing - Domains & Paths – Ugly Domains U 3 3. Simplifying the Topology…. • Concavity, Holes, Disconnectedness ref boris beizer • Complexity ! 48

Domain Testing - Domains & Paths – Ugly Domains U 3 3. Simplifying the Topology…. ref boris beizer • Smoothing out concavity • Filling in Holes 49

Domain Testing - Domains & Paths – Ugly Domains U 3 3. Simplifying the Topology…. Correct: • Connect disconnected boundary segments • Extend boundaries to infinity ref boris beizer • Joining the pieces 50

Domain Testing - Domains & Paths – Ugly Domains U 3 4. Rectifying Boundary Closures. • Force a Bounding Hyperplane to belong to the Domain. Consistent Direction Inclusion / Exclusion Consistency ref boris beizer • make closures in one direction for parallel boundaries with closures in both directions 51

Domain Testing - Domains & Paths – Domain Testing U 3 General DT Strategy 2. Define test strategy for each possible bug related to boundary 3. Test points for a domain useful to test its adjacent domain. 4. Run the tests. By post test analysis determine if any boundaries are faulty & if so how? 5. Run enough tests to verify every boundary of every domain ref boris beizer 1. Select test points near the boundaries. 52

Domain Testing - Domains & Paths – Domain Testing U 3 DT for Specific Domain Bugs Generally, • Exterior point • Epsilon neighborhood ref boris beizer • Interior point • Extreme point • On point 53

U 3 Domain Testing - Domains & Paths – Domain Testing DT for Specific Domain Bugs Off Points ref boris beizer Extreme point Boundary point Epsilon neighborhood On Points 54 Domain D 1

Domain Testing - Domains & Paths – Domain Testing U 3 1. 1 -d Domains 3. Equality & inequality Predicates 4. Random Testing ref boris beizer 2. 2 -d Domains 55 5. Testing n-dimensional Domains

U 3 Domain Testing - Domains & Paths – Domain Testing A B x Bugs with open boundaries B Closure Bug x B Shift left Bug Shift Right Bug A A x 1 x B ref boris beizer Testing 1 -d Domains : A x x 1 56

U 3 Domain Testing - Domains & Paths – Domain Testing 1 -d Domains : A x B Missing Boundary A x x Extra Boundary B x A C x x Bugs with Closed Boundaries : Similar to the above x 1 ref boris beizer B Bugs with open boundaries 57

Domain Testing - Domains & Paths – Domain Testing U 3 2. Testing 2 -d Domains • Boundary Shift : up / down • Tilted Boundary • Extra Boundary • Missing Boundary ref boris beizer • Closure bug 58

ref boris beizer Domain Testing - Domains & Paths – Domain Testing U 3 2 n : Domains share tests 3 n : no sharing of tests by domains 59 Strategy for domain testing in 2 -dim

U 3 Domain Testing - Domains & Paths – Domain Testing 3. Equality & Inequality Predicates • An Equality predicate defines a line in 2 -d A c’ c d B b To avoid bugs c ref boris beizer a 60

Domain Testing - Domains & Paths – Domain Testing U 3 4. Random Testing ref boris beizer • A Point in the center : verifies computation 61

Domain Testing - Domains & Paths – Domain Testing U 3 5. Testing n-Dimensional Domains (strategy) n-dimensions, p boundary segments • Extreme pt shared : 2 * p points • Equalities over m-dimensions create a subspace of n-m dimensions • Orthogonal domains with consistent boundary closures, orthogonal to the axes & complete boundaries • Independent testing ref boris beizer • (n+1)*p test cases : n on points & 1 off point 62

Domain Testing - Domains & Paths – Domain Testing U 3 Procedure for solving for values Simple procedure. Need tools. 2. Identify variables which appear in domain-defining predicates, such as control-flow predicates ref boris beizer 1. Identify input variables 63

Domain Testing - Domains & Paths – Domain Testing U 3 Procedure 4. Predicate expression with p # predicates. Find # domains : < 2 p 5. Solve inequalities for extreme points 6. Use extreme points to solve for nearby on points … ref boris beizer 3. Interpret all domain predicates in terms of input variables: • Transform non-linear to linear • Find data flow path 64

Domain Testing - Domains & Paths – Domain Testing U 3 Effectiveness • Cost effective • Hardware logic testing – tool intensive ref boris beizer • Bugs on boundaries, extreme points 65

Domain Testing - Domains & Paths – Domain Testing U 3 Domains & Interface Testing • Range Variable Domain Classify Range Function / Routine ref boris beizer • Domain 66

U 3 Domain Testing - Domains & Paths – Domain Testing Domains & Interface Testing Domain Routine 1 Range for Domain for Routine 1 Routine 2 • Span compatibility Routine 2 Range for Routine 2 ref boris beizer Variable 67

Domain Testing - Domains & Paths – Domain Testing U 3 Domains & Interface Testing ref boris beizer • Closure compatibility 68

Domain Testing - Domains & Paths – Domain Testing U 3 Interface Range/Domain Compatibility Testing • Find an inverse function ref boris beizer • Test each var independently • Build a super domain 69

Domain Testing - Domains & Paths – Domain Testing U 3 ref boris beizer Finding the values / solving inequalities 70

Domain Testing - Domains & Paths – Domain Testing U 3 Domains and Testability • Polynomial • Rectangular to polar • Generic & rational => Taylor series ref boris beizer • Linearizing transformations 71

Domain Testing - Domains & Paths – Domain Testing U 3 Domains and Testability Perform transformations for better testing. ref boris beizer • Co-ordinate transformations 72

Domain Testing - Domains & Paths – Domain Testing U 3 Domains and Testability ref boris beizer • Canonical Program Form 73

U 3 Domain Testing – Questions from the previous year’s exams 1. Explain Nice & Ugly Domains. 2. What is domain testing? Discuss applications of domain testing 4. What is the purpose of Domain Testing? Give its schematic representation 5. Define the following concepts. 1. Domains 2. Domain closure 3. Domain dimensionality Bug assumptions for Domain Testing 4. ref boris beizer 3. Explain the domain boundary bugs for two dimensional domains 6. Explain simple domain boundaries & compound predicates. 7. Classify what can go wrong with boundaries, then define a test strategy for each case in domain testing. 74

1. Discuss with example the equal - span range/Doman compatibility bugs. (8 M)* 2. Discuss in detail about testability of Domains. (8 M)* 3. What is meant by Domain Dimensionality. (8 M) 4. What is meant by nice - domain? Give an example for nice two – dimensional domain. (8 M). * 5. Discuss (8 M) i. Linear domain boundaries ii. Non linear domain boundaries iii. Complete domain boundaries iv. Incomplete domain boundaries 6. Explain various properties related to Ugly-domains. (8 M) * 7. State and Explain various restrictions at domain testing processes. (10 M) 8. What is meant by domain testing? Discuss the various applications of domain testing? (10 M) * 9. With a neat diagram, explain the schematic representation of domain testing. (6 M) 10. Explain how one-dimensional domains are tested? (10 M) 11. Discuss in detail the domains and interface testing. (16 M) ref boris beizer • • • • 75

To Unit 5…Paths, Path Products & Regular Expressions U-3 ref boris beizer Domain Testing 76