CIS 4932 Software Testing WhiteBox Testing 22000 Unit

CIS 4932 Software Testing White-Box Testing 2/2000 Unit Test Concepts

WHITE-BOX TESTING Stimuli Software under Test Response(s) Testing to ensure that software does not do what is not supposed to do. Test ALL of it! No-Surprise Tick -- Tick. Software!! -- Tick 2 2/2000 Unit Test Concepts

WHITE-BOX TESTING Stimuli Software under Test Response(s) • Focus is thorough execution of program elements during the testing process. • Warning: Tests only what is built, not what was intended! 3 2/2000 Unit Test Concepts

WHITE-BOX TESTING • Concept of coverage. Numeric measure of thoroughness of testing, relative to • Statements • Branches • Conditions • Paths 4 2/2000 Unit Test Concepts

CONTROL FLOW GRAPH • Defines the flow of control through unit. 1 2 4 3 5 5 2/2000 Unit Test Concepts

CONTROL FLOW GRAPH TERMINOLOGY • 5 NODES 1 • sequential blocks of code terminated by a branch • 3 PATHS: 2 4 3 • [1, 2, 3, 5], [1, 2, 5], [1, 4, 5] 5 • 2 BRANCHES • 1 and 2 are decision nodes 6 2/2000 Unit Test Concepts

CONTROL FLOW GRAPH COVERAGE • One test case forces execution of one path (red) • Paths are determined by branches (decision nodes) • A thorough test set forces execution of all paths (red, green, blue). 1 2 4 3 5 7 2/2000 Unit Test Concepts

COVERAGE LEVELS (%) • Statement -- a statement has been executed at least once during testing • Branch -- each outcome of a branch has been performed at least once during testing • Path -- a path through the code has been executed at least once during testing • Condition -- a condition has evaluated to true and to false at least once during testing 8 2/2000 Unit Test Concepts

CONTROL FLOW GRAPH COVERAGE MEASUREMENT • For 2 test cases (red, green) • Node (statement) cov = 4/5 • Branch cov = 1/2 [2] • Path cov = 2/3 • Acceptable coverage levels 1 2 4 3 5 • Statement cov = 90% • Branch cov = 80% • Path cov = 70% 9 2/2000 Unit Test Concepts

BRANCH vs CONDITION COVERAGE • Code example 1 if (x<1 && y>1) x = x + y; else y = y - x; 2 4 • 100% Branch coverage [1] • (x=0, y=2), (x=1, y=2) [TT, FT] • But not 100% Condition coverage • Need case TF (x=0, y=1) 10 2/2000 Unit Test Concepts

THE PROBLEM WITH COMPOUND CONDITIONS • Makes complex logic appear simpler than it really is • Test cases may be omitted • Logic results in 3 paths, not 2!! if (x<1) {if (y>1) x=x+y; else y=y-x; } else y=y-x; 1 2 3 5 4 11 2/2000 Unit Test Concepts

THE PROBLEM WITH PATH COVERAGE • Not all paths are feasible • No test case can force path [1, 2, 3, 4, 5]. Consecutive decisions mutually exclusive. if (x<1) y=2; if (x >= 1) y=3; z=y; 1 2 3 4 5 12 2/2000 Unit Test Concepts

Measuring Path Testing Difficulty 1. Mc. Cabe metric -- logical code complexity 1. Formula: 1 + #decisions in control flow graph 2. Test Significance: #basis paths through code 3. Design use: complexity of code 4. Test use: min #test cases for 100% path coverage 2. Mc. Cabe measures test (development) difficulty 13 2/2000 Unit Test Concepts

How To Design White Box Tests • Test cases must execute different paths • Decision tables • Rows -- elementary conditions in the code • Columns -- combinations of conditions in the code • Column based test case forces flow through different logic paths in the code • Decision table built from code reflects what was built versus intended (from spec) • Decision analysis for white-box testing. 14 2/2000 Unit Test Concepts

WHITE-BOX TESTING TOOLS • Tools instrument source code and gathers coverage data when tests • Compiled languages • Script languages -- coming to market • Some provide test case design assistance • List of paths not covered • Data conditions to force branch/path • Graphic depiction of graph coverage 15 2/2000 Unit Test Concepts

Problem P 1 Code (v 1) & Decision Table Age>80 | Y N N N if (Age>80 || Weight>300 | - Y N N Weight>300) Age<=12 | Y N N N return 0; Age>65 | Y N N if (Age <= 12) return 1; Weight<120 | Y N if (Age > 65) Pills = | 0 0 1 2 2 C return 2; ---------------if (Weight < 120) return 2 Note: C: 2+(Weight/120)/50 else return 2+(Weight-120)/50; Mc. Cabe = 6 16 2/2000 Unit Test Concepts

Problem P 1 Code (v 2) & Decision Table Age>80 | Y N N N Weight>300 | - Y N N N if (Age>80 || Age<=12 | Y N N N Weight>300) Age>65 | Y N N return 0; if (Age <= 12) Weight<120 | - Y N return 1; Pills = | 0 0 0 1 2 2 C if (Age > 65 || (Age<=65 && Weight<120)) return 2; return 2+(Weight-120)/50; Mc. Cabe = 7 17 2/2000 Unit Test Concepts

Your Turn -- White-Box Testing (1) Construct Decision Table ______ | pills=0; if (Age < 80 && Weight <300) { pills=1; if (Age >= 65) pills=2; else if (Age > 12) pills=2+(Weight-120)/50; } return pills; ____________ | Pills = | - - - --------------Note: C: 2+(Weight/120)/50 18 2/2000 Unit Test Concepts

Your Turn -- P 1 (2) Derive White-Box Test Cases Case Age 1 2 3 4 5 6 7 8 9 ___ ___ ___ Weight ___ ___ ___ Pills ___ ___ ___ 19 2/2000 Unit Test Concepts

OBSERVATIONS -WHITE-BOX TEST CASES • Code may not be complete with respect to input combinations from the specification • Decision table constructed from code is simpler -- subset of black-box table • Claim: black-box test cases force coverage of logic • Unless the code implements the wrong (a different) function 20 2/2000 Unit Test Concepts

MAIN POINTS WHITE-BOX TESTING • White-box = logic testing • Limitation: can't tell what's missing • Don't forget exceptions -- throwing, catching, propagating (debugger) • Perform decision analysis of code • Coverage tools help. • Use black-box test cases. 21 2/2000 Unit Test Concepts