Using Visual Models for Test Case Design Robert

Using Visual Models for Test Case Design Robert Sabourin President Ami. Bug. Com, Inc. Montreal, Canada rsabourin@amibug. com 12/14/2021 © Robert Sabourin, 2008 1 Ami. Bug. Com, Inc.

Visual Models • Robert Sabourin , Software Evangelist • President • Ami. Bug. Com Inc. • Montreal, Quebec, Canada • rsabourin@amibug. com • www. amibug. com 12/14/2021 © Robert Sabourin, 2008 2 Ami. Bug. Com, Inc.

Edsger W. Dijkstra • “Program testing can be used to show the presence of bugs, but never to show their absence” 12/14/2021 © Robert Sabourin, 2008 3 Ami. Bug. Com, Inc.

Tony Buzan 12/14/2021 © Robert Sabourin, 2008 4 Ami. Bug. Com, Inc.

Visual Models Testing Approaches • Equivalence Classes • Usage Scenarios • Decision Tables • State Models • Quality Factors 12/14/2021 © Robert Sabourin, 2008 5 Ami. Bug. Com, Inc.

Visual Models • Requirements – Use cases – Functional requirements – Quality factors – Constraints – Written requirements – Implicit requirements 12/14/2021 © Robert Sabourin, 2008 6 Ami. Bug. Com, Inc.

Visual Models • Usage Scenarios – Identify classes of users – Identify how users will use system – Describe scenarios – Use Story board or similar approaches – Identify variations 12/14/2021 © Robert Sabourin, 2008 7 Ami. Bug. Com, Inc.

Quality Factors Importance Different Application Types 12/14/2021 © Robert Sabourin, 2008 8 Ami. Bug. Com, Inc.

Taking AIM Equivalence Classes 12/14/2021 © Robert Sabourin, 2008 9 Ami. Bug. Com, Inc.

Taking AIM • Equivalence class – – – A subset of all possible test values to a variable Each member assumed provide the same info Each variable may have many classes Equivalence class are not mutually exclusive Focus testing Reduce the number of test cases 12/14/2021 © Robert Sabourin, 2008 10 Ami. Bug. Com, Inc.

Sets – Venn Diagram Equivalence Classes a b c a d b b a c f d c e 12/14/2021 © Robert Sabourin, 2008 11 Ami. Bug. Com, Inc.

Equivalence Classes – Sources • Requirements – – Business logic Capabilities Ranges Constraints • Code – Decisions – Intermediate computations • Data 12/14/2021 – Input fields – Database – Internal structures © Robert Sabourin, 2008 12 Ami. Bug. Com, Inc.

AIM - Equivalence Classes 12/14/2021 © Robert Sabourin, 2008 13 Ami. Bug. Com, Inc.

Example: Date Field Equivalence Classes Mind Map 12/14/2021 © Robert Sabourin, 2008 14 Ami. Bug. Com, Inc.

Example: Claim Amount Equivalence Classes Mind Map 12/14/2021 © Robert Sabourin, 2008 15 Ami. Bug. Com, Inc.

Boundary Testing • Test with extreme values – Lower & upper boundaries – Any edge conditions – Above and below extreme values 12/14/2021 © Robert Sabourin, 2008 16 Ami. Bug. Com, Inc.

Taking AIM Usage Scenarios 12/14/2021 © Robert Sabourin, 2008 17 Ami. Bug. Com, Inc.

Scenario Based Testing • Scenarios – Typical, real, usage scenarios for each user type • Could be based on Use Case Analysis or Story Boards – Individual scenarios can cover many functions! 12/14/2021 © Robert Sabourin, 2008 18 Ami. Bug. Com, Inc.

12/14/2021 © Robert Sabourin, 2008 19 Ami. Bug. Com, Inc.

12/14/2021 © Robert Sabourin, 2008 20 Ami. Bug. Com, Inc.

12/14/2021 © Robert Sabourin, 2008 21 Ami. Bug. Com, Inc.

Scenario Based Testing • User Experience Test Case q. Parameterize experience q. Walk through scenario from start to end q. Use pre selected input for each case q. Always run every test as if it were a user experience 12/14/2021 © Robert Sabourin, 2008 22 Ami. Bug. Com, Inc.

Scenario Based Testing 12/14/2021 © Robert Sabourin, 2008 23 Ami. Bug. Com, Inc.

TOFT Testing • Task Oriented Functional Testing – Can the user accomplish useful tasks correctly? 12/14/2021 © Robert Sabourin, 2008 24 Ami. Bug. Com, Inc.

Taking AIM Decision Tables 12/14/2021 © Robert Sabourin, 2008 25 Ami. Bug. Com, Inc.

Decision Tables Logic Modeling q q q 12/14/2021 Application Logic Business Rules Regulations Multiple Conditions Multiple Actions © Robert Sabourin, 2008 26 Ami. Bug. Com, Inc.

Decision Tables Conditions Rules Actions 12/14/2021 © Robert Sabourin, 2008 27 Ami. Bug. Com, Inc.

www. wikipedia. com Decision Tables 12/14/2021 © Robert Sabourin, 2008 28 Ami. Bug. Com, Inc.

Decision Tables Construction 1. 2. 3. 4. 5. 12/14/2021 Identify Conditions Identify Actions Relate Conditions to Actions with Rules Logic Reduction Tests each Rule © Robert Sabourin, 2008 29 Ami. Bug. Com, Inc.

Decision Tables Example 12/14/2021 © Robert Sabourin, 2008 30 Ami. Bug. Com, Inc.

Decision Tables Example 12/14/2021 © Robert Sabourin, 2008 31 Ami. Bug. Com, Inc.

Decision Table Example 12/14/2021 © Robert Sabourin, 2008 32 Ami. Bug. Com, Inc.

Decision Table Example Value Ranges 12/14/2021 © Robert Sabourin, 2008 33 Ami. Bug. Com, Inc.

Decision Table Example Binary Ranges 12/14/2021 © Robert Sabourin, 2008 34 Ami. Bug. Com, Inc.

Taking AIM State Models 12/14/2021 © Robert Sabourin, 2008 35 Ami. Bug. Com, Inc.

Gerald M. Weinberg “A state is a situation which can be recognized if it occurs again” An Introduction to General Systems Thinking Dorset House 12/14/2021 © Robert Sabourin, 2008 36 Ami. Bug. Com, Inc.

State Models “Stateful” Systems q q q 12/14/2021 Transactions Embedded Systems Process Workflow User Interface © Robert Sabourin, 2008 37 Ami. Bug. Com, Inc.

State Models Transition i State B State A Transition j 12/14/2021 © Robert Sabourin, 2008 38 Ami. Bug. Com, Inc.

State Models Construction 1. 2. 3. 4. 5. 6. 12/14/2021 Identify: States Identify: Transitions Identify: Triggers and Outcomes Test: Get to Each State Test: Exercise Each Transition Test: Cover Each Path © Robert Sabourin, 2008 39 Ami. Bug. Com, Inc.

State Models San Diego Singles Pattern Transition i State A Transition k State B State C Transition j Transition l 12/14/2021 © Robert Sabourin, 2008 40 Ami. Bug. Com, Inc.

State Models San Diego Singles Pattern Get Together Single Departed Together Non Existing Separate Come to be 12/14/2021 © Robert Sabourin, 2008 41 Ami. Bug. Com, Inc.

State Models Stack Pattern Attempted Pop Push Empty 12/14/2021 Push beyond Max Loaded Pop beyond Min Attempted Push Pop © Robert Sabourin, 2008 Full Pop 42 Ami. Bug. Com, Inc.

State Models Coffee Machine power up service needed no cups OR no coffee OR sensor jam reset button coin inserted coin return cup removed coin return make coffee 12/14/2021 idle button pushed © Robert Sabourin, 2008 inserting coins right amount entered user choose 43 Ami. Bug. Com, Inc.

State Models Coffee Machine Test. Id t. Tid 01 t. Tid 02 t. Tid 03 t. Tid 04 t. Tid 05 t. Tid 06 Start State Service Idle Inserting Choose Make Input Rest Coin Right $ Button No cup Cup Outcome ? ? ? End State Idle Inserting Choose Make Service Idle Tabular Form 12/14/2021 © Robert Sabourin, 2008 44 Ami. Bug. Com, Inc.

State Models Coffee Machine power up service needed reset button coin inserted idle inserting coins coin return no cups OR no coffee OR sensor jam Cup removed make coffee coin return button pushed right amount entered user choose • Is model really complete? • How do you get to power down state? • How do we count coins? • How about bad coins? 12/14/2021 © Robert Sabourin, 2008 45 Ami. Bug. Com, Inc.

State Models Web Application • Each page can be considered a State • User actions trigger transitions • Variable output is the outcome 12/14/2021 © Robert Sabourin, 2008 46 Ami. Bug. Com, Inc.

State Models Web Application • Example: mail. yahoo. ca 12/14/2021 © Robert Sabourin, 2008 Starting Page 47 Ami. Bug. Com, Inc.

State Models Web Application • Example: mail. yahoo. ca 12/14/2021 © Robert Sabourin, 2008 Valid ID, Password 48 Ami. Bug. Com, Inc.

State Models Web Application • Example: mail. yahoo. ca 12/14/2021 © Robert Sabourin, 2008 Mail Session Page 49 Ami. Bug. Com, Inc.

State Models Web Application • Example: mail. yahoo. ca 12/14/2021 © Robert Sabourin, 2008 Invalid ID, Password 50 Ami. Bug. Com, Inc.

State Models Web Application • Example: mail. yahoo. ca 12/14/2021 © Robert Sabourin, 2008 ID Error Page 51 Ami. Bug. Com, Inc.

State Models Web Application • Example: mail. yahoo. ca Acknowledged Valid ID Mail Session Start Completed Session 12/14/2021 © Robert Sabourin, 2008 ID Error Invalid ID 52 Ami. Bug. Com, Inc.

Null Request 12/14/2021 els Paid Us er C an ce ls nc Policy Activated Issued Us er C an ce Ca Receive Payment Approved Us Us er Create Policy Order submitted Credit and Risks OK State Models Transaction ls er C an ce ls Cancel Active Late Pay Warning No Pay © Robert Sabourin, 2008 53 Ami. Bug. Com, Inc.

State Models Tools of the Trade q q q 12/14/2021 Baskets Index Cards Excel Visio Power Point Case Tools © Robert Sabourin, 2008 54 Ami. Bug. Com, Inc.

Taking AIM Quality Factors 12/14/2021 © Robert Sabourin, 2008 55 Ami. Bug. Com, Inc.

Usability Testing • Testing done to help design effective user interfaces • Part of software design process • Generally done by human factors and ergonomic experts • Checklist approaches, used in system testing 12/14/2021 © Robert Sabourin, 2008 56 Ami. Bug. Com, Inc.

Taking AIM 12/14/2021 © Robert Sabourin, 2008 57 Ami. Bug. Com, Inc.

Performance Testing • Verify and measure applications performance • Response time from input event to outcome • Throughput or volume (transactions/time) • Operational characteristics • Often automated! 12/14/2021 © Robert Sabourin, 2008 58 Ami. Bug. Com, Inc.

Stress Testing • Testing operational characteristics of application within a harshly constrained environment – Limit processor speed – Limit memory – Limit disk space – Diminish access to shared resources 12/14/2021 © Robert Sabourin, 2008 59 Ami. Bug. Com, Inc.

Analysis & Design Quebec City Bridge, 1916 – Construction collapse – Stress due to scale Tacoma Narrows, 1940 12/14/2021 – Collapse during normal operation – Stress due to instability – Wind was only 42 mph © Robert Sabourin, 2008 60 Ami. Bug. Com, Inc.

Taking AIM • Quality Factors – Scale – Meter – Goal 12/14/2021 © Robert Sabourin, 2008 61 Ami. Bug. Com, Inc.

Taking AIM 12/14/2021 © Robert Sabourin, 2008 62 Ami. Bug. Com, Inc.

Measurements • Response time – Minimum – Maximum – Average • CPU Usage • Memory – available – page faults/second • Disk – % Disk Full 12/14/2021 © Robert Sabourin, 2008 63 Ami. Bug. Com, Inc.

Measurements • Network – Bandwidth • Web Servers – Files/Sec – Bytes/Sec – Maximum Connections – Errors 12/14/2021 © Robert Sabourin, 2008 64 Ami. Bug. Com, Inc.

Measurements • Database Servers – Transactions/Second – Cache hit ratios • Functionality – Pass? – Fail? – Relation to load 12/14/2021 © Robert Sabourin, 2008 65 Ami. Bug. Com, Inc.

Taking AIM 12/14/2021 © Robert Sabourin, 2008 66 Ami. Bug. Com, Inc.

Taking AIM Comments about Mind Maps 12/14/2021 © Robert Sabourin, 2008 67 Ami. Bug. Com, Inc.

Mind Maps Some Benefits 1. 2. 3. 4. 5. 12/14/2021 Visual Communication Creative Design Documentation Note taking © Robert Sabourin, 2008 68 Ami. Bug. Com, Inc.

Mind Maps Some Risks 1. 2. 3. 4. 12/14/2021 Models wrong but useful May be too big Possible incorrect focus Reuse out of context © Robert Sabourin, 2008 69 Ami. Bug. Com, Inc.

Thank You • Questions? 12/14/2021 © Robert Sabourin, 2008 70 Ami. Bug. Com, Inc.