x Unit Test Patterns Adapted by JP from

x. Unit Test Patterns Adapted by JP from: Negar Koochakzadeh Venkat Mantripragada AND Meszaros’s book 1

Agile Development Cycles User Stories Task edit Working Software Test 2

System Under Test o It may have Depended-on components (DOCs) You may not be able to modify DOCs. Thus you need to figure out how to control and observe such components. Controllability and observability are essential for testing! 3

Four phase testing Image From: xunit Test Patterns, G. Meszaros 4

Test Case ‘Generation’ o Recorded Test (esp. for GUI) n When to use: o We do not expect a lot of changes for the system. o Scripted Test n We write it by hand o ‘Automated’ Test case generation n Model-based testing In this presentation we discuss the art of writing tests. 5

Recorded Test. Complete lets you quickly create automated tests by recording actions in desktop, web and mobile applications. Test. Complete can record data input, item selection, website navigation, form submission, mobile gestures and other user actions. You can also create checkpoints during test recording. After recording a test, you can review and improve it in Test. Complete built-in editors. The tool can then rerun your tests automatically. Demo: https: //www. youtube. com/watch? v=Tu 8 RL ORwjv. Y Image From: xunit Test Patterns, G. Meszaros 6

Scripted Test Notice the test scripts are directly embedded into the SUT. A tool like Junit is used to run them automatically. Here a fixture reduces to the annotation to mark a test method… Image From: xunit Test Patterns, G. Meszaros Demo 7

About Mockito: https: //www. youtube. com/watch? v=e. ILy 4 p 99 ac 8 https: //www. youtube. com/watch? v=Hs. Q 9 Ow. KA 79 s Terminology 8

Coding Objectives Comparison From: xunit Test Patterns, G. Meszaros 9

A Sobering Thought Answer: Test Automation!! 10

Goals of test automation o o o Improve quality Understand the SUT Reduce the risk (of error, of omission, of delay) Easy to run Easy to write Easy to maintain 11

Why are Tests so Important? (no value if there is no quality!) 12

Economics of Maintainability Image From: xunit Test Patterns, G. Meszaros 13

Principles in test automation o TDD: Write the test first o Each test should be: n n Small and simple Independent of other tests Repeatable Self-checking Fully Automated o First do “State verification” (controllability) o Make sure you are in the right state to test what you want to test and then “Behavior Verification” (observability) 14

Benefits of Automated Tests 15

? Which part can be automated? Image From: xunit Test Patterns, G. Meszaros 16

Common features of XUnit family o Specify a test as a test Method o Specify the expected results within the test method in the form of calls to Assertion Methods o Aggregate the tests into test suites that can be run as a single operation o Run one or more tests to get a report on the results of the test run. 17

Some XUnit Tools o o o o o C: C++: Delphi: Java. Script: Matlab: . Net: PHP: Python: CUnit, Check, RCUNIT CPPUnit, Cpp. Unit. Lite, Cxx. Test DUnit JUnit, Test. NG JSUnit ml. Unit cs. Unit, NUnit, Mb. Unit, x. Unit PHPUnit, Testilence Py. Unit, Trial 18

A Recipe for Success 1. Write some tests n start with the easy ones! JP: really? ? 2. Note the Test Smells that show up 3. Refactor to remove obvious Test Smells n Apply appropriate x. Unit Test Patterns 4. Write some more tests n possibly more complex 5. Repeat from Step 2 until: n All necessary tests are written n No smells remain 19

What is a Test Smell? ØA Smell is a symptom of a problem in a test code. ØNot necessarily the actual cause üThere may be many possible causes for the symptom üSome root causes may contribute to several different smells ØNot all problems are considered as smells üSmells must pass “Sniffability test” (ie be obvious to detect) (This is terminology from Kent Beck) 20

What is a Xunit Test Pattern? 21

Kinds of Test Smells o Code Smells n Recognized by looking at test code o Behavior smells n Effects the outcome of tests as they execute o Project Smells n Recognized by project managers. Root cause can be one or more code/ behavioral smells 22

Code Smells A problem visible when looking at test code: o Tests are hard to understand o Tests contain coding errors that may result in n Missed bugs n Erratic Tests o Tests are difficult or impossible to write n No test API on SUT n Cannot control initial state of SUT n Cannot observe final state of SUT o Sniff Test for Code Smells (as for any smell) n Problem with the test must be visible to test automater or test reader 23

General Code Smells o o o Obscure Test Conditional Test Logic Hard-to-Test Code Duplication Test Logic in Production 24

Obscure Test “Test is hard to understand” o Common Causes: Verbose Test n So much test code that it obscures the test intent Eager Test n Several tests merged into one Test Method General Fixture n Fixture contains objects irrelevant for this test Obtuse Assertion n Using the wrong kind of assertion Hard-Coded Test Data n Lots of “Magic Numbers” or Strings used when creating objects 25

Eager Testing too public void test. Flight. Mileage_as. Km 2() throws Exception many { // set up fixture // exercise constructor functionalities Flight new. Flight = new Flight(valid. Flight. Number); // verify constructed object assert. Equals(valid. Flight. Number, new. Flight. number); assert. Equals("", new. Flight. airline. Code); assert. Null(new. Flight. airline); new. Flight. set. Mileage(1122); // set up mileage int actual. Kilometres = new. Flight. get. Mileage. As. Km(); // try mileage translator int expected. Kilometres = 1810; assert. Equals( expected. Kilometres, actual. Kilometres); new. Flight. cancel(); // now try it with a canceled flight try { new. Flight. get. Mileage. As. Km(); fail("Expected exception"); } catch (Invalid. Request. Exception e) { assert. Equals( "Cannot get cancelled flight mileage", e. get. Message()); } } 26

Irrelevant Information public void test. Add. Item. Quantity_several. Quantity () { final int QUANTITY = 5; Address billing. Address = new Address("1222 1 st St SW", "Calgary", "Alberta", "T 2 N 2 V 2", "Canada"); Address shipping. Address = new Address("1333 1 st St SW", "Calgary", "Alberta", "T 2 N 2 V 2", "Canada"); Customer customer = new Customer(99, "John", "Doe", new Big. Decimal("30"), billing. Address, shipping. Address); Product product = new Product(88, "Some. Widget", new Big. Decimal("19. 99")); Invoice invoice = new Invoice(customer); // Exercise SUT invoice. add. Item. Quantity(product, QUANTITY); Hard to determine which values affect the outcome 27

Another Code Smell: Conditional Test Logic o Tests containing conditional logic (IF statements or loops) o Hard to verify correctness. o A cause of Buggy Tests (Project Smell) 28

Conditional Test Logic actual = null; // verify Vancouver is in the list i = flights. From. Calgary. iterator(); while (i. has. Next()) { Flight. Dto flight. Dto = (Flight. Dto) i. next(); if (flight. Dto. get. Flight. Number(). equals( expected. Calgary. To. Van. get. Flight. Number())) { actual = flight. Dto; assert. Equals("Flight from Calgary to Vancouver", expected. Calgary. To. Van, flight. Dto); break; } } } Which code path is the one actually executed 29

Another Code Smell: Test Code Duplication o Same code sequences appear many times in many tests o More code to modify when something changes o A cause of Fragile Tests (Behavior Smell) 30

Test Code Duplication public void test. Invoice_add. Two. Line. Items_same. Product() { Invoice inv = create. Anon. Invoice(); Line. Item exp. Item 1 = new Line. Item(inv, product, QUANTITY 1); Line. Item exp. Item 2 = new Line. Item(inv, product, QUANTITY 2); inv. add. Item. Quantity(product, QUANTITY 1); inv. add. Item. Quantity(product, QUANTITY 2); List line. Items = inv. get. Line. Items(); assert. Equals("number of items", line. Items. size(), 2); Line. Item actual = (Line. Item)line. Items. get(0); // Verify first item assert. Equals(exp. Item 1. get. Inv(), actual. get. Inv()); assert. Equals(exp. Item 1. get. Prod(), actual. get. Prod()); assert. Equals(exp. Item 1. get. Quantity(), actual. get. Quantity()); actual = (Line. Item)line. Items. get(1); // Verify second item assert. Equals(exp. Item 2. get. Inv(), actual. get. Inv()); assert. Equals(exp. Item 2. get. Prod(), actual. get. Prod()); assert. Equals(exp. Item 2. get. Quantity(), actual. get. Quantity()); } At this point let’s look an a set of examples…. 31

Another Code Smell: Test Logic in Production “The production code contains logic that should be exercised only during tests” o o Test Hook For Tests Only Test Dependency in Production Equality Pollution 32

Another Code Smell: Hard to Test Code can be hard to test for a number of reasons: n n Too closely coupled to other software No interface provided to set state and/or to observe state Root Cause is lack of Design for Testability n Comes naturally with Test-Driven Development n Likely to have to be retrofitted to legacy software Temporary Workaround is Test Hook n In other words, build controllability and observability in your production code. n Becomes Test Logic in Production (code smell) if not removed! o But this assumes you will find another way to obtain controllability and observability…. 33

An example of a pattern to address code smells: Test Double Pattern Look at the 4 inside-cover pages: a pattern may be used for several smells! 34

Testability Patterns 35

Behavior Smells o A problem seen when running tests. o Tests fail when they should pass n or pass when they should fail (rarer) o The problem is with how tests are coded n not a problem in the SUT o Sniff Test: n Detectable via compile or execution behavior of tests 36

General Behavior Smells o o o Assertion Roulette Erratic Test Fragile Test Frequent debugging Manual Intervention Slow Tests 37

Assertion Roulette o Symptom: n One or more unit tests are failing in the automated build and you cannot tell why without rerunning the tests in your IDE. When you cannot reproduce the problem in your IDE you have no idea what is going wrong. o Impact: n It takes longer to determine what is wrong with the code. n Bugs that cannot be reproduced cannot be fixed. o Root Cause: n Missing/Unclear Assertion Messages 38

Erratic Test o Interacting Tests n When one test fails, a bunch of other tests fail for no apparent reason because they depend on other tests’ side effects o Unrepeatable Tests n Tests can’t be run repeatedly without intervention o Test Run War n Seemingly random, transient test failures n Only occurs when several people testing simultaneously (!!) o Resource Optimism n Tests depend on something in the environment that isn’t available o Non-Deterministic Tests n Tests depend on non-deterministic inputs 39

Fragile Tests Causes: o Interface Sensitivity n Every time you change the SUT, tests won’t compile or will start failing n You need to modify lots of tests to get things “Green”again n Greatly increases the cost of maintaining the system o Behavior Sensitivity n Behavior of the SUT changes but it should not affect test outcome n Caused by being dependent on too much of the SUT’s behavior. 40

Fragile Tests Causes: o Data Sensitivity n Alias: Fragile Fixture n Tests start failing when a shared fixture is modified o e. g. New records are put into the database o Context Sensitivity n Something outside the SUT changes o e. g. System time/date, contents of another application 41

Frequent Debugging o Symptom: n One or more tests are failing and you cannot tell why without resorting to the debugger. This seems to be happening a lot lately! o Impact: n Debugging is a very time-intensive activity. n While it may help you find the bug, it won’t keep it from coming back. o Root Causes: n Missing Unit Tests n Poor Assertion Messages 42

Manual Intervention o Symptom: n A test requires a person to perform some manual action each time it is run o Impact: n May result in frequent debugging n High test maintenance cost n Makes it impractical to have a fully automated Integration build and regression test process o Causes: n Manual Fixture Setup n Manual Result Verification n Manual Event Injection 43

Slow Tests “It takes several minutes to hours to run all the tests” o Impact n Lost productivity caused by waiting for tests n Lost quality due to running tests less frequently o Causes n Slow Component Usage e. g. Database n Asynchronous Test (e. g. Delays or Waits) n General Fixture e. g. too much fixture being setup 44

Project Smells o A Test Smell that a project manager is likely to observe o Symptoms are typically developer behavior or feedback from other organizations o There may be metrics that point out the smell n e. g. Number of bugs found in Acceptance Test o Root cause is often Code or Behavior Smells o Cannot be addressed directly n Solution is to address underlying smells 45

General Project Smells o o Buggy Tests Developers Not Writing Tests High Test Maintenance Cost Production Bugs 46

Buggy Tests o Symptoms: n Tests are failing when they shouldn’t (the SUT works fine) o Impact: n No one trusts the tests any more o Possible Causes: n Erratic Tests n Fragile Tests n Untested Test Code 47

Developer’s Not Writing Tests o Symptoms: n No tests can be found when you ask to see the o unit tests for a task, o customer tests for a User Story, n Lack of clarity about what a user story or task really means o Impact: n Lack of safety net n Lack of focus o Possible Causes: n Hard to Test Code? n Not enough time? n Don’t have the skills? n Have been told not to? n Don’t see the value? 48

High Maintenance Costs o Symptoms: n A lot of effort is going into maintaining the tests o Impact: n Cost of building functionality is increasing n People are agitating to abandon the automated test o Possible Causes: n Erratic Test n Fragile Test n Buggy Test n Obscure Test n Hard to Test Code 49

Production Bugs o Symptoms: n Bugs are being found in production o Impact: n Expensive trouble-shooting n Development team’s reputation is in jeopardy o Possible Causes: n Lost/Missing Tests n Slow Tests n Untested Code n Hard-to-Test Code n Developers Not Writing Tests 50

Effective test automation After test generation by considering all paths and the features and organization specified, our test still may have these bad smells: o Slow Tests o Test Code Duplication o Obscure Tests o Buggy Tests so we need Refactoring. 51