PS 3 Acceptance Test AN EXAMPLE Based on

PS 3: Acceptance Test AN EXAMPLE Based on Testing Extreme Programming By Lisa Crispin, Tip House

Goal 2 Check the functionality of the entire system as specified by the project's customer. Acceptance tests should not include specific knowledge about the internals of the system and are said to be "black box" tests. These tests touch the system only at predefined APIs or GUIs. To be effective, system-level tests often require large amounts of domain-specific data. The customer's involvement in creating this data and specifying system behavior is critical. Software Engineering Spring 2011

Example Application: Seller Reputation 3 Software Engineering Spring 2011

Rate a transaction 4 Select a seller to rate from a list of sellers Select a transaction to rate Rate the transaction Accurate description Communication Quick shipping Reasonable shipping charges Software Engineering Spring 2011

Major phases 5 High level Acceptance Tests Write stories Define some acceptance tests for each story Find Hidden Assumptions in the stories Customer’s view; User’s view; Programmer’s view Add High level Acceptance Tests based on these assumptions Obtain Data for testing Bad, sad, happy scenarios Write executable tests Write code for each acceptance test defined Bridge the code to the application Software Engineering Spring 2011

High Level Acceptance Test 6 Story I: Rate Transaction Story The user can provide a rating for a transaction he took part in, expressing the degree to which he is satisfied with it in the following properties: Accurate description, Communication, Quick shipping, Reasonable shipping charges, and textual comment. Define high-level acceptance tests for the Story. Action Data Rate the seller by Accurate description, Communication, Quick shipping, Reasonable shipping charges, and add a textual comment. An unrated Success: the rating transaction that for the transaction exists in the is updated database Software Engineering Spring 2011 Expected Results

Obtaining hidden Assumptions 7 User’s view Story Programmer’s view Customer’s view Software Engineering Spring 2011

Step 1 – Customer view 8 I'm the customer. We need a way to track performance of sellers in our system in order to compute the reputation of this seller. For this purpose we need the users that interacted with this seller to rate the seller by 4 different aspects. We don’t want users that did not have experience with the seller to rate it – so only user who interacted with the seller may provide ratings for him. We don’t want the user to provide multiple good ratings or bad ratings – so we allow only one rating per transaction. What business problem is it solving? How does it solve it? Users would like to know a seller reputation before initiating a transaction with him. By collecting users rating we use some Trust/Reputation algorithm to infer the seller’s reputation. Software Engineering Spring 2011

Step 2: User view 9 I'm a user, I want to provide a rating for a transaction I made What are the worst and best things that can happen? What would really irritate me? Worst thing: I can't locate the transaction I want to rate. It's in the system, but I can't find it. Two rating properties appear to have a similar meaning —I can't tell the difference. Best thing: I can see all the transactions I have performed in the system sorted by date, seller rating –state (rated/unrated). I can see the rating I gave for the rated ones. I can also select an unrated transaction form my list and rate it. How can I screw up? How should the system respond? Can I accidentally rate the same transaction more than once? Can I rate a seller that I didn’t interact with? Can I change my opinion – change the rating I already gave a seller upon completion of a transaction? Software Engineering Spring 2011

Step 3: Programmer view 10 I'm the programmer. What's the simplest implementation that could possibly work? A list of all transactions each indicated as rated or not. Click on an unrated transaction to select it and to rate it. Click on a rated transaction impossible. OR? We need some way to identify a transaction Do we? by a number? Software Engineering Spring 2011

Step 4: Tester’s view 11 How likely is the implementation to solve the business problem? Fairly likely. ( under honest and cooperating users assumptions) How likely is the implementation to solve the related problems? Fairly likely. How likely is the implementation to avoid irritating the user? We need a way to indicate the user that no more than one rating is allowed per transaction There is no option to modify a rating already provided The value they can provide is within a predefined range Software Engineering Spring 2011

Hidden assumptions 12 Identify problematic hidden assumptions in the story The system provides a list of the user’s transactions The user can view rated transactions. The user cannot edit rated transaction. All four properties are required fields. The rating values range within a known scale. Textual comments are optional ? Software Engineering Spring 2011

Additional tests (based on the assumptions) 13 Action Data Expected Results 1. 1. Rate the seller by the four properties. An unrated transaction that exists Success: the rating for the in the database transaction is updated 1. 2. Rate the seller by the four properties An unrated transaction, exceed maximum values for the four properties. Failure: should not be able to enter numbrs out of range 2. 1. find the List of unrated transactions the user had with a seller. A user , a seller, existing transactions of the two with “closed” field =No Success: The list of transactions the user had with the seller is not empty 2. 2. find the List of all transactions the user had with a seller. A user , a seller, no transactions of the two with “closed” field =No Fail: The list of transactions the user had with the seller is empty … … What other acceptance test can be added according to the assumptions? Software Engineering Spring 2011

Writing Executable Acceptance tests 14 The rate. Trans. Story. High-level acceptance tests for this story: Rating an unrated transactions using correct values: succeeds – the transaction properties are updated, transaction is closed Rating an unrated transactions using out-of –range values: fails– the transaction properties are not updated, transaction remains opened. Data: Users : Bob, Clair Sellers: Auto. Shop, Buy 4 Cheap Transactions: Bob, Auto. Shop - unrated transaction 1223, Clair, Buy 4 Cheap– rated transaction 1226 This captures the essence of the test but leaves out the details. The details can go into an executable test, as follows: assert. True(rate. Trans(1223, 1, 2, 3, 4, ”bad experience”) ) assert. True(rate. Trans(1223, 1, 2, 3, 4, ””) ) assert. False(rate. Trans(1223, 7, 2, 3, 4, ”Called right on time”) ) assert. False(rate. Trans(1226, 3, 2, 3, 4, ”unreliable”) ) assert. False(rate. Trans (1227, 1, 2, 3, 4, ”blabla”) - Can blabla cause the test failure? Software Engineering Spring 2011

Sampled Test Data Table 16. 3 15 Defined with the customer Happy, sad, bad… Characterist ics Opened Trans ID Accurate Communic ations Shipping Costs Comment Exxpected Results Valid Y 1223 1 2 2 3 bad experience Transaction successfully updated Valid Y 1223 1 2 5 4 Invalid Y 1223 0 1 2 1 THE WORST!! Rating score is out of range Invalid Y 1223 9 2 4 1 Nice but not proffesional Rating score is out of range Invalid N 1226 1 2 5 3 Very polite Transaction is already rated 1227 1 2 2 3 Bla bla Invalid Transaction ID Invalid Software Engineering Spring 2011 Transaction successfully updated

Testing Method per Action 16 For each action of the high level acceptance test– write a testing method: public class rate. Trans. Story. Test { public void test. Rate. Trans() { // Test data and assumptions for action Rate go here } public void test. List. Trans() { // Test data and assumptions for action List go here } } Software Engineering Spring 2011

Testing Method per Action 17 public class rate. Trans. Story. Test extends Seller. Reputation. Test { public rate. Trans. Story. Test (String name) {super(name); } \ A pass-thru constructor public void test. Rate. Trans() { assert. True(rate. Trans(“ 1223”, 1, 2, 3, 4, ”blabla”) ) ; assert. True(rate. Trans(“ 1223”, 1, 2, 3, 4, ””) ) ; assert. False(rate. Trans(“ 1223”, 7, 2, 3, 4, ”blabla”) ) ; assert. False(rate. Trans(“ 1226”, 3, 2, 3, 4, ”blabla”) ) ; assert. False(rate. Trans (“ 1227”, 1, 2, 3, 4, ”blabla”); } public void test. List. Trans() { rate. Trans(<T_ID>, … <score 1>, <score 2>, <score 3>, } } Software Engineering Spring 2011 <score 4><comment>) should be bridged to the real application

Bridge by inheritance 18 import junit. framework. *; //Define an application-test class that extends JUnit' s Test. Case import Seller. Reputation. *; public class Seller. Reputation. Test extends Test. Case {// bridge class public Seller. Reputation. Test(String name) {super(name); } public void assert. False( boolean condition) { assert. True(!condition); } // Make every story acceptance test a sub-class of the application test class. Public boolean rate. Trans(String t. Id, int att 1, int att 2, int att 3, int att 4, string comment){ Seller. Reputation. Trans trans= new Seller. Reputation. Trans(); return trans. rate(t. Id, att 1, att 2, att 3, att 4, comment); } public boolean list. Trans( String u. Id, String s. Id) { A Bridge to the rate ……. operation as implemented } in the application } Software Engineering Spring 2011

Bridge by inheritance 19 The Seller. Reputation. Trans class in the Seller. Reputation system represents a transaction. The Seller. Reputation. Trans has a rate method that sets the transaction rating criteria: Accurate description, Communication, Quick shipping, Reasonable shipping charges and comment. If the specified values are all valid, it then attempts to update the transaction, if the update is successful it returns true otherwise - if the update fails or if any of the specified values is invalid, it returns false The Seller. Reputation. User class in the Seller. Reputation system represents a user. The Seller. Reputation. User has a trans. List method that searches for a list of transactions the user had with a specified seller If the user had interactions with the specified seller, it produces the list of transactions, if the list is nonempty it returns true otherwise - if the list is empty or if seller id is invalid, it returns false Software Engineering Spring 2011

Bridge by inheritance 20 Test. Case (junit ) • assert. True • assert. Equal • … Seller. Reputation. Test • assert. False • rate. Trans • list. Trans • … rate. Trans. Story. Test • test. Rate. Trans • test. List. Trans • … Software Engineering Spring 2011 Functionality provided by testing framework, e. g. junit Project Test Defines the bridge to the application Actual tests

Bridge by Composition 21 Test. Case (junit ) • set. Up() • assert. True • assert. Equal • … Seller. Reputation. Test • set. Up() • assert. False • rate. Trans {. . bridge. . } • list. Trans {. . bridge. . } • … rate. Trans. Story. Test • test. Rate. Trans {. . rate. Trans. . } • test. List. Trans{. . list. Trans. . } • … Software Engineering Spring 2011 <<Interface >> bridge Seller. Reputation. Bridge • rate. T() • list. T() • ……. real implements Proxy Bridge Real Bridge • rate. T() • list. T() • ……. .

Bridge by Composition 22 How to avoid recompile the tests when replacing bridges ? • Seller. Reputation. Test class receives the bridge as parameter (setter and/or in constructor). • Seller. Reputation. Test class reads configuration file to select the right bridge (in set. Up()). • Seller. Reputation. Test class uses external class to receive the bridge (next slide). • Any other methods you can come up with. Software Engineering Spring 2011

Bridge by Composition 23 Test. Case (junit ) • set. Up() • assert. True • assert. Equal • … Seller. Reputation. Test • set. Up() • assert. False • rate. Trans {. . bridge. . } • list. Trans {. . bridge. . } • … rate. Trans. Story. Test • test. Rate. Trans {. . rate. Trans. . } • test. List. Trans{. . list. Trans. . } • … Software Engineering Spring 2011 <<Interface >> Seller. Reputation. Bridge protected void set. Up() throws Exception { bridge super. set. Up(); • rate. T() … • list. T() realthis. bridge=Driver. get. Bridge(); • ……. … } implements Proxy Bridge Real Bridge public static Seller. Reputation. Bridge get. Bridge() { Proxy. Bridge bridge=new Proxy. Bridge (); // add when real bridge is ready • rate. T() bridge. real = new Real. Bridge (); • list. T() return bridge; • ……. . } <<use>> Driver get. Bridge()