Play Testing Software Testing Play Testing Team Structures

Play Testing Software Testing Play Testing Team Structures Nov 20, Fall 2006 IAT 410 1

Software Testing g Goal: “Prove” that the software works – Matches the functional specification – Matches the performance specification g Real Proof: NP-Hard problem – Simulate all the possible runs – Exponential growth in testing time – Each branch point multiplies possible program state by 2 Nov 20, Fall 2006 IAT 410 2

Testing Method g Run the program using a suite of data – Data suite exercises as much of the program’s code as possible – Each data set is accompanied by expected results – Where the results are unexpected, there is a bug! Nov 20, Fall 2006 IAT 410 3

Generating Test Suites g Black Box Testing – Generate test data based purely on interpretation of the specifications – Don’t look at the code! – Code is a Black Box Nov 20, Fall 2006 IAT 410 4

Generating Test Suites g White Box Testing – Use the code to generate tests with the purpose of exercising all blocks of code g Central concept: Coverage g Cover all of the parts of the programs code – Ensure that all lines of code run at least once Nov 20, Fall 2006 IAT 410 5

Combine Both Approaches g Black Box + White Box test suites – Complementary roles – Hard to tell from the data what approach was used – Aims to generate that ever-elusive “proof” g Test suites: – Great for testing Compilers – Perhaps not so great for interaction and games Nov 20, Fall 2006 IAT 410 6

Random Testing g Some applications are hard to test: g Random test – Give up the idea of formal coverage – Hope for stochastic coverage! g Sometimes interaction this depends on actual human – Need to get lots of people – Beta testing by customers Nov 20, Fall 2006 IAT 410 7

Play Testing g Play testing follows software testing – Overlap of phases g Software Testing + More – Tuning game play – Tuning that Flow experience! g Discover Intellectual Space – Does the player Get it? ? – Less important in established genres Nov 20, Fall 2006 IAT 410 8

Team Structure g Fundamental g Don’t issue: Willful blindness fall in love with your code! g Having testing team members be different from your coders is vital g A Software Testing team g A Play Testing Team Nov 20, Fall 2006 IAT 410 9

Play Testing Team g Internal team – Manages play-testing people – Does own play tests – Recruits outsiders for a week or two g External team – – – A population sample Varying skills Unfamiliar with product Not in love with your product! Not yet bored with the product Nov 20, Fall 2006 IAT 410 10

External Testers g Do they like the game? – They may lie to you – Politeness effect • If they can’t say why they like it, you have a problem g Do they get frustrated? – What common areas? – Often skill-based • Internal team judges player’s skill, also Nov 20, Fall 2006 IAT 410 11

External Testers g FAQ – Be prepared to deal with the frequent questions in the final game Nov 20, Fall 2006 IAT 410 12

External Testers: Half-Life g People near Valve’s offices who sent in registration cards g Designers sit quietly while player struggles – Designer takes notes g Typical 2 -hour session – 100 action items g First 20 -30 sessions absolutely vital – Learn what was fun – 200 sessions total Nov 20, Fall 2006 IAT 410 13

Half-Life: Fine tuning g Add instrumentation – Player position, time, health, weapons – Activities: • Game save, dying, being hurt, solving a puzzle, fighting a monster… g Graph series of sessions together – Spot too long with no encounters – Spot long periods of too much health – Spot long periods of too little health Nov 20, Fall 2006 IAT 410 14

Half-Life g Most important playtesting outcome: g Clearly identified good and bad ideas g Allowed people to… Nov 20, Fall 2006 IAT 410 15

Half-Life g Most important playtesting outcome: g Clearly identified good and bad ideas g Allowed people to Abandon Bad Ideas g Playtesting Nov 20, Fall 2006 provides the evidence needed IAT 410 16

Advice g Don’t get defensive! – The tester opinion is important g Testers: Stick to your findings! – Respectfully point out problems g Mix up the hardware – Be honest about the specs on the box Nov 20, Fall 2006 IAT 410 17

More Advice g Discourage Legacies – Don’t let bad decisions live forever! – MS-DOS! Nov 20, Fall 2006 IAT 410 18