TIME BOXED TESTING BCS SIGIST 13 th July

TIME BOXED TESTING BCS SIGIST 13 th July 1998 Graham Thomas - OSI Group 1

Abstract There is great pressure upon developers today to improve productivity and effectiveness. To achieve this there is a move away from the traditional structured methodologies towards more dynamic, iterative and RAD approaches. This is being combined with Object and Component based techniques, and delivered with a new generation of IDE’s, to produce thin client, web based, voice and data products. So how do we test it? ¯ Propose Time Boxed Testing ¯ Outline an approach ¯ Discuss the problems ¯ Give a flavour of the fun ? 2

Agenda m Development Lifecycle m Testing Helix m Testing Stage by Stage m Traceability m Testing Process m How to sell time boxed testing m Lessons Learnt 3

The Project m Telephony based project m 2 types of proprietary hardware m Client Server running Windows NT m Interfaces to legacy systems m Rational Rose case tool (using UML) m Objectory process m Visual Studio m Was considering Java & thin clients m Incremental/Iterative development 4

Incompatible Models? Rqmts. UAT Sys Test Analysis Depth Design Breadth Iterative (RAD) Model Build Int Test Unit Test Classic V Model 5

Difficult to Represent Test Stages unit Component Integration Depth unit Component Integration Breadth System Acceptance 6

Project Lifecycle ¬ ® . . . 7

Testing Lifecycle ¬ ® . . . 8

A dual spiral life cycle ? ¬ ® Rqmts Analysis Design Build B R D A E P C D . . . Plan Design Construct Execute 9

Testing Lifecycle m Unit Testing m Component Integration Testing m System Testing m Acceptance Testing m Year 2000 Millenium Compliance m Testing Processes and Activities m Tools & Traceability m Resources m Planning 1

Scope of Test Phases Conical Model Component Unit Testing Integration System Testing Iterative User Acceptance Testing Release 1

Tuned Test Lifecycle Testing Activities Brought Forward Component Unit Testing Integration Testing Iterative System Testing - Non functional Requirements - Performance Analysis User Acceptance Testing - Scenario Testing - Acceptance Criteria Release 1

Unit Testing m Iterative Time-Boxed Activity m Specific to delivered components m ‘White Box’ testing techniques employed m External Interfaces tested m Use of tools for Static & Dynamic analysis Unit Testing Sub Unit Testing Unit Integration Testing Unit Testing External Interface Testing Static Analysis Dynamic Analysis Quality Assurance 1

Unit Testing Sub Unit Package Unit Sub Unit Sub Unit Testing Unit Testing Package Unit Integration Testing Previously tested & checked-in Stubs, Drivers, Harnesses 1

Component Integration Testing m Iterative Time-Boxed Activity m Testing prioritised by Business m End to End process threads where possible m Contains elements of following test phases m Build regression test packs Component Integration Package Integration Performance Analysis Non Functional Requirements Component Integration Testing Functional Requirements Regression Test Scenario Regression Test 1

Component Integration Testing Step 1 - Package Integration Component Package Integration must precede Component Integration Package Component 1

Component Integration Testing Step 2 - Component Integration Component Component Drop 1 Component Component Component Drop 2 . . . 1

Component Integration Testing Drop by Drop ® ¬ Component Component Component Component Component Component . . . Automated Regression Test Pack 1

System Testing m One off activity m Structured testing techniques m End to End process threads m Ties together the development iterations m Regression test already constructed Non Functional Requirements Regression Test System Testing Performance Analysis Design Verification 1

Acceptance Testing m One off activity m Test that the computer system integrates with business processes m Structured testing techniques m End to End process threads m Regression test already constructed Scenario Regression Testing Acceptance Criteria Testing Business Process Testing Acceptance Testing Data Take On & Conversion Testing Usability Testing 2

Y 2000 - Millennium Compliance 6 Date Scenarios This Century Across Wholly Century next Boundary Century Jan 1 st Feb 29 th Dec 31 st m Separate 2 Business Tests Financial Year ? Fiscal Periods ? test runs m Carried out in sequence m Some tests can be exercised post Y 2000 2

Testing Lifecycle Traceability High Level Function User Requirements Functional Requirement Non FR Use Case Development Repository Scenario Test Case Test Script Test Planning 2

Testing Processes and Activities Process Quality Interface Problem Mgmt Conf. Mgmt. Interface Test Management Activity Entry Test Execution Test Reporting Test Planning Test Automation Monitor Test Design Regression Test Measure Test Preparation Manage Analysis of Change Test Stage Sign off Exit 2

Processes m Essential l Configuration Management l Impact Analysis using a Test Planning repository l Quality Management m Good practice l Fault Reporting - on the Intranet - Live l Gathering metric information 2

Tools m Full set of testing tools is required l Static analysis l Dynamic Analysis l Interactive debugging l Test Automation/Capture Replay essential l Test Planning Repository l Performance monitoring (stress/load) l Fault Reporting system 2

Monitoring 2

Monitoring 2

Monitoring 2

Monitoring 2

Test Resources Unit Testing Component Integration Testing TEST TEAM Developers System Testing Business User Acceptance Testing Business Users Testers Secondment 3

Test Planning Decision Test Approach Definition Test Strategy Testing Process Development Test drop ¬ Test drop ® Non Functional Testing Delivery System Test Acceptance Test Y 2000 Test Time-boxed Testing 3

Selling the Approach m Target the following; l Test Team l Systems Architect l Development team l Standards & Methods l Business Representatives l Project Sponsor m Via l Animated Presentation l Test Strategy document 3

So how easy was this? 3

So What went well? m Systems m The Architect liked it presentation was well received 3

What was difficult? m Initial cool response from other members of the test team m Development l Team were uneasy Dev. Why test anyway, our code is wonderful and our test harnesses will do all the testing? m Business Representatives were unhappy about; l Anything less than 100% testing l Bringing their effort forward in the development lifecycle l The difference between the ‘Use Case’ modeling they had done, and the test planning that they would need to carry out 3

What was difficult? (2) m Standards & Methods Saw nothing wrong with their approach, it was either; • structured in which case their approach worked fine, or • it was incremental, in which case their approach worked fine. • They did not do iterative development, but if they did their approach would still work. This took several meetings to gain approval. m Project l Sponsor Cancelled the project. (Time Boxed Testing was not cited as the reason !!!) 3

So what fun did we have ? 3

What did I learn from this? m Organisations change their development approach more quickly than their testing approach m There is a reluctance to do anything less than 100% testing m Testing is a key and integral part of any development lifecycle m The more you think about it, the more sense Time Boxed Testing makes 3

Contact Information Graham Thomas OSI the management support company e-mail: gthomas@osi. co. uk Phone: 0171 323 3353 Copies of slides including animated gifs at www. osi. co. uk 3