CSE Senior Design II Test Planning Manfred Huber

CSE Senior Design II Test Planning Manfred Huber Based on an earlier presentation by Mike O’Dell, UTA

1 Test Planning … when? Ø Occurs throughout the product life cycle § § Requirement completeness Architectural integrity Component design precision Implementation correctness Ø But all too often we wait until the end to “validate” what we have already done. CSE 4317 2

1 System Verification System Definition SRD: System Requirements MAP: All Specified Requirements System Validation Test ADS: Architecture Specification MAP: All Subsystem & Layer Interfaces & Interactions Integration Test DDS: Detailed Design Specification MAP: All Module Interfaces & Interactions Component Test (a. k. a. Function Test) MAP: All Modules Implementation Unit Test System Verification CSE 4317 3

1 System Verification Ø Unit Test: verifies that EVERY module (Hardware & Software) specified in the DDS operates as specified. Ø Component/Function Test: verifies integrity of ALL inter-module interfaces and interactions. Ø Integration Test: verifies integrity of ALL intersubsystem/layer interfaces and interactions. Ø System Verification Test: verifies ALL requirements are met. CSE 4317 4

1 System Test Plan Contents Ø Introduction Ø Executive Summary (purpose, scope, etc. ) Ø References § SRD (requirements drive the system-level tests you do) § ADS (architecture drives the subsystem and integration testing) § Etc. (detailed design, standards, sample plans, etc. that are test drivers) Ø Test Item(s) – What will be tested CSE 4317 5

1 System Test Plan Contents Ø Design decomposition (relationship of specific modules to test plan, test cases) Ø Limitations of product under test (restrictions, assumptions, caveats, etc. ) Ø Other product-level restraints on testing Ø Risks (in test phase) § Specific risks that may affect testing/test outcome § Impact assessment and management plan as it relates to these risks CSE 4317 6

1 System Test Plan Contents Ø Features to be tested § List of specific features (from the USERS’ viewpoint) that you will test Ø Features NOT to be tested § List of those features (USERS’ perspective) that cannot/will not be tested § Describe rationale for not testing each item (not in current release, insufficient test capability, low risk feature that is hard to test, not a user-visible feature) CSE 4317 7

1 System Test Plan Contents Ø Approach (Strategy) § Overall test strategy (what, why, and how of your test plan) § Number of hardware and software configurations tested/not tested § Plan to deal with defects identified (re-integration, regression) § Identify metrics for overall success (number of bugs found/corrected/open, number of critical features passed, etc. ) § Identify any special requirements for testing CSE 4317 8

1 System Test Plan Contents Ø Item Pass/Fail criteria § List specific criteria for each unit/module/feature to be tested (number and severity of defects, any specific tests that indicate system failure, etc. ) Ø Test Deliverables § List what is to be delivered by this test plan (plan document, test cases, error logs, etc. ) § Incident/Bug/Problem Reports (IRs) Ø Test Schedule (should be consistent with your published project plan) CSE 4317 9

1 System Test Plan - Resources Ø IEEE Std 829 -1998, IEEE Standard for Software Test Documentation Ø Test Plan Outline for IEEE Format (from Gerrard Consulting) CSE 4317 10