COSC 4406 Software Engineering Haibin Zhu Ph D
COSC 4406 Software Engineering Haibin Zhu, Ph. D. Dept. of Computer Science and mathematics, Nipissing University, 100 College Dr. , North Bay, ON P 1 B 8 L 7, Canada, haibinz@nipissingu. ca, http: //www. nipissingu. ca/faculty/haibinz 1
Lecture 10 Software Testing Strategies 2
Software Testing is the process of exercising a program with the specific intent of finding errors prior to delivery to the end user. Verification: refers to the set of activities that ensure that software correctly implements a specific function. Validation: refers to a different set of activities that ensure that software that has been built is traceable to customer requirements. 3
What Testing Shows errors requirements conformance performance an indication of quality 4
Who Tests the Software? developer Understands the system but, will test "gently" and, is driven by "delivery" independent tester Must learn about the system, but, will attempt to break it and, is driven by quality 5
Testing Strategy unit test system test integration test validation test 6
Testing and SLDC 7
Testing Strategy n n We begin by ‘testing-in-the-small’ and move toward ‘testing-in-the-large’ For conventional software n n n The module (component) is our initial focus Integration of modules follows For OO software n our focus when “testing in the small” changes from an individual module (the conventional view) to an OO class that encompasses attributes and operations and implies communication and collaboration 8
Strategic Issues n n n n Specify product requirements in a quantifiable manner long before testing commences. State testing objectives explicitly. Understand the users of the software and develop a profile for each user category. Develop a testing plan that emphasizes “rapid cycle testing. ” Build “robust” software that is designed to test itself Use effective formal technical reviews as a filter prior to testing Conduct formal technical reviews to assess the test strategy and test cases themselves. Develop a continuous improvement approach for the testing process. 9
Test Steps 10
Unit Testing module to be tested results software engineer test cases 11
Unit Testing module to be tested interface local data structures boundary conditions independent paths error handling paths test cases 12
Selective Testing n Common errors: n n n Misunderstood or incorrect arithmetic precedence Mixed mode operations Incorrect initializations Precision inaccuracy Incorrect symbolic representation of an expression The errors that should be uncovered n n n n Comparison of different data types Incorrect operators or precedence Expectation of equality when precision error makes equality unlikely Incorrect comparison of variables Proper or nonexistent loop termination Failure to exit when divergent iteration is encountered Improperly modified loop variables 13
Unit Test Environment driver interface local data structures Module boundary conditions independent paths error handling paths stub test cases RESULTS 14
Integration Testing Strategies Integration testing is a systematic technique for constructing the software architecture while at the same time conducting tests to uncover errors associated with interfacing. Options: • the “big bang” approach • an incremental construction strategy 15
Top Down Integration A B F top module is tested with stubs G stubs are replaced one at a time, "depth first" C as new modules are integrated, some subset of tests is re-run D E 16
Bottom-Up Integration A B G drivers are replaced one at a time, "depth first" C D F E worker modules are grouped into builds and integrated cluster 17
Sandwich Testing A B F Top modules are tested with stubs G C D E Worker modules are grouped into builds and integrated cluster 18
Regression Testing n n It is the re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects. It is the activity that helps to ensure that changes do not introduce unintended behaviors or additional errors. 19
Smoke Testing n n A common approach for creating “daily builds” for product software Smoke testing steps: n Software components that have been translated into code are integrated into a “build. ” n n A series of tests is designed to expose errors that will keep the build from properly performing its function. n n A build includes all data files, libraries, reusable modules, and engineered components that are required to implement one or more product functions. The intent should be to uncover “show stopper” errors that have the highest likelihood of throwing the software project behind schedule. The build is integrated with other builds and the entire product (in its current form) is smoke tested daily. n The integration approach may be top down or bottom up. 20
Object-Oriented Testing n n begins by evaluating the correctness and consistency of the OOA and OOD models testing strategy changes n n the concept of the ‘unit’ broadens due to encapsulation integration focuses on classes and their execution across a ‘thread’ or in the context of a usage scenario validation uses conventional black box methods test case design draws on conventional methods, but also encompasses special features 21
Broadening the View of “Testing” It can be argued that the review of OO analysis and design models is especially useful because the same semantic constructs (e. g. , classes, attributes, operations, messages) appear at the analysis, design, and code level. Therefore, a problem in the definition of class attributes that is uncovered during analysis will circumvent side effects that might occur if the problem were not discovered until design or code (or even the next iteration of analysis). 22
OOT Strategy n class testing is the equivalent of unit testing n n n operations within the class are tested the state behavior of the class is examined integration applied three different strategies n n n thread-based testing—integrates the set of classes required to respond to one input or event use-based testing—integrates the set of classes required to respond to one use case cluster testing—integrates the set of classes required to demonstrate one collaboration 23
Validation testing n n Focus is on software requirements Test if the product is the right one n n n Configuration review n n The function or performance characteristics conforms to specification. A deviation from specification is uncovered and a deficiency list is created. To ensure that all elements of the software configuration have been properly developed Alpha/Beta testing n n n Focus is on customer usage Alpha test is conducted at the developer’s site. Beta test is conducted at the end-user’s site. 24
System testing n System testing focuses on system integration n n Recovery testing n n verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration Stress testing n n forces the software to fail in a variety of ways and verifies that recovery is properly performed, MTTR is evaluated. Security testing n n It is a series of different test whose primary purpose is to fully exercise the computer-based system. executes a system in a manner that demands resources in abnormal quantity, frequency, or volume Performance Testing n test the run-time performance of software within the context of an integrated system 25
Debugging: A Diagnostic Process 26
Debugging processes 27
Debugging Effort time required to correct the error and conduct regression tests time required to diagnose the symptom and determine the cause 28
Symptoms & Causes symptom and cause may be geographically separated symptom may disappear when another problem is fixed cause may be due to a combination of non-errors cause may be due to a system or compiler error symptom cause may be due to assumptions that everyone believes symptom may be intermittent 29
Consequences of Bugs infectious damage catastrophic extreme serious disturbing mild annoying Bug Type Bug Categories: function-related bugs, system-related bugs, data bugs, coding bugs, design bugs, documentation bugs, standards violations, etc. 30
Debugging Techniques brute force / testing backtracking induction deduction 31
Debugging: Final Thoughts 1. Don't run off half-cocked, symptom you're seeing. think about the 2. Use tools (e. g. , dynamic debugger) to gain more insight. 3. If at an impasse, get help from someone else. 4. Be absolutely sure to conduct regression tests when you do "fix" the bug. 32
Summary n Software Test n n n n To uncover errors that were made inadvertently. Strategic Issues Test Strategies for conventional software Test Strategies for OO software Validation testing System testing Debugging 33
- Slides: 33