Component Testing Ednaldo Dilorenzo de Souza Filho ednaldo

Component Testing Ednaldo Dilorenzo de Souza Filho ednaldo. filho@cesar. org. br 10/24/2021 1

Summary n n n n n Introduction Fundaments of Testing Software Reuse Component Testing Motivation Test-Driven Development Third-Party Testing Regression Testing based on Version Changes of Components Testability Analysis for Software Components White on Black 10/24/2021 2

Introduction n n Testing is an important process to support assurance; As software becomes more pervasive and is used more often to perform critical tasks, it will be required to be of higher quality; Because testing requires the execution of the software, it is often called dynamic analysis [Harrold]; Testing consists in compare the outputs from the executions with expected results to identify those test cases on which the software failed; Testing cannot show the absence of faults, if can show only their presence. 10/24/2021 3

Fundaments of Testing n Testing goals: ¡ ¡ ¡ n Find faults in a software program; A good test case have a high probability to find errors; A good test process should find errors; Testing fundamentals: ¡ ¡ ¡ Testing should be connected with client requirements; Testing should be planned before testing execution; Isolation of suspected components; Testing should start in individual components; Complete testing is impossible; Testing shouldn’t be executed for developers; 10/24/2021 4

Software Reuse n n n Increased reliability: Reuse is supposed to have a positive effect on the quality of the software entity reused; Reduce Process Risk: Software reuse can reduce risks inherent in software projects; Effective use of specialists: Software reuse can also contribute to an effective use of specialists; Standards compliance: Improve its standards compliance, which in turn can have other benefits; Accelerated development: Software reuse can accelerate development; [Beydeda] 10/24/2021 5

Component Testing Motivation n The Ariane 5 Lesson [Weyuker] ¡ ¡ n n n In June 1996, during the maiden voyage of the Ariane 5 launch vehicle, the launcher veered off course and exploded less than one minute after take-off; The explosion resulted from insufficiently tested software reused from Ariane 4 launcher; Context-Dependent Testing of a Component [Beydeda]; Insufficient Documentation of a Component; Component User’s Dependence on the Component Provider; 10/24/2021 6

Component Testing Motivation n Problems with systems built from reusable components: ¡ ¡ ¡ n n Performance problems; Fitting selected components together; Absence of low-level understanding; Developing a component for a particular project with no expectation of reuse, testing proceed as usual; Significant additional testing should be done for changing priority; Modify or not the source code? Facilitating Reusable Component Testing ¡ ¡ Associate a person or a team for maintaining each component; Add software specification; Add test suite; Add pointers between the specification and parts of the test suite; 10/24/2021 7

Test-Driven Development [Janzen] n n Requires writing automated tests prior to developing functional code; Used in agile methods; To promote testing to an analysis and design step the practice of refactoring must be introduced; TDD in industry ¡ ¡ n 18 percent to 50 percent more external test cases than code produced by corresponding control groups; Less time debugging code; TDD in academia ¡ ¡ Early detection of defects; Improvement in software quality and programmer productivity; 10/24/2021 8

Third-Party Testing and the Quality of Software Components [Councill] n n n 10 contribuitors devoted 70 person-hour to developing a definition of component; Ariane Project; Certification business introduced a Maturity Model focused on large organizations; 99 percent of all US businesses are small businesses; Producers provide the developing and testing procedures, purchasers or their agents conduct second-party testing and independent testing organizations perform third-party testing; 10/24/2021 9

Third-Party Testing and Stirrings of the New Software Engineering [Councill 2] n Licensed software engineers must manage ¡ ¡ n Intense requirements elicitation; Comprehensive requirements specifications that address all states endeavor to assure fail-safe and fault-tolerant operations; Designs that can be tested on paper; The anticipation of third-party testing; Regulatory software component certification ¡ ¡ Software engineer licensing; Mandated standards for third-party testing of software components; 10/24/2021 10

Regression Test Selection Based on Version Changes of Components [Sajeev] n n n Regression test strategy test components tested before to assure new components didn’t affected it; Most regression testing techniques are based on white box strategies by analyzing code; Use of UML (Unified Modeling Language) and its associated with OCL (Object Constraint Language) to formally specify the test selection process; 10/24/2021 11

Regression Test Selection Based on Version Changes of Components [Sajeev] n Regression Test Selection ¡ ¡ Let P be a program, and P’ be a new version of program created by replacing some of the components in P with new versions; Let T be a set of test cases that is already run on P; The aim of regression testing is to show that P’ satisfies T; Base case n ¡ T = T’ Optimal case n T’ T 10/24/2021 12

Regression Test Selection Based on Version Changes of Components [Sajeev] 10/24/2021 13

Regression Test Selection Based on Version Changes of Components [Sajeev] ¡ Test Selection Strategy n n n Step 1: Select all test sequences which include a direct call to a modified method; Step 2: Select all test sequences which include a call to any method which in turn uses, directly or indirectly, a modified method of a revised component; Union of two steps bring the test sequences for regression testing; 10/24/2021 14

Testability Analysis For Software Components [Nguyen] n n n Testability is a quality factor, which can be predicted as soon as the system is specified, to indicate the ease (or difficulty) for testing the system; A component is said to be domain-observable, if distinct outputs are produced from distinct inputs; A component is said to be a domain-controllable, if its output domain is covered from its input domain; The testability of a system is based on controllability and observability of the components in the system; Controllability is defined as the ease to generate the inputs of a component from the inputs of the system; Observability is defined as the ease to propagate the outputs of a component to the final outputs of the system; 10/24/2021 15

Testability Analysis For Software Components [Nguyen] n Testability Model 10/24/2021 16

Testability Analysis For Software Components [Nguyen] n Testability measures ¡ ¡ ¡ Information channel Coefficient of information loss for a module Module capacity Edge capacity Information transfer Testability measures 10/24/2021 17

Testability Analysis For Software Components [Nguyen] n Static Single Assignment Form 10/24/2021 18

Testability Analysis For Software Components [Nguyen] n Case Study 10/24/2021 19

Testability Analysis For Software Components [Nguyen] n Case Study Observability and Controlability values 10/24/2021 20

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] n n n It would be ideal to test a program with its entire input domain. A more practical work is to construct a test suite for testing. A test suite constructed should be as comprehensive as possible, and it should be as small as possible. Approach for reduce the test case based on specification (Black box) using white box information. In the black box approach, the category-partition method (CPM) and the classification-tree method are particularly useful. 10/24/2021 21

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] n Background ¡ ¡ ¡ In partition testing approach input domain of a program is divided into subsets, called subdomains; The tester identifies relevant aspects of the specification for testing called classification; The subdomains associated with the classification are called classes; Test cases are formed and every incompatible classes identified said to be illegitimate; Only legitimate classes are considered reducing the number of test cases in the test suite; 10/24/2021 22

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] n Background 10/24/2021 23

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] n n n Information of the paths executed by the test cases are used for helping test case reduction; When two test cases are identical differing by only one class they are called a matching pair; Classes differing from a matching pair are called differentiating classes, or a differentiating class pair; Matching Pair 10/24/2021 Differentiating Classes 24

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] n Based on the white box analysis we can verify which test cases can be omitted; Here tc 4 or tc 5 can be ommited 10/24/2021 25

White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] 10/24/2021 26

References n n n [Beydeda] Sami Beydeda and Volker Gruhn, Testing Commercial-of -the-Shelf Components and Systems, Springer, 2005. [Councill] William T. Councill, Third-Part Testing and the Quality of Software Components, IEEE Software, Quality Time, July 1999. [Councill 2] William T. Councill, Third-Part Testing and Stirrings of the New Software Engineering, IEEE Software, Quality Time, July 1999. [Chen] T. Y. Chen, P. L. Poon, S. F. Tang and Y. T. Yu, White on Black: A White-Box-Oriented Approach for Selecting Black-Box. Generated Test Cases, IEEE, 2000. [Janzen] David Janzen and Hossein Saiedian, Test-Driven Development: Concepts, Taxonomy, and Future Direction, IEEE Computer Society, Cover Feature, Semtember 2005; [Sajeev] A. S. M. Sajeev and Bugi Wibowo, Regression Test Selection Based on Version Changes of Components, IEEE, Proceedings of the Tenth Asia-Pacific Software Engineering Conference, 2003; 10/24/2021 27

References n n [Nguyen] T. B. Nguyen, M. Delaunay and C. Robach, Testability Analysis For Software Components, IEEE, Proceedings of the International Conference on Software Maintnance, 2002. [Harrold] M. J. Harrold, Testing a Roadmap, College of Computing, Georgia Institute Technology, 2000. 10/24/2021 28