TESTING FOR THE RELIABILITY OF A SOFTWARE SARAT

TESTING FOR THE RELIABILITY OF A SOFTWARE SARAT CHANDRA YADAVALLI CSC 532 TERM PAPER

Software Reliability Engineering Introduction n n Reliability for software is the probability of failure free operation of a computer programme in a specified environment for some specified interval of natural units or time. Software reliability engineering is the standard that empowers the testers and developers to simultaneously Ø Ensure that product reliability meets user needs Ø Speed the product to market faster Ø Reduce product cost

Software Reliability Engineering Process Diagram 1. List Associated Systems 2. Develop Operational Profiles 3. Engineer “Just Right” Reliability 4. Prepare for Test 5. Execute Test 6. Guide Test Requirements and Architecture Design and Implementation Test

Developing Operation Profiles n n Developing operational profiles will give the information about how users will employ the product being built With the information given, both development and test can be made more realistic

Terminology n Operation is a major system logical task, n Operational Profile (OP) is simply the set n OP can be represented in 2 ways: of short duration which returns control to the system when complete of operations and their probabilities of occurrence. Ø Ø Tabular representation Graphical representation

ØTabular Representation

ØGraphical Representation

Methodology n n n Identify the initiators of operations Choose between tabular or graphical representation Create an operations list for each initiator and consolidate the results Determine the occurrence rates of the individual operations Determine the occurrence probabilities

Preparing for Test n n n The operational profile information is applied to planning for efficient test. Testing is the process of executing a program with the intention of finding design errors in a given environment. Testing can only prove the incorrectness of the software but not its correctness.

Concepts n n n Run is a specified instance of an operation. A run is characterized by the operation and its input state Input variable can be either direct or indirect A test case is the partial specification of a run through the naming of its direct input variables and their values. A run involves the execution of a test case. Test procedure is a controller that sets up environmental conditions and that invokes randomly selected test cases at random times.

Direct and Indirect Input Variables for Process Fax Call Operation of Fone Follower

Procedure n Preparing test cases – It can be done by simply recording in the field all the input variables needed to initiate the runs that make up the field execution of the software n Preparing test procedures

Preparing test cases n n n Estimating the number of new test cases needed for the current release Allocating the number of new test cases among the systems to be tested Allocating the number of new test cases for each system among its new operations Specify the new test cases Adding the new test cases to the test cases from previous releases.

Estimating the number of new test cases needed for the current release n Two factors must be taken into account: Ø Ø n n n Time Cost Compute the number of test cases you have time to prepare Compute the number of test cases you can afford to prepare Take the minimum of these as the number of test cases you will plan to prepare

Allocating the number of new test cases for each system among its new operations n n n Convert the graphical representation into tabular Identify the rarely occurring critical new operations and determine how many test cases to preassign to each Determine the allocation probabilities for the other new operations Preassign one test case to each infrequent other new operations Assign the remaining test cases to the remaining other new operations in accordance with the allocation probabilities

Allocating the number of new test cases among the systems to be tested n n n Bulk of the test cases should be allocated to the product itself Majority of test cases are allocated to the number of operations of the variation that occur and do not occur for the product Give particular weight to the differing operations that have high occurrence probabilities

Allocation of Test cases to Operations of Fone Follower

Test cases Specification n Select test cases within operations with equal probability Prepare test scripts for the selected test cases When invoked, the test cases must convey the operation they belong to and their direct input variables to the system under test

Conclusion “A laudable standard for software reliability is Five 9’s - software which works 99. 999% of the time” We have discussed n The procedure to identify the operational profiles used in defining the test cases. n Procedure to prepare for test

References n n Text Book : Software Reliability Engineering – John D. Musa http: //members. aol. com/John. DMusa/ARTweb. htm Lyu, M. (Editor). 1996. Handbook of Software Reliability Engineering , ISBN 0 -07 -039400 -8, Mc. Graw-Hill, New York. Tierney, J. 1997. SRE at Microsoft. Keynote speech at 8 th International Symposium On Software Reliability Engineering, November 1977, Albuquerque, NM. n n n Challenges in Software Reliability and Testing - Philip J. Boland, Department of Statistics, National University of Ireland – Dublin. Y. K. Malaiya and J. Denton “ Module Size Distribution and Defect Density, ” Proc. IEEE International Symposium on Software Reliability Engineering, Oct. 2000, pp. 62 -71 Operational Profile Specification, Test Case Generation, and Reliability Estimation for Modules - D. M. Woit, Queen’s University, February 1994. N. Li and Y. K. Malaiya “ROBUST: A Next Generation Software Reliability Engineering Tool” Proc. IEEE Int. Symp. on Software Reliability Engineering, pp. 375 -380, Oct. 1995. Predicting Software Reliability from Testing Taking into Account Other Knowledge about a Program - Antonia Bertolino (IStituto di Elaborazione della Informazione del CNR, Pisa, Italy), Lorenzo Strigini (Centre for Software Reliability, City University, London).