Subjects Software Testing Methodologies UNIT1 Introduction And Flowgraphs

Subjects: Software Testing Methodologies UNIT-1 Introduction And Flowgraphs And Path Testing

The nightmare list and when to stop testing 1. List your worst software nightmares. State them in terms of the symptoms they produce and see how your user will react to those symptoms. 2. Convert the consequences of each nightmare into a cost. Usually this is a labor cost for correcting the nightmare. 3. Order the list from the costliest to the cheapest and then discard the low-concern nightmares

4. Based on your experience, measured data and statistics postulate the bugs that are likely to create the symptoms expressed by each nightmare. 5. For each nightmare you’ve developed a list of possible causative bugs, order that list by decreasing probability. 6. Rank the bug types in order of decreasing importance to you.

7. Design tests(based on your knowledge of test techniques) and design your quality assurance inspection process using the methods that are most effective against the most important bugs 8. If a test is passed, then some nightmares or parts of them go away. If test is failed, then nightmare is possible, but upon correcting bug, it too goes away 9. Stop testing when probability of all nightmares has been shown to be inconsequential

3. A Taxonomy of bugs 3. 1 General 3. 2 Requirements, Features and Functionality Bugs 3. 2. 1 Requirements and Specifications 1. Requirements and specifications developed can be incomplete, ambiguous or self-contradictory 2. They may be misunderstood or impossible to understand 3. The specification may assume, but not mention, other specification and prerequisite that are known to the specifier but not to the designer

4. Requirements are major source of expensive bugs 5. The range is from a few percent to 50% depending on application and environment 3. 2. 2 Feature Bugs 1. Specification Problems usually create corresponding feature problems 2. A feature can be wrong, missing, or superfluous. 3. A missing feature or case is the easiest to detect and correct

4. A wrong feature could have deep design implications 5. Removing features might complicate the software, consume more resources, and foster more bugs. 3. 2. 3 Feature Interaction 1. Providing clear, correct, implementable and testable feature specifications is not enough

2. Features usually come in groups of related features. 3. The features of each group and the interaction of features within group are usually well tested. 4. The problem is unpredictable interactions between feature groups or even between individual features. 5. E. g your phone is provided with call holding and call forwarding. Call holding allows you to put a new incoming call on hold while you continue with first caller

Call forwarding allows you to redirect incoming calls to some other phone no. Here are some simple feature interacting questions: a)How about holding a third call when there is already a call on hold? b)Forwarding forwarded calls? And so on Every application has its peculiar set of features and a much bigger set of unspecified feature interaction potentials and therefore feature interaction bugs.

3. 2. 4 Specification and feature bug remedies 1. Most feature bugs are rooted in human-to -human communication problems. 2. One solution is the use of high-level, formal specification languages or systems. Short term support: Specification languages facilitate formalization of requirements and inconsistency & ambiguity analysis. With formal specifications, partially to fully automatic test case generation is possible

Long term support: Assume we have a great specification language and that it can be used to create unambiguous, complete specification, complete test and test criteria. The long-term impact of formal specification languages and systems will probably be that they will influence the design of ordinary programming languages so that more of current specification can be formalized. This approach will reduce, but not eliminate, specification bugs.

3. 2. 5 Testing techniques Techniques which are based on a behavioural description of software such as transaction flow(4), syntax testing(9), domain testing(6), logic testing(10) and state testing(11) are useful in testing functional bugs. 3. 3 Structural Bugs 3. 3. 1 Control and Sequence Bugs 1. Control and Sequence Bugs include paths left out, unreachable code, improper nesting of loops, loop back or loop termination criteria incorrect CONTD. .

Missing process steps, duplicated processing, unnecessary processing, rampaging GOTO’s etc. , Most control-flow bugs are easily tested and caught in unit testing. Control and sequence bugs at all levels are caught by structural tessting, more specifically path testing(3), combined with a bottom-line functional test based on a specification

These bugs are partially prevented by language choice(e. g languagues that restrict control flow options) style, most important lots of memory 3. 3. 2 Logic Bugs 1. Bugs in logic, especially those related to misunderstanding how case statements and logic operators behave singly and in combinations, include inconsistent cases, improper layout of cases, improper negation of boolean expression CONTD. .

For e. g. , using ‘greater than’ as the negation of ‘less than’, confusing exclusive OR with inlusive OR. 3. 3. 2 Processing Bugs 1. Processing bugs include arithmetic bugs, algebraic, mathematical function evaluation, algorithm selection and general processing. 2. E. G. , improper use of greater-than, greater than or equal, less –than, less-than or equal, assumption of equality to zero in floating point, improper comparison between different formats as in ASCII to binary or integer to floating point

3. 3. 4 Initialization Bugs 1. These bugs are less harmful but can affect performance. 2. Typical bugs are as follows: forgetting to initialize work space, registers, a bug in first value of a loopcontrol parameter; accepting an initial value without a validation check; and initializing to the wrong format, data representation or type. 3. 3. 5 Data flow Bugs and Anomalies 1. A data flow anomaly occurs when there is a path along which we expect to do Something unreasonable with data

E. G: Using an uninitialized variable, attempting to use a variable before it exists, modifying data and then not storing or using the result, or initializing twice 3. 4 Data Bugs 3. 4. 1. General Data bugs include all bugs that arise from the specification of data objects, their formats, no of such objects and their initial values

3. 4. 2 Dynamic vs static 1. Dynamic data are transitory, what ever their purpose, they have a relatively short lifetime, typically the processing time of one transaction. E. G. , Object used to store data of diff types 2. Static data are fixed in form and content, what ever their purpose, they appear in the source code or database, directly or indirectly E. G. , a number, a string of characters

3. 4. 3. Information, parameter and control 1. Static or dynamic data can serve in one of three roles 2. What constitutes control or inf is a matter of perspective can shift from one processing level to other. 3. 4. 4 Contents, Structure and Attributes Contents: The actual bit pattern, character string, or number put into a data structure. Structure: The size and shape and nos that describe the data object, i. , memory locations used to store the content

Attributes: Specifications of meaning & semantics associated with content e. g. , integer 3. 5 Coding Bugs. 1. Given good source-syntax checking, the most common pure coding errors are typographical Followed by errors caused by not understanding the operation of an instruction or statement. 3. 6 Interface, Integration and System Bugs 3. 6. 1 The external interfaces are the means used to communicate with the world.

E. g. , invalid timing or sequence assumptions related to external signals, misunderstanding, external input and output formats etc. , 3. 6. 2 Internal interfaces 1. Internal Interfaces are more controlled. 2. E. G. , input and output format bugs, inadequate protection against corrupted data, wrong subroutine call sequence. 3. 6. 3 Hardware Architecture 1. Software bugs related to hardware architecture originate mostly from misunderstanding how the hardware works.

E. G. , paging mechanism ignored, address generation error, I/O device operation or instruction error. 3. 6. 4 Operating System 1. Program bugs related to the operating system are a combination of hardware architecture and interface bugs, caused by a misunderstanding of what OS Does. 3. 6. 5 Software Architecture 1. Software architecture bugs are often the kind that are called interactive

E. G. , failure to block or unblock interrupt, assume that code is reentrant or not reentrant, assumption that called routine is resident or not resident, assumption that calling program is resident or not resident and so on. 3. 6. 6 Control and sequence bugs: refer 3. 3. 1 3. 6. 7 Resource Management Problems: 1. Memory is subdivided into dynamically allocated resources such as buffer blocks, queue blocks, task control blocks. 2. Similarly external mass storage units such as discs are subdivided into memory resource pools.

E. G. , Required resource not obtained, wrong resource used, resource already in use, race condition in getting a resource and so on 3. 6. 8 Integration Bugs 1. Integration bugs are having to do with the integration of, and with the interfaces between, presumably working and tested components. 2. Most of these bugs result from inconsistencies or incompatibilities between components. 3. All methods used to transfer data directly or indirectly between components and all methods by which components share data can host integration bugs and are therefore proper targets for integration testing

3. 6. 9 System Bugs 1. System Bugs is a catch-all phrase covering all kinds of bugs that cannot be ascribed to components or to their simple interactions, but result from the totality of interactions between many components such as programs, data, hardware and OS. A)All test techniques can be useful at all levels, from unit to system B)There can be no meaningful system testing until there has been thorough component and integration testing.

FLOW GRAPHS AND PATH TESTING Testing: Path Testing is the name given to a Path family of test techniques based on judiciously selecting a set of test paths through the program. 2. 1. 1. If the set of paths are properly chosen then we have achieved some measure of test thoroughness. For example, pick enough paths to assure that every source statement has been executed at least once. 2. 1. 2 Motivation The common sense idea of executing every statement and branch atleast once under some test comes to almost every person who examines software testing in depth

2. 1. 3 THE BUG ASSUMPTION 1. The bug assumption for the path testing strategies is that something has gone wrong with the software that makes it take a different path than intended. o As an example "GOTO X" where "GOTO Y" had been intended. 2. 2 CONTROL FLOW GRAPHS: o The control flow graph is a graphical representation of a program's control structure. It uses the elements named process blocks, decisions, and junctions Flow Graph Elements: A flow graph contains four different types of elements. (1) Process Block (2) Decisions (3) Junctions (4) Case Statements

Figure 2. 1: Flow graph Elements

2. 2. 2 Process Block: � A process block is a sequence of program statements uninterrupted by either decisions or junctions. � It is a sequence of statements such that if any one of statement of the block is executed, then all statement thereof is executed. � Formally, a process block is a piece of straight line code of one statement or hundreds of statements. � A process has one entry and one exit. It can consist of a single statement or instruction, a sequence of statements or instructions, a single entry/exit subroutine, a macro or function call, or a sequence of these. 2. 2. 3 Decisions 1. A decision is a program point at which the control flow can diverge

2. Machine language conditional branch and conditional skip instructions are examples of decisions. 3. Most of the decisions are two-way but some are three way branches in control flow. 2. 2. 3 Case Statements 1. A case statement is a multi-way branch or decisions. 2. 2. 4 Junctions: 1. A junction is a point in the program where the control flow can merge. Examples of junctions are: the target of a jump or skip instruction in ALP, a label that is a target of GOTO.

2. 2. 5 CONTROL FLOW GRAPHS Vs FLOWCHARTS: 1. A program's flow chart resembles a control flow graph. 2. In flow graphs, we don't show the details of what is in a process block. 3. In flow charts every part of the process block is drawn. 4. The flowchart focuses on process steps, where as the flow graph focuses on control flow of the program. 5. The act of drawing a control flow graph is a useful tool that can help us clarify the control flow and data flow issues.

2. 2. 6 NOTATIONAL EVOULTION: The control flow graph is simplified representation of the program's structure. Figure 2. 2: Program Example (PDL)

Figure 2. 4: Control Flowgraph for example in Figure 2. 2

PATH TESTING - PATHS, NODES AND LINKS: Path: a path through a program is a sequence of instructions or statements that starts at an entry, junction, or decision and ends at another, or possibly the same junction, decision, or exit. A path may go through several junctions, processes, or decisions, one or more times. Paths consist of segments. The segment is a link - a single process that lies between two nodes. A path segment is succession of consecutive links that belongs to some path. The length of path measured by the number of links in it and not by the number of the instructions or statements executed along that path.

FUNDAMENTAL PATH SELECTION CRITERIA: There are many paths between the entry and exit of a typical routine. Every decision doubles the number of potential paths. And every loop multiplies the number of potential paths by the number of different iteration values possible for the loop. Defining complete testing: 0. Exercise(=WORK OUT) every path from entry to exit 1. Exercise every statement or instruction at least once 2. Exercise every branch and case statement, in each direction at least once

For X negative, the output is X + A, while for X greater than or equal to zero, the output is X + 2 A.

Path Testing : � Execute all possible control flow paths through the program: typically, this is restricted to all possible entry/exit paths through the program. Statement Testing (P 1): � Execute all statements in the program at least once under some test. If we do enough tests to achieve this, we are said to have achieved 100% statement coverage. Branch Testing (P 2): � Execute enough tests to assure that every branch alternative has been exercised at least once under some test.

Figure 2. 9: An example flowgraph to explain path selection

Testing Blindness 1. Most prog languages permit compound predicates at decisions – predicates of form A OR B or A AND B. and boolean expressions E. G X=17 (that is IF X=17 THEN ----) The opp branch is X NE 17, Which is equivalent to X>17 OR X<17. ---

E. G IF A GOTO BOB ELSE GOTO COB BOB: DO SOMETHING -------GOTO SAM COB: DO SOMETHING ELSE ------GOTO SAM ------SAM: IF X DO ALPHA ELSE DO BETA

Testing blindness is a pathological situation In which the desire path is achieved for the wrong reason. a)Assignment blindness: Occurs when the buggy predicate appears to work correctly because the specific value chosen for an assignment statement works with both the correct and incorrect predicate Correct Buggy X: =7 x: =7 -----IF Y>0 THEN IF X+Y>0 THEN

Testing blindness is a pathological situation In which the desire path is achieved for the wrong reason. a)Equality blindness: Occurs when the path selected by a prior predicate results in a value that works both for the correct and buggy predicate Correct Buggy Y: =2 THEN Y: =7 THEN -----IF X+Y>3 THEN IF X>1 THEN

Testing blindness is a pathological situation In which the desire path is achieved for the wrong reason. a)Self blindness: Occurs when the buggy predicate is a multiple of the correct predicate and as a result is indistinguishable along that path Correct Buggy X: =A -----IF X-1>0 THEN IF X+A-2>0 THEN

REVIEW: ACHIEVABLE AND UNACHIEVABLE PATHS: 1. We want to select and test enough paths to achieve a satisfactory notion of test completeness. 2. Extract the programs control flowgraph and select a set of tentative covering paths. 3. For any path in that set, interpret the predicates along the path as needed to express them in terms of the input vector. In general individual predicates are compound or may become compound as a result of interpretation. 4. Trace the path through, multiplying the individual compound predicates to achieve a boolean expression such as(A+BC) (D+E) (FGH) (IJ) (K) (l) (L).

5. Multiply out the expression to achieve a sum of products form: ADFGHIJKL+AEFGHIJKL+BCDFGHIJKL+BCEFGHIJKL 6. Each product term denotes a set of inequalities that if solved will yield an input vector that will drive the routine along the designated path. 7. Solve any one of the inequality sets for the chosen path and you have found a set of input values for the path. 8. If you can find a solution, then the path is achievable. 9. If you cant find a solution to any of the sets of inequalities, the path is un achievable. 10. The act of finding a set of solutions to the path predicate expression is called PATH SENSITIZATION.

HEURISTIC PROCEDURES FOR SENSITIZING PATHS: 1. This is a workable approach, instead of selecting the paths without considering how to sensitize, attempt to choose a covering path set that is easy to sensitize and pick hard to sensitize paths only as you must to achieve coverage. 2. Identify all variables that affect the decision. 3. Classify the predicates as dependent or independent. 4. Start the path selection with un correlated, independent predicates.

5. If coverage has not been achieved using independent uncorrelated predicates, extend the path set using correlated predicates. 6. If coverage has not been achieved extend the cases to those that involve dependent predicates. 7. Last, use correlated, dependent predicates.

PATH INSTRUMENTATION: Path instrumentation is what we have to do to confirm that the outcome was achieved by the intended path. Co-incidental Correctness: The coincidental correctness stands for achieving the desired outcome for wrong reason.

The above figure is an example of a routine that, for the (unfortunately) chosen input value (X = 16), yields the same outcome (Y = 2) no matter which case we select. Therefore, the tests chosen this way will not tell us whether we have achieved coverage. For example, the five cases could be totally jumbled and still the outcome would be the same.

Interpretive Trace Program: An interpretive trace program is one that executes every statement in order and records the intermediate values of all calculations, the statement labels traversed etc. If we run the tested routine under a trace, then we have all the information we need to confirm the outcome and, furthermore, to confirm that it was achieved by the intended path. The trouble with traces is that they give us far more information than we need. In fact, the typical trace program provides so much information that confirming the path from its massive output dump is more work than simulating the computer by hand to confirm the path.

Traversal Marker or Link Marker: A simple and effective form of instrumentation is called a traversal marker or link marker. Name every link by a lower case letter. Instrument the links so that the link's name is recorded when the link is executed. The succession of letters produced in going from the routine's entry to its exit should, if there are no bugs, exactly correspond to the path name.

Figure : Single Link Marker Instrumentation

Why Single Link Markers aren't enough: Unfortunately, a single link marker may not do the trick because links can be chewed by open bugs.

Double Link Marker Instrumentation.

1. The solution to the problem of single link marker method is to implement two markers per link: one at the beginning of each link and on at the end. 2. The two link markers now specify the path name and confirm both the beginning and end of the link.