Course Notes Set 9 Functional Testing Computer Science

Course Notes Set 9: Functional Testing Computer Science and Software Engineering Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -0

Functional Testing • Dynamic (running) black-box (blindfolded) testing • Also known as behavioral testing • Based on specification - if one not available, the software is the spec • Exam inputs/outputs: needs test cases Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -1

Purpose of Testing • Test-to-pass - make sure the software minimally works - don’t push it to the limit - apply simplest and/or straightforward cases - not to find bugs, initially - do this test FIRST Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -2

Purpose of Testing • Test-to-fail - after test-to-pass - design and run test cases with the purpose to break the software - probe the known and unknown weaknesses - errors forcing Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -3

Functional (Black Box) Testing • Knowing the specified function (requirements), design test cases to ensure that those requirements are met. – Example : Sort (list); • Structural Testing - How well is the code exercised? • Functional Testing - How well does Sort perform its intended function? • In general, complete functional testing is not feasible – Attempting to test every possible input to the function • A randomly selected set of test cases is statistically insignificant – “Not all test cases are created equally” • Test case selection – Based on characteristics of input and output sets relative to specified functionality. Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -4

Functional Testing • Types of errors looked for during functional testing – – – Incorrect function or missing function Interface errors External database errors Performance errors (including stress testing) Initialization/termination errors • Tests are designed to answer the following questions – – – How is functional validity tested? What classes of input will make good test cases? Is the system particularly sensitive to certain input values? How are the boundaries of a data class isolated? What data rates and data volume can the system tolerate? What effect will specific combinations of data have on system operations? [Adapted from Software Engineering 4 th Ed , by Pressman, Mc. Graw-Hill, 1997] Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -5

Goals and Methods of Functional Testing • Goals – Produce test cases that reduce the overall number of test cases – Generate test cases that will tell us something about the presence or absence of errors for an entire class of input • Methods/Approaches – – – Equivalence partitioning Boundary value analysis Matrix of functional possibilities Cause-effect graphing Decision Tables Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -6

Equivalence Partitioning • It is impossible to test all cases • Equivalence partitioning provides a systematic means for selecting cases that matter and ignoring those that don’t • An equivalence class or equivalence partition is a set of test cases that tests the same aspect or reveals the same bugs e. g. , If X >= 15 then do-this else do-that (- 15) 15 Auburn University Computer Science and Software Engineering (15 ) COMP 6710 Course Notes Slide 9 -7

Equivalence Partitioning • Equivalence partitions – groups for similar inputs, outputs, and/or operation of the software e. g. , file-name, 1. . 255 characters - valid characters - invalid characters - valid length - invalid length Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -8

Equivalence Partitioning e. g. , copy operation - copy menu - c or C - Ctrl-c or Ctrl-Shift-c • Fully tested in the first effort, equivalence partitioning (1 case each) test for new versions • Goal: to reduce the set of possible test cases • Too few partitions => may not reveal all catchable bugs Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -9

Equivalence Partitioning • Equivalence partitioning divides the input domain of a program into classes of data from which test cases can be derived – Ideally, each test case could uncover classes of errors, thereby reducing the total number of test cases that must be developed • Input condition - some kind of condition placed on the input – Typically a specific value, a range of values, a set of related values, or a Boolean condition • Equivalence Class - a set of valid or invalid states for input conditions – Range - 1 valid and 2 invalid equivalence classes are defined – Specific Value - 1 valid and 2 invalid equivalence classes are defined – Set - 1 valid and 1 invalid equivalence class are defined – Boolean - 1 valid and 1 invalid equivalence class are defined [Adapted from Software Engineering 4 th Ed , by Pressman, Mc. Graw-Hill, 1997] Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -10

Example Ϭ¹¹¹¹¹ Þßàfunction in_list (input 1: name_type; ϧÏÏÏÏÏÏÏÏÏÏinput_list : list_names) ϧÏreturn boolean is Ϫ˹¹¹¹ ÏϧÏíÏp : list_names; Ïϧbegin ÏϨ¹¹Ïp : = input_list; ÏϨ¹¹±while not(p = null) loop ÏϧÏÏ·¹³´if Ada. Strings. Fixed. Index ÏϧÏϯϵ§(Source => String(p. name), ÏϧÏϯϵ§Pattern => String(input 1)) /= 0 then Ï ÄÏϯϵ¾¹¹Ïreturn true; ÏϧÏϯ϶´else ÏϧÏϯϸ¾¹¹Ïp : = p. next_name; ÏϧÏϯÏÈÏend if; ÏϧÏÏ°end loop; Ï Ä¹¹Ïreturn false; Auburn University Computer Science and Software Engineering Equivalence Classes: (1) Inputs where input 1 is in the list (2) Inputs where input 1 is not in the list Specific Input Partitions: List input 1 Empty ? One element In the list One element Not in the list >One element First element >One element Last element >One element Middle element >One element Not in list Test Cases List <nil> bird, cat, owl dog, pig, chicken input 1 ? bird fish bird chicken Output false true . . . COMP 6710 Course Notes Slide 9 -11

Boundary Value Analysis • Range : a. . b ( ) a (( ) b – Example : 100. . 200 • Test cases : 99, 100, 101, 199, 200, 201 • Number of values – Test cases that exercise minimums and maximums • Apply the above to the output conditions – Try to drive output to invalid range • Internal data structures with boundaries – Example : A(1. . 100) with test cases A(0), A(1), A(2), A(99), A(100), A(101) – A(0) and A(101) should generate exceptions Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -12

Boundary Condition Test Cases • If software can operate on the edge of its capabilities, it will almost certainly operate well under normal conditions • For I = 1 to 10 data (I) = -1; end; 10 elements, data(0), data(1), data(2) data(9), data(10), data(11) Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -13

Boundary Condition Test Cases • Boundary conditions for a legitimate triangle • Boundary conditions for side classification • Boundary conditions for angle classification • Valid input/extremes Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -14

Boundary Condition Test Cases • Types of Boundary conditions numeric character position speed location size quantity Also, extremes first/last min/max start/finish over/under empty/full Shortest/longest slowest/fastest largest/smallest … Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -15

Boundary Condition Test Cases • Partitions - boundary - one or two valid points inside the boundary - one or two invalid points outside the boundary e. g. , First – 1 / Last + 1 Smallest – 1 / Largest + 1 Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -16

Sub-boundary Conditions • Also known as Internal boundaries • Bit, nibble, byte, word, K, M, G, T • Why? E. g. , 256 commands, 15 are frequently used. Needs only a nibble. 0 XXXX nibble, 1 XXXX byte Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -17

Sub-boundary Conditions • ASCCI table – boundaries not obvious • Default, empty, blank, null, zero, none (may be of a separate equivalence partition and treated individually) • Invalid, wrong, incorrect, garbage data (test to fail) Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -18

Matrix of Functional Possibilities • Input/Output Conditions – If the number of combinations of input/output is manageable, then consider using a matrix of functional possibilities – Especially useful if the input/output combinations are enumerated in the requirements specification • Example : Input (or output) will be a combination of {A, B} and {x, y, z} Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -19

Example : The Triangle Problem • Input – 3 floating point numbers • Processing – Determine if the 3 numbers form a triangle • If not, print message “Not a Triangle” • If it is a triangle – Classify according to side : equilateral, isosceles, scalene – Classify according to largest angle : acute, right, obtuse • Output – List the 3 numbers – List the classification or “Not a triangle” Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -20

MFP for the Triangle Problem Additional Functional Test Cases (if any): Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -21

Cause Effect Graphing • Causes (input conditions) and effects (actions) are listed for a module, and an identifier is assigned to each • A cause-effect graph is developed – Looking for causes without effects – Looking for effects without causes • The graph is converted to a decision table (if a decision table has been used as a design tool, developing the graph and table is not necessary) • Decision table rules are converted to test cases Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -22

Cause-Effect Graph. Symbology c 1 e 1 Constraints Identity a a a E c 1 e 1 “Not” I O b b c Exclusive c 1 b Only One Inclusive e 1 “And” c 2 a a R c 1 e 1 “Or” c 2 Auburn University Computer Science and Software Engineering M b Requires b Masks COMP 6710 Course Notes Slide 9 -23

Cause-Effect Graphing Example • The CHANGE subcommand - used to modify a character string in the “current line” of the file being edited – Inputs • Syntax : C /string 1/string 2 • String 1 represents the character string you wish to replace – 1 -30 characters – Any character except ‘/’ • String 2 represents the character string that is to replace string 1 – 0 -30 characters – Any character except ‘/’ • At least one blank must follow the command name “C” Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -24

Cause-Effect Graphing Example – Outputs • Changed line is printed to the terminal if the command is successful • “NOT FOUND” is printed if string 1 cannot be found • “INVALID SYNTAX” is printed if the command syntax is incorrect – System Transformations • If the syntax is valid and string 1 can be found in the current line, then string 1 is removed and string 2 replaces it • If the syntax is invalid or string 1 cannot be found, the line is not changed Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -25

Cause-Effect Graphing Example • Cause 1: The first nonblank character following the “C” and one or more blanks is a ‘/’ • Cause 2: The command contains exactly two ‘/’ characters • Cause 3: String 1 has length 1 • Cause 4: String 1 has length 30 • Cause 5: String 1 has length 2 -29 • Cause 6: String 2 has length 0 • Cause 7: String 2 has length 30 • Cause 8: String 2 has length 1 -29 • Cause 9: The current line contains an occurrence of string 1 • Effect 1: The changed line is typed • Effect 2: The first occurrence of string 1 in the current line is replaced by string 2 • Effect 3: NOT FOUND is printed • Effect 4: INVALID SYNTAX is printed Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -26

Complete Cause-Effect Graph c 1 e 1 c 2 c 3 c 4 i 1 i 3 e 2 c 5 c 6 e 3 c 7 i 2 c 8 c 9 e 4 Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -27

Converting to a Decision Table Auburn University Computer Science and Software Engineering COMP 6710 Course Notes Slide 9 -28