MUTATION TESTING MUTATION TESTING Mutation testing is a

MUTATION TESTING

MUTATION TESTING • Mutation testing is a technique that focuses on measuring the adequacy (quality) of test data (or test cases). • Modify a program by introducing a single small change to the code • A modified program is called mutant • A mutant is said to be killed when the execution of test case cause it to fail. The mutant is considered to be dead • A mutant is an equivalent to the given program if it always produce the same output as the original program • A mutant is called killable or stubborn, if the existing set of test cases is insufficient to kill it • A mutation score for a set of test cases is the percentage of nonequivalent mutants killed by the test suite • The test suite is said to be mutation-adequate if its mutation score is 100%

the following program P that finds rank corresponding to the first time the maximum value appears in the array 1. 2. 3. 4. 5. 6. 7. 8. main(argc, argv) int argc, r, i; char *argv[]; { r = 1; for i = 2 to 3 do if (atoi(argv[i]) > atoi(argv[r])) r = i; printf("Value of the rank is %d n", r); exit(0); }

Test. Cases • • • Test case 1: Input: 1 2 3 Output: Value of the rank is 3 Test case 2: Input: 1 2 1 Output: Values of the rank is 2 Test case 3: Input: 3 1 2 Output: Value of the rank is 1

let us mutate the program P. We can start with the following changes: 1. Mutant 1: Change line 5 to for i = 1 to 3 do 2. Mutant 2: Change line 6 to if (i > atoi(argv[r])) r = i; 3. Mutant 3: Change line 6 to if (atoi(argv[i]) >= atoi(argv[r])) r = i; 4. Mutant 4: Change line 6 to if (atoi(argv[r]) > atoi(argv[r])) r = i;

• Mutants 1 and 3: The programs will completely pass the test suite. In other words, mutants 1 and 3 are not killed. • Mutant 2: The program will fail test case 2. • Mutant 4: The program will fail test case 1 and test case 2.

calculate the mutation score • Mutation score = 100×D/(N −E), where ▫ D: is the dead mutants, ▫ N: the total number of mutants, ▫ E: the number of equivalent mutants. • Assuming that mutants 1 and 3 are nonequivalent. • mutation score is 50%. It is low because we assumed that mutants 1 and 3 are nonequivalent to the original program • we need to add new test cases to kill these two mutants

1 and 3 are equivalent mutants or those are killable? ? !! • • analyze mutant 1 --- is an equivalent mutant. add a fourth test case as follows: Test case 4: Input: 2 2 1 P : the output “Value of the rank is 1” mutant 3 : the output “Value of the rank is 2. ” Thus, this test data kills mutant 3, which give us a mutation score of 100%.

Steps to build a robust test suite • Step 1: Begin with a program P and a set of test cases T known to be correct. • Step 2: Run each test case in T against the program P. ▫ If it fails (o/p incorrect) P must be modified and restarted. Else, go to step 3 • Step 3: Create a set of mutants {Pi }, each differing from P by a simple, syntactically correct modification of P.

• Step 4: Execute each test case in T against each mutant Pi. • If the o/p is differ the mutant Pi is considered incorrect and is said to be killed by the test case • If Pi produces exactly the same results: ▫ P and Pi are equivalent ▫ Pi is killable (new test cases must be created to kill it)

• Step 5: Calculate the mutation score for the set of test cases T. • Mutation score = 100×D/(N −E), • Step 6: If the estimated mutation adequacy of T in step 5 is not sufficiently high, then design a new test case that distinguishes Pi from P, add the new test case to T, and go to step 2.