What is mutation testing in software testing?

Overview

Software testing is an essential part of the Software Development Life Cycle (SDLC) as it ensures the quality of a software product. Out of the several software testing techniques employed, we will discuss the mutation testing technique.

Definition

Mutation testing is when an already built program is modified to remove ambiguities in the source code and get rid of buggy code.

Mutation testing is classified as a white-box testing technique.

Process

• Several mutant programs are generated by introducing faults in the original program’s source code.

Note: Each mutant program must contain a single fault.

• Test cases for the original program are applied to mutant programs as well as the original program.
• The results from the mutant program and original program are compared.
• If the results differ from the original program, the mutant program is killed, meaning it is removed from future tests. Otherwise, the mutant program is kept, and the test is repeated with better test cases.

Types of mutation testing

Value mutation

In value mutation, the value of a variable is modified to produce a mutant program. Following is an example:

Original program

int divisor = 93283933;
int dividend = 123;
int remainder = dividend % divisor;
cout << remainder;

Output:

123

Mutant program

int divisor = 93;
int dividend = 123;
int remainder = dividend % divisor;
cout << remainder;

Output:

30

In the example above, the value of divisor was changed from 93283933 to 93 to generate the mutant program. Since the output of the mutant program is different from the original program, the mutant program is killed.

Decision mutation

In decision mutation, a logical or arithmetic operator is modified to produce a mutant program. Take a look at the example below:

Original program

int x = 10;
int y = 20;
if (x <= y)
  cout << "x is less than or equal to y";
else
  cout << "x is greater than y";

Output:

x is less than or equal to y

Mutant program

int x = 10;
int y = 20;
if (x > y)
  cout << "x is less than or equal to y";
else
  cout << "x is greater than y";

Output:

x is greater than y

In the example above, the ‘less than equal to’ operator (<=) was replaced with the greater than operator (>) to generate the mutant program. Since the output of the mutant program is different from the original program, the mutant program is killed.

Statement mutation

In statement mutation, a statement is removed or replaced by another statement to produce a mutant program. Take a look at the example below:

Original program

int x = 10;
int y = 20;
if (x <= y)
  cout << "x is less than or equal to y";
else
  cout << "x is greater than y";

Output:

x is less than or equal to y

Mutant program

int x = 10;
int y = 20;
if (x <= y)
  cout << "x is less than or equal to y";
else
  cout << "y is NOT greater than x";

Output:

x is less than or equal to y

In the example above, the else statement is changed from cout << "x is greater than y"; to cout << "y is NOT greater than x"; to generate the mutant program. Since the output of the mutant program is the same as the original program, the mutant program is not killed, and further testing is required.

Advantages

• Mutation testing has a 100% chance of detecting ambiguities and errors in source code.
• The entire code base is covered in this type of testing.

Disadvantages

• Mutation testing is costly and time-consuming since a lot of mutant programs have to be generated.
• Since it is a very time-consuming process, automation testing is necessary.
• Since the nature of this testing revolves around changing the source code, it does not qualify for black-box testing.