Mutation Testing
What is mutation testing?
Mutation testing is a white box method in software testing where we insert errors purposely into a program (under test) to verify whether the existing test case can detect the error or not. In this testing, the mutant of the program is created by making some modifications to the original program.
The primary objective of mutation testing is to check whether each mutant created an output, which means that it is different from the output of the original program. We will make slight modifications in the mutant program because if we change it on a massive scale than it will affect the overall plan.
When we detected the number of errors, it implies that either the program is correct or the test case is inefficient to identify the fault.
Mutation testing purposes is to evaluate the quality of the case that should be able to fail the mutant code hence this method is also known as Fault-based testing as it used to produce an error in the program and that why we can say that the mutation testing is performed to check the efficiency of the test cases.
What is mutation?
The mutation is a small modification in a program; these minor modifications are planned to typical low-level errors which are happened at the time of coding process.
Generally, we deliberate the mutation operators in the form of rules which match the data and also generate some efficient environment to produce the mutant.
Types of mutation testing
Mutation testing can be classified into three parts, which are as follows:
- Decision mutations
- value mutations
- Statement mutations
Let us understand them one by one:
Decision mutations
In this type of mutation testing, we will check the design errors. And here, we will do the modification in arithmetic and logical operator to detect the errors in the program.
Like if we do the following changes in arithmetic operators:
- plus(+)→ minus(-)
- asterisk(*)→ double asterisk(**)
- plus(+)→incremental operator(i++)
Like if we do the following changes in logical operators
- Exchange P > → P<, OR P>=
Now, let see one example for our better understanding:
Value mutations
In this, the values will modify to identify the errors in the program, and generally, we will change the following:
- Small value à higher value
- Higher value à Small value.
For Example:
Statement Mutations
Statement mutations means that we can do the modifications into the statements by removing or replacing the line as we see in the below example:
In the above case, we have replaced the statement r=15 by s=15, and r=25 by s=25.
How to perform mutation testing
To perform mutation testing, we will follow the below process:
- In this, firstly, we will add the errors into the source code of the program by producing various versions, which are known mutants. Here every mutant having the one error, which leads the mutant kinds unsuccessful and also validates the efficiency of the test cases.
- After that, we will take the help of the test cases in the mutant program and the actual application will find the errors in the code.
- Once we identify the faults, we will match the output of the actual code and mutant code.
- After comparing the output of both actual and mutant programs, if the results are not matched, then the mutant is executed by the test cases. Therefore the test case has to be sufficient for identifying the modification between the actual program and the mutant program.
- And if the actual program and the mutant program produced the exact result, then the mutant is saved. And those cases are more active test cases because it helps us to execute all the mutants.
Advantages and disadvantages of Mutation Testing
Advantages
The benefits of mutation testing are as follows:
- It is a right approach for error detection to the application programmer
- The mutation testing is an excellent method to achieve the extensive coverage of the source program.
- Mutation testing helps us to give the most established and dependable structure for the clients.
- This technique can identify all the errors in the program and also helps us to discover the doubts in the code.
Disadvantages
The drawbacks of mutant testing are as follows:
- This testing is a bit of time taking and costlier process because we have many mutant programs that need to be created.
- The mutation testing is not appropriate for Black-box testing as it includes the modification in the source code.
- Every mutation will have the same number of test cases as compare to the actual program. Therefore the significant number of the mutant program may need to be tested beside the real test suite.
- As it is a tedious process, so we can say that this testing requires the automation tools to test the application.