What is Fault Injection in Software Testing?

Software Fault Injection deliberately injects faults into a software system to determine whether the system can withstand error conditions. It is a practice of stress testing or monkey testing the software by injecting faults that result in disruptive events, observing how the software responds to the events and implementing improvements.

Fault Injection is a complementary technique to software testing for improving software performance and resiliency. The usual testing methods validate the correct behavior of the software, whereas Fault Injection evaluates the correct behavior of software with faults that are injected intentionally. Let’s understand further why fault Injection is a complementary technique in software testing for improving software performance and resiliency.

Software Fault Injection Mechanism

In the case of software testing, the faults are injected into software to accomplish the following:

• Evaluate if the software can withstand and recover from error conditions
• Identify potential defects or security vulnerabilities before production 
• Evaluate software robustness.
• Improve the design of the system for resilience and performance

The Software Fault Injection works based on the Fault-Error-Failure-Cycle mechanism. The cycle consists of the following steps.

1. Fault: Faults (adjusted or hypothesized) are injected into the code either in compile time or run time.
2. Error: The faults, when activated, lead to multiple errors and change the software’s state to an undesired state.
3. Failure: The undesired state caused due to multiple errors that fail the software. For example, outage of service.

Source

Software Fault Injection Types

Fault Injection can be categorized into two types:

• Compile-time Fault Injection: A fault is injected into the source code during the compile time. The code changes can include the addition or modification of a code.
•  Run-time Fault Injection—A fault is injected into software while running. A fault can be injected dynamically to external libraries or added to other dependencies during runtime. The run-time Fault Injection can be further classified as:
  • Time-based trigger— A trigger is set to inject a fault at a specified time.
  • Interrupt-based trigger— A trigger is set to inject using a specific location in the code or event in the system. 

The following table lists the differences between Compile-time and Run-time Fault Injections:

Fault Injection Environment

The Fault Injection environment typically consists of the following components.

• Workload Generator: Contains software workloads.
• Fault Injector: Injects faults into the target software while the commands are executed from the workload generator.
• Monitor: Monitors the execution of the commands and collects data as required.
• Data Collector: Performs online data collection.
• Data Analyzer: Performs data processing and analysis.
• Controller: Controls the experiment. The controller is a program that can run on the target software system or on a separate computer. 

Source

Compile-time Fault Injection Examples

This section lists some examples of how faults are injected during compile time by modifying the code. The code injected through this method results in errors similar to the errors that the programmers unintentionally commit.

Example: Code Modification

int main()
{
int a = 10;
while ( a > 0 )
{
cout << "GFG";
a = a - 1;
}
return 0;
}

Modified Code:

int main()
{
int a = 10;
while ( a > 0 )
{
cout << "GFG";
a = a + 1; // '-' is changed to '+'
}
return 0;
}

In the modified code, a fault is injected by modifying the code from “a=a-1” to “a=a+1”. The variable “a” value increases and never meets the while condition. Therefore, the while loop goes into an infinite loop and never ends.

Example: Code Insertion

The following example shows how a fault is injected during the compile time by inserting the code instead of modifying the code. In this case, an additional code is added to change the parameter’s value or value.

Original Code:

int main()
{
int a = 10;
while ( a > 0 )
{
cout << "GFG";
a = a - 1;
}
return 0;
}

Modified Code:

int main()
{
int a = 10;
while ( a > 0 )
{
cout << "GFG";
a = a - 1;
a++; // Additional code
}
return 0;
}

In this example, an additional line of code “a++” is added to change the value of the variable “a”. 

Run-time Fault Injection Example

The following figure shows the exception triggered when a fault is injected on a dummy .NET WinForm application named TwoCardPokerGame.exe. A C# program that runs in the backend at run time alters the behavior of the software when you click the Evaluate button. In this situation, the Two Card Poker application cannot handle the exception and displays an error message.

Source

Sample Fault Injection code:

using System;
namespace FaultHarness {
class Program {
static void Main(string[] args) {
try {
Console.WriteLine("\nBegin TestApi Fault Injection environmnent session\n");
// create fault session, launch application
Console.WriteLine("\nEnd TestApi Fault Injection environment session");
}
catch (Exception ex) {
Console.WriteLine("Fatal: " + ex.Message);
}
}
} // class Program
} // ns

When to use Fault Injection in Software Testing?

Fault Injection is a recommended approach to assess the fault tolerance of software with multiple dependent components. The software can be considered fault tolerant only when all dependent software layers and components are fault tolerant.

Fault Injection testing can be used when your software is built using one or more third-party software or APIs and if it is deployed in a multi-platform or cross-platform environment.  The software with such dependencies is prone to dependency disruptions. If an API used in the software fails, it propagates the failure to the dependent component in the software.  Fault Injection is an acknowledged method for assessing the dependability of software systems. It can also evaluate software during the initial stage of the SDLC.

Fault Injection tools

Numerous tools are available in the market, using which you can inject a fault into your software. The following are the widely used tools.

•  Xception
•  beStorm
•  Holodeck
•  Grid-FIT
•  Orchestra
•  ExhaustiF
•  The Mu Service Analyzer

Follow-Up Read: Essential Tools for Remote Software Testing

Advantages and Disadvantages of Software-based Fault Injection

Summary

Software Fault Injection is a recommended approach to evaluate the tolerance and reliability of the software. Primarily, it is a promising approach to evaluate the software that is deployed in a complex and cross-platform environment. However, with fault injection, it’s crucial to test on a real device cloud that facilitates manual and automated testing on the most-preferred testing infrastructure. 

On the BrowserStack infrastructure, you can access: 

• BrowserStack Live: Cross browser testing on desktop & mobile
• BrowserStack App Live: Interactive Mobile App Testing on an interactive cloud of real iOS & Android devices
• BrowserStack Automate: Run your UI test suite in minutes with parallelization on a real browser and device cloud.
• BrowserStack App Automate: Automated mobile app testing on real mobile devices.
• BrowserStack Enterprise: Ship quality releases at the speed of Agile and enable your teams to test continuously, at scale.
• Percy Visual Testing: Integrate Percy into your existing CI/CD pipeline and review visual changes with every commit.

Sign Up & Start Testing