What is Smoke Testing?

Smoke Testing is a very crucial step in the testing cycle. Whenever a new build is received from the development team, Smoke Testing ensures whether the new build is healthy or not. It is also known as “Day-0” testing. 

This guide discusses Smoke Testing, its types, how it differs from other types of Testing, and how to perform Smoke Testing.

What is Smoke Testing?

Smoke Testing is a software testing technique that basically determines whether a new build delivered by the Development team is bug-free or not. It gives the go-ahead to the QA team to move further with their testing rounds. 

It is also known as Confidence Testing or Build Verification / Build Promotion Testing and is made up of a collection of tests that are performed on each build received during the software development life cycle to determine its health. 

In simple words, Smoke tests verify whether the most important features work as expected and that there are no showstopper issues in the build that can potentially lead to blocking the entire testing team. It helps in deciding if the build is flawed or not and hence, prevents the entire team from wasting time or resources.

Why is Smoke Testing important?

Smoke Testing is important because it:

• Ensures that the product is testable and doesn’t let the QA teams get blocked.
• Uncovers major issues at the beginning of the sprint itself. This ensures that both Development and QA teams have enough time to fix and test the issue further. 
• Makes sure that the critical or acute functionalities of the product are working fine.
• Helps determine the system or the application’s stability.

Types of Smoke Testing

There are three different types of Smoke Testing:

1. Manual Testing: As the name suggests, in the case of Manual Testing, QA teams conduct Smoke Testing manually. QA teams write, develop and execute their smoke tests manually for existing as well as new features. Although this is very time taking, Manual Testing omes in with the benefit of human intelligence and acumen. 
2. Automated Testing: In this case, QA teams use automation tools to perform the tests instead of doing the tests on their own manually. Automation Testing can be a boon to the entire organization, as it saves a significant amount of time and resources. Additionally, it saves the testers from doing the mundane tasks of doing the same tests over and over again.
3. Hybrid Testing: Hybrid Testing, as the name suggests, is a combination of both Manual and Automated tests. It involves automation in some cycles of Smoke Testing as well as some amount of Manual intervention by the testers to test the application. Hybrid Testing is said to be better than the other two as it has the attributes of both methods and hence, can prove to be advantageous. 

Advantages of Smoke Testing

Some of the major advantages of Smoke Testing are:

• It helps in finding critical issues at an early stage
• It reduces the time and resources of the team, which eventually decreases the risk.
• It makes the application robust
• It reduces integration issues.
• It gives enough time to rectify major issues if any. 

Smoke Test Cases: Example

The below table illustrates a scenario of an E-Commerce website and some sample Smoke Tests for the same.When to perform Smoke Testing?

Now you must be wondering when we need to actually perform Smoke Testing. The answer is simple; Smoke Testing needs to be performed whenever there are some new features or capabilities added to the software, and a new build is delivered. In this new build, testers need to ensure that neither of the major key functionalities that were already existing is getting hampered, nor the new functionality is blocked due to any issue. 

Once a new build has been delivered by the dev team, some of the QA team members can start with Smoke Tests. They can prepare a list of cases that ensures all key features of the functionality are not affected and perform the tests. If the tests are successful, the build is promoted in the QA environment, and the rest of the QA team can further move on with the Functional Testing of the new feature as well as existing features. If the tests fail, the build is rejected, and the QA environment keeps running with the old stable build unless a new build is further delivered by the Dev team, which has all the issues addressed. This cycle continues till the end of Code Freeze.

For Example – In the example shared above, once a build is received which has the fix for Checkout functionality, Smoke Tests should ensure that the existing features of Login and Addition of Items to the Cart are working seamlessly, along with the Checkout functionality bug fix.

What happens if Smoke Testing is not performed?

If Smoke Tests are not performed at the early stages, and critical issues are found towards the later stages of the Software Development Life Cycle, it can incur a heavy cost on the entire team with respect to time, resources as well as money. These issues can also affect the overall release cadence and pose a major threat to the product image overall.

Differences between Smoke and Sanity Testing

Let us now take a look at some of the major differences between Smoke and Sanity Testing.

Differences between Smoke Testing and Regression Testing

Next, let us take a look at some of the major differences between Smoke and Regression Testing.

Automation and Smoke Testing

Automated Smoke Tests can drastically reduce the overall time spent, as we don’t need to perform Smoke tests only once. These need to be run again and again whenever there is a new build delivered.

Now, depending on the size and scale of the application, the no of test cases can vary. For a very mature product, the total number of test cases will usually be on the higher side. That can take even one full day for a tester to test manually. Imagine in that case, one person will have to spend one full day after every 2-3 days only for Smoke Tests. That can be terrible with respect to cost as well as with testers, as they are going to be burdened with the same task again and again. Plus if you report issues only after a day or two, half of the team already wasted some time in order to wait for the new build. 

Thus, Automated Smoke Testing is the way ahead. It can bring the overall time spent on it to some minutes and, as a result, improve the overall product development cycle. Testers can report issues faster and consume the latest builds faster. As a result, they can deliver more features in much lesser time. 

There are a lot of tools available for automating Smoke Tests. For example, Selenium, Rest Assured, TestNG, Cucumber, and PhantomJS are some of the most widely used tools today for automating smoke tests. Here, Selenium can be used for UI Automation, whereas Rest Assured can be used for Backend Automation. Using Cucumber or TestNG, one can maintain test cases, and PhantomJS is an option for integration of the tests with Continuous Integration tools such as Jenkins. One can select any tool or framework which meets the requirements best of their teams.

How to Perform Automated Smoke Testing?

Now that we know about the benefits of Automated Smoke Tests let us look at the steps for doing the same.

Step 1 Select a Testing Framework

The first step here is to choose a framework that meets all the requirements of the project. All the frameworks come with their own set of pros and cons, one needs to evaluate well because changing the framework at a later stage can prove to be very expensive.  Some of the majorly used tools or frameworks today are

1. Selenium – Selenium is an open-source project that provides a range of tools and libraries in order to support browser automation. It supports a lot of scripting languages like PHP, Java, Ruby, C++, JS, etc. It can easily be integrated with popular frameworks like Maven, Ant, etc. 
2. Appium: Appium is another open-source tool for running automation scripts and testing native, mobile-web, as well as hybrid applications on Android or iOS using a web driver. There are two versions available for Appium. One is a Node.js based tool, and a second is a desktop tool. However, both are used for mobile regression test automation. 
3. REST Assured: REST Assured is a framework that is built for REST service validation in Java and uses Java Libraries. In Rest Assured, the Java library acts like a headless client to act upon the Rest web services, and they are also capable of validating the HTTP responses from the server. The tool helps in improving the quality and maintainability of tests.
4. Maven: Maven is an automation tool that is used primarily for Java projects. It can also manage scripts written in C#, Ruby, Scala, and other languages.
5. BrowserStack: BrowserStack is a cloud web and mobile testing platform that provides the ability to test websites and mobile applications across on-demand browsers, operating systems, and real mobile devices. You can access 3000+ different devices and browser versions.  You can run Selenium, Cypress, Playwright,  Puppeteer, Appium, Espresso, and XCUI tests seamlessly using BrowserStack. It also offers integration with popular CI CD tools like Jenkins, Team City, Circle CI, etc. Plus, using tools like BrowserStack Automate you can access 3000+ real device browser combinations to test your cases thoroughly. Also, BrowserStack’s Cloud Selenium Grid allows you to run parallel tests and as a result, help in achieving faster and more efficient automation testing.

Run Automated Tests on Real Devices for Free

Step 2 Select a Test Runner (Optional)

In case your framework doesn’t come with a Test Runner already, you need to select one, ensuring that it works smoothly when integrated with chosen framework. 

Some of the widely used Test Runners are:

1. TestNG
2. Cucumber

Step 3 Write the code

Once the Framework and Test Runner are selected and built, the coding part begins, where we need to build our utilities and write test cases using the same. Once the tests are automated, we can create a Test Suite like Smoke, Sanity, Regression, etc.

Step 4 Integrate the Tests With a CI Platform

Next step is to integrate these tests with a CI Platform in order to run your tests. Where you can also schedule your tests or integrate it with the build development flow so that your smoke tests get triggered automatically once a new build is out, or your Sanity tests are triggered automatically every day at 12 pm regardless of the build. Some of the options available are

1. Jenkins
2. CircleCI
3. Semaphore 
4. GitLab CI

Step 5 Schedule or Integrate your tests to run frequently 

As mentioned in the previous point, CI tools can help in running the tests without any human intervention. So, once the tool is selected and the tests are integrated, we can either schedule them to run every day at an nth time, or we can integrate them with the Development team’s pipeline that will trigger the tests whenever a new build is out. This way, we can reap most of the benefits of a CI tool. 

Conclusion

Now, we know how Smoke Testing plays a very crucial role in the overall Software Development lifecycle. However, there are still a lot of places where this is usually being ignored due to time or resource crunch, but one must understand that dedicating time and energy towards Smoke Testing or its Automation is definitely going to reap benefits over the longer run. Plus, as the product grows, the need for Smoke Testing intensifies more. 

In this growing era of automation and cut-throat competition, QA teams must attempt to place a Smoke Testing solution through automation in place and enjoy the benefits of time and resource-management along with a very stable and robust product. 

It is always recommended to run tests on real devices to ensure all the real user conditions are taken into consideration while testing for more accurate test results. BrowserStack allows access to 3000+ real devices and browsers for a comprehensive testing experience.

Try BrowserStack Now