TestNG vs Cucumber: Core Differences

Functional testing is pivotal before any feature release for a software application, to check if the feature is working as per the business requirements. While there are different frameworks that are used to perform functional testing of web applications, this article compares TestNG and Cucumber to highlight when to use which.

What is TestNG Framework?

TestNG is a comprehensive testing framework written in Java that can be used for writing unit tests, functional tests, integration tests, and end-to-end tests. It is inspired by JUnit and is developed by Cédric Beust.

TestNG simplifies writing tests by providing Annotations. TestNG Annotations are nothing but a note written on top of the methods which signifies what and how the method should behave. The annotation are special methods that start with “@”

How to write tests in TestNG?

Let’s take a look at the unit test case where we will be checking if the given PIN is correct or not in a banking application:

Step 1: Writing the business logic of the test 

• The core logic which needs to be tested with the required scenarios is placed here
• Check the length of the given input PIN
• This is marked by the annotation @Test

Step 2: Insert appropriate TestNG annotations

There might be a requirement where a task needs to be repeatedly done before running each test case. TestNG can be used to automate it. TestNG provides annotations like @BeforeTest and @AfterTest

In this case, since it is required to open the banking app, enter the PIN and close the baking app after every test, hence using annotations as follows:

• @BeforeTest – Open the Banking app
• @Test – Has the business logic to verify if the given PIN is valid or not
• @AfterTest – Closes the banking app

Step 3: Create an XML file with required groups and classes, where tests need to be executed by using any modern IDE tool such as IntelliJ or Eclipse.

Step 4: Run TestNG

What is Cucumber Framework?

Cucumber is a testing tool that supports Behaviour Driven Development (BDD). It provides a way to write tests that can be even understood by non-technical folks. Cucumber is written in Ruby.

How to write tests in Cucumber?

In Cucumber, the tests are written files that have an extension called .feature written in Gherkin. Like YAML, Gherkin is a line-oriented language that uses indentation to define structure.

A .feature file contains a feature defining high-level functionality and scenarios describing individual test cases and how they behave within a feature. A particular feature can have multiple scenarios.

Writing scenarios:

The scenarios in Cucumber framework can be written in the following format “Given-When-Then”.

Example:

This example showcases a feature called “checking the bank balance from the banking app”. 

The feature functionality can be described as 

• User logins to the banking app, 
• Selects the check balance option, 
• Bank balance displays when a user enters the correct 4-digit PIN.

Feature: Checking bank balance from the banking app.

• Scenario 1: Check the balance by entering the correct PIN.

Given user opens the banking application and selects Check Balance

When user enters PIN as 1234

Then bank balance should be displayed

• Scenario 2: Check the balance by entering 5-digit PIN

Given user opens the banking application and selects Check Balance

When user enters PIN as 12345

Then app should show a notification that an incorrect pin has been entered

These kinds of tests are easily understood by non-technical users (business stakeholders) as it has been written descriptively.

TestNG vs Cucumber: When to use which

Although TestNG and Cucumber are used for testing features, the way of testing features differs a lot in each of the cases. Let’s take a look at TestNG vs Cucumber to understand the differences between them

By analyzing the differences between TestNG and Cucumber, here’s how you can decide when to use which: 

• When there is a need to test an important feature that needs to be understood by the business stakeholders, Cucumber can be used. 
• To test a technically implemented complex business logic/algorithm where behavior is not a deciding factor TestNG can be used.

Whether you choose TestNG for TDD or Cucumber for BDD, to perform automated functional testing, it is recommended to run the tests on real devices for better accuracy as opposed to emulators and simulators. By leveraging real device cloud, like BrowserStack you can access 3000+ browser-device combinations and widen the test coverage of your web applications. This helps in ensuring that the test cases are running as expected even in real user conditions.

Test on Real Devices for Free