TestNG Groups

TestNG is one of the most popular and widely used open-source and free Java testing frameworks used in the automation testing suite.

Inspired by JUnit and NUnit, it is designed to simplify the process of writing and executing the tests in Java. It supports various types of testing such as functional, unit, and end-to-end testing.

Testers love TestNG as it supports annotations, data-driven testing, test dependency, test grouping, reporting, and integration with build tools.

What are TestNG Groups?

Selenium is often used with the TestNG framework as it provides a way to write test cases systematically, generate test reports, and assign test case priorities.

Along with these, it also provides a compelling feature known as TestNG groups. It is a way to organize and categorize test methods, which allows running a specific set of test cases based on the tester’s needs.

Typically, a large automation suite contains all kinds of test cases such as smoke, sanity and regression. Testers can assign a group to all these test cases at the time of creation and using the TestNG xml file they can execute the required group of test cases using TestNG group feature.

Below is a TestNG class with each test case assigned to a TestNG group:

import org.openqa.selenium.By;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import org.testng.Assert;

import org.testng.annotations.Test;

 

public class TestNGDemo {          

           @Test(groups = {"smoke"})

           public void test1() {

                          System.out.println("This is Test case 1");

           }
          

           @Test(groups = {"sanity"})

           public void test2() {

                          System.out.println("This is Test case 2");

           }
          

           @Test(groups = {"regression"})

           public void test3() {

                          System.out.println("This is Test case 3");

           }

          
           @Test(groups = {"smoke", "regression"})

           public void test4() {

                          System.out.println("This is Test case 3");

           }


}

Create a TestNG xml file for the above test class and add the <groups> tag to run group specific test cases.

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="TestNGGroup">

  <test thread-count="5" name="Test">

  <groups>

  <run>

  <include name="smoke"></include>

  </run>

  </groups>

<classes>

   <class name="com.qa.testcases.TestNGDemo"/>

</classes>

  </test> <!-- Test -->

</suite> <!-- TestNGGroup -->

Run the testing.xml file and observe that only test1, which is of the smoke group, gets executed, and the rest are excluded from the test execution.

Benefits of TestNG Groups

TestNG group has several benefits as it helps to improve the test case management, organization, and execution in an automation test suite. Below are some of the major ones:

• Categorization and selective test execution: TestNG group allows the tester to categorize the test cases based on functionality or any other criteria. Categorization helps the testers to run a specific subset of test cases without executing the entire automation test suite using <groups> and <include> tags. Testers can also exclude a group of test cases by using an exclude tag.
• Parallel Execution: For larger automation suites, tests can be grouped and run in parallel to speed up and shorten the execution time.
• Easier test case maintenance: When test cases are grouped properly, it becomes easy for the tester to add or update any existing test case. Any new test case added and assigned a group will be automatically added to the particular group of test cases at the time of execution using TestNGxml file.
• Dependency management: To manage dependencies between tests, groups can be used to ensure that certain prerequisites are run before the dependent test cases.

Notes:

• Groups are declared in the testing.xml file and can be found inside <test> or <suite> tag.
• Groups defined in <test> tag apply only to that particular test tag.
• Groups defined in <suite> tag apply to all the test tags in the testing.xml file.

Tutorial: How to Group Test cases in TestNG

Below are the steps you should follow to group the test cases and then later execute them using testing.xml:

1. SetUp TestNG in IDE: Install TestNG and configure in your respective IDE (Eclipse or IntelliJ). Add TestNG dependency in the Maven’s pom.xml if you are using Maven for building your project.

<dependency>

<groupId>org.testng</groupId>

<artifactId>testng</artifactId>

<version>7.4.0</version> <!-- Check for the latest version from Maven Repository-->

<scope>test</scope>

</dependency>

Also, install the TestNG plugin from the marketplace in Eclipse/ IntelliJ IDE.

2. Assign test methods with groups attribute:

To use test groups, you have to use the “groups” attribute with the @Test annotation for the test method you wish to apply groups. The syntax to use groups in @Test method is as follows.

@Test(groups=”group_name”)

           public void testMethodName(){

                          //Code

           }

You can also apply more than one group to any test method. Some test cases may fall into multiple groups, and you can do that with the syntax below.

@Test(groups=”group_name1”, “group_name2“)

           public void testMethodName(){

                          //Code

           }

In the above code, testMethodName is a part of two groups “group_name1” and “group_name2”.

Code example:

public class TestNGDemo {

          
           @Test(groups = {"smoke"})

           public void test1() {

                          System.out.println("This is Test case 1");

           }
          

           @Test(groups = {"sanity"})

           public void test2() {

                          System.out.println("This is Test case 2");

           }
          

           @Test(groups = {"regression"})

           public void test3() {

                          System.out.println("This is Test case 3");

           }
          

           @Test(groups = {"smoke", "regression"})

           public void test4() {

                          System.out.println("This is Test case 3");

           } 

}

3. Specify the test groups in the TestNG xml file:

Define test groups in the TestNG xml file using the <groups>, <run>, <include> and <exclude> tags. <run> tag defines which groups of tests should be executed during a particular test run. The <include> and <exclude> tags specify which groups to include or exclude from the test execution.

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="TestNGGroup">

  <test thread-count="5" name="Test">

  <groups>

  <run>

  <include name="smoke"></include>

  <exclude name="regression"></exclude>

  </run>

  </groups>

<classes>

   <class name="com.qa.testcases.TestNGDemo"/>

</classes>

  </test> <!-- Test -->

</suite> <!-- TestNGGroup -->

4. Execute the TestNG xml file with the test groups configuration.

After the test groups are configured in the test methods and the TestNG xml configuration is done, run the TestNG xml file and see that the tests which were included will only be executed.

Steps to Use TestNG Groups (With Examples)

TestNG groups allow you to categorize test cases for better management and execution flexibility. You can run test cases within the same group, multiple groups, or even nested groups to organize complex test scenarios.

TestNG also provides options to include or exclude groups during execution, using annotations or XML configuration, and supports regular expressions for advanced grouping and filtering.

1. Steps to Run Test Cases Within the Same Group in TestNG

Below are the steps to Run Test Cases Within the Same Group in TestNG

Step 1: Create test class and add methods and assign the same group to them using @Test annotation.

public class BrowserStack { 

           WebDriver driver = new ChromeDriver();

 

           @Test(priority = 1, description = "Verify BrowserStack page title", groups = { "smoke" })

           public void verifyBSTitle() {

                          driver.get("https://www.browserstack.com/");

                          Assert.assertEquals(driver.getTitle(), "Most Reliable App & Cross Browser Testing Platform | BrowserStack");

           }

 

           @Test(priority = 2, description = "Verify Google page title", groups = { "smoke" })

           public void verifyGoogleTitle() {

                          driver.get("https://www.google.com/");

                          Assert.assertEquals(driver.getTitle(), "Google");

           }

          

           @Test(priority = 3, description = "Verify Get Started for Free click", groups = { "regression" })

           public void clickGetStartedForFree() {

                          driver.get("https://www.browserstack.com/");

            driver.findElement(By.cssSelector("a#signupModalProductButton")).click();

           Assert.assertTrue(driver.getCurrentUrl().contains("users/sign_up"));

           }         

          

           @Test(priority = 4, description = "Verify Pricing click", groups = { "smoke" })

           public void clickPricing() {

                          driver.get("https://www.browserstack.com/");

                      driver.findElement(By.cssSelector("a[title='Pricing']")).click();

                      Assert.assertTrue(driver.getCurrentUrl().contains("pricing"));

           }                                      

}

Step 2: Create a TestNG xml file and define groups which you want to execute.

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="TestNGGroup">

  <test thread-count="5" name="Test">

  <groups>

  <run>

  <include name="smoke"></include>

  </run>

  </groups>

<classes>

   <class name="com.qa.testcases.BrowserStack"/>

</classes>

  </test> <!-- Test -->

</suite> <!-- TestNGGroup -->

Step 3: Run the xml and see the result.

From the above snap, it is evident that only the test methods assigned smoke as a group were executed, and other test methods were ignored.

2. Steps to run test cases within multiple Groups

Executing test methods in the same group is straightforward. In the code example below, you will understand how to execute a test method that is a part of multiple groups. There might be a requirement in your automation project where you would require a single test method to be a part of multiple groups.

For example, Login-Logout should be a part of smoke, sanity, and regression. TestNG groups help in this scenario by assigning multiple groups to a single test method and configuring the TestNG xml file accordingly.

Step 1: Create a test class add methods into it, and assign multiple groups to a single test method.

public class BrowserStack { 

           WebDriver driver = new ChromeDriver();

 

           @Test(priority = 1, description = "Verify BrowserStack page title", groups = { "smoke" , "sanity"})

           public void verifyBSTitle() {

                          driver.get("https://www.browserstack.com/");

                          Assert.assertEquals(driver.getTitle(), "Most Reliable App & Cross Browser Testing Platform | BrowserStack");

           }

 

           @Test(priority = 2, description = "Verify Google page title", groups = { "smoke" })

           public void verifyGoogleTitle() {

                          driver.get("https://www.google.com/");

                          Assert.assertEquals(driver.getTitle(), "Google");

           }

          

           @Test(priority = 3, description = "Verify Get Started for Free click", groups = { "smoke", "regression" })

           public void clickGetStartedForFree() {

                          driver.get("https://www.browserstack.com/");

            driver.findElement(By.cssSelector("a#signupModalProductButton")).click();

           Assert.assertTrue(driver.getCurrentUrl().contains("users/sign_up"));

           }         

          

           @Test(priority = 4, description = "Verify Pricing click", groups = { "sanity" })

           public void clickPricing() {

                          driver.get("https://www.browserstack.com/");

                      driver.findElement(By.cssSelector("a[title='Pricing']")).click();

                      Assert.assertTrue(driver.getCurrentUrl().contains("pricing"));

           }

}

Step 2: Create a TestNG xml file and define the groups that you want to execute.

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="TestNGGroup">

  <test thread-count="5" name="Test">

  <groups>

  <run>

  <include name="smoke"></include>

<include name="regression"></include>

  </run>

  </groups>

<classes>

   <class name="com.qa.testcases.BrowserStack"/>

</classes>

  </test> <!-- Test -->

</suite> <!-- TestNGGroup -->

Step 3: Run the xml file and see the result.

The test methods in group smoke, regression, or both are executed, and the rest of the test methods are not executed.

3. Groups within Groups in TestNG

TestNG provides the flexibility of providing groups inside another group and executing them according to your needs. This may be called nested groups or meta groups.

The code below will help you understand better.

Step 1: Create a test class, add methods to it, and assign groups.

public class BrowserStack { 

           WebDriver driver = new ChromeDriver();

 

           @Test(priority = 1, description = "Verify BrowserStack page title", groups = { "smoke"})

           public void verifyBSTitle() {

                          driver.get("https://www.browserstack.com/");

                          Assert.assertEquals(driver.getTitle(), "Most Reliable App & Cross Browser Testing Platform | BrowserStack");

           }

 

           @Test(priority = 2, description = "Verify Google page title", groups = { "smoke" })

           public void verifyGoogleTitle() {

                          driver.get("https://www.google.com/");

                          Assert.assertEquals(driver.getTitle(), "Google");

           }

          

           @Test(priority = 3, description = "Verify Get Started for Free click", groups = { "regression" })

           public void clickGetStartedForFree() {

                          driver.get("https://www.browserstack.com/");

            driver.findElement(By.cssSelector("a#signupModalProductButton")).click();

           Assert.assertTrue(driver.getCurrentUrl().contains("users/sign_up"));

           }         

          

           @Test(priority = 4, description = "Verify Pricing click", groups = { "regression" })

           public void clickPricing() {

                          driver.get("https://www.browserstack.com/");

                      driver.findElement(By.cssSelector("a[title='Pricing']")).click();

                      Assert.assertTrue(driver.getCurrentUrl().contains("pricing"));

           }

}

Step 2: Create a TestNG xml file and create a super group including all the groups you want to run in an execution.

The xml should look something like below:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="TestNGGroup">

  <test thread-count="5" name="Test">

  <groups>

  <define name= "smoke_group">

           <include name="smoke"></include>

  </define>

  <define name= "regression_group">

           <include name="regression"></include>

  </define>

  <run>

  <include name="smoke_group"></include>

   <include name="regression_group"></include>

  </run>

  </groups>

<classes>

   <class name="com.qa.testcases.BrowserStack"/>

</classes>

  </test> <!-- Test -->

</suite> <!-- TestNGGroup -->

Step 3: Run the xml file and see the result.

You will see that all the test methods defined under the super group will be executed.

4. How to Exclude/Include Test Cases with Groups

With the above examples, you must be aware that to run the test methods falling under any group, you need to use <include> tag.

Similarly, if you need to exclude any group of test cases, you may define it under <exclude> tag. The groups that are excluded will not be executed.

public class BrowserStack { 

           WebDriver driver = new ChromeDriver();

 

           @Test(priority = 1, description = "Verify BrowserStack page title", groups = { "smoke"})

           public void verifyBSTitle() {

                          driver.get("https://www.browserstack.com/");

                          Assert.assertEquals(driver.getTitle(), "Most Reliable App & Cross Browser Testing Platform | BrowserStack");

           }

 

           @Test(priority = 2, description = "Verify Google page title", groups = { "smoke" })

           public void verifyGoogleTitle() {

                          driver.get("https://www.google.com/");

                          Assert.assertEquals(driver.getTitle(), "Google");

           }

          

           @Test(priority = 3, description = "Verify Get Started for Free click", groups = { "smoke", "regression" })

           public void clickGetStartedForFree() {

                          driver.get("https://www.browserstack.com/");

            driver.findElement(By.cssSelector("a#signupModalProductButton")).click();

           Assert.assertTrue(driver.getCurrentUrl().contains("users/sign_up"));

           }         

          

           @Test(priority = 4, description = "Verify Pricing click", groups = { "regression" })

           public void clickPricing() {

                          driver.get("https://www.browserstack.com/");

                      driver.findElement(By.cssSelector("a[title='Pricing']")).click();

                      Assert.assertTrue(driver.getCurrentUrl().contains("pricing"));

           }
 

}

 

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="TestNGGroup">

  <test thread-count="5" name="Test">

  <groups>

           <run>

           <include name="smoke"></include>

           <exclude name="regression"></exclude>

           </run>

  </groups>

<classes>

   <class name="com.qa.testcases.BrowserStack"/>

</classes>

  </test> <!-- Test -->

</suite> <!-- TestNGGroup -->

If you run the above testng.xml file, you will observe that only the test methods under the smoke group will be executed, and all the test methods under regression will not be executed.

This means that even if your test method has smoke and regression groups, it will not be executed as we have instructed the testng.xml file to exclude the regression group.

5. Regular Expressions and TestNG Groups

TestNG provides a powerful way to include and exclude tests dynamically based on group names. You can use “.*,” which states that anything that matches the given pattern in TestNG xml will be executed.

In the below example, test methods belong to groups smoke, sanity, and regression.

public class BrowserStack { 

           WebDriver driver = new ChromeDriver();

 

           @Test(priority = 1, description = "Verify BrowserStack page title", groups = { "smoke"})

           public void verifyBSTitle() {

                          driver.get("https://www.browserstack.com/");

                          Assert.assertEquals(driver.getTitle(), "Most Reliable App & Cross Browser Testing Platform | BrowserStack");

           }

 

           @Test(priority = 2, description = "Verify Google page title", groups = { "sanity" })

           public void verifyGoogleTitle() {

                          driver.get("https://www.google.com/");

                          Assert.assertEquals(driver.getTitle(), "Google");

           }

          

           @Test(priority = 3, description = "Verify Get Started for Free click", groups = { "smoke", "sanity" })

           public void clickGetStartedForFree() {

                          driver.get("https://www.browserstack.com/");

            driver.findElement(By.cssSelector("a#signupModalProductButton")).click();

           Assert.assertTrue(driver.getCurrentUrl().contains("users/sign_up"));

           }         

          

           @Test(priority = 4, description = "Verify Pricing click", groups = { "regression" })

           public void clickPricing() {

                          driver.get("https://www.browserstack.com/");

                      driver.findElement(By.cssSelector("a[title='Pricing']")).click();

                      Assert.assertTrue(driver.getCurrentUrl().contains("pricing"));

           }

}

Create a TestNG xml file to include the groups which start with the letter s using s.*

In this example it means that all the test methods pertaining to groups smoke and sanity will only be executed.

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="TestNGGroup">

  <test thread-count="5" name="Test">

  <groups>

           <run>

           <include name="s.*"></include>

           </run>

  </groups>

<classes>

   <class name="com.qa.testcases.BrowserStack"/>

</classes>

  </test> <!-- Test -->

</suite> <!-- TestNGGroup -->

Group Your TestNG Tests on BrowserStack using Automate

Step 1. Download BrowserStack Project Zip from the GitHub page.

Step 2. Once it is downloaded, unzip it in a desired location in your local system.

Step 3. Import the project in Eclipse via File-> Import -> General -> Projects from Folder or Archive in Eclipse.

Step 4. Once the project is imported, it should have a structure like the one below.

Open the browser.yml file containing all the required capabilities to run the tests on BrowserStack.

Step 5. Set username and password in the browserstack.yml file available at the root directory.

Step 6. You can run the test cases on multiple devices and browser combinations in the BrowserStack cloud. To do so, select the required combinations from the selection list on this page.

Step 7. Copy and replace the platforms object in the browserstack.yml file like below.

platforms:

 - os: Windows

           osVersion: 10

           browserName: Chrome

           browserVersion: latest



  - deviceName: iPhone 13

           osVersion: 15

           browserName: Chromium

           deviceOrientation: portrait

This is for running the test cases on 2 combinations. If you wish to run only on a single device, you may edit the platforms accordingly.

Step 8. In the Eclipse Marketplace, search for BrowserStack > click Install > Finish.

Step 9. Add the above BrowserStack program under src/test/java folder and com.browserstack package. This class should extend SeleniumTest as it has the setup and teardown methods.

public class BrowserStack extends SeleniumTest{

WebDriver driver = new ChromeDriver();

@Test(priority = 1, description = "Verify BrowserStack page title", groups = { "sanity" })

public void verifyBSTitle() {

driver.get("https://www.browserstack.com/");

Assert.assertEquals(driver.getTitle(), "Most Reliable App & Cross Browser Testing Platform | BrowserStack");

}

@Test(priority = 2, description = "Verify Get Started for Free click", groups = { "smoke", "sanity" })

public void clickGetStartedForFree() {

driver.get("https://www.browserstack.com/");

driver.findElement(By.cssSelector("a#signupModalProductButton")).click();

Assert.assertTrue(driver.getCurrentUrl().contains("users/sign_up"));

}

@Test(priority = 3, description = "Verify Pricing click", groups = {"smoke"})

public void clickPricing() {

driver.get("https://www.browserstack.com/");

driver.findElement(By.cssSelector("a[title='Pricing']")).click();

Assert.assertTrue(driver.getCurrentUrl().contains("pricing"));

}

}

Step 10. Run TestNG XML and view the test result on the Automate dashboard.

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="TestNGGroup">

 <test thread-count="5" name="Test">

 <groups>

  <run>

  <include name="smoke*"></include>

  </run>

 </groups>

   <classes>

     <class name="com.qa.testcases.BrowserStack"/>

   </classes>

 </test> <!-- Test -->

</suite> <!-- TestNGGroup -->

After running the testng.xml, you will observe that clickGetStartedForFree and clickPricing test cases will be executed on BrowserStack’s Automate as they are tagged to the “smoke” group.

Talk to an Expert

Conclusion

TestNG is a popular and most widely used testing framework for Selenium automation. It provides ways to write the test cases in proper order and helps with parallel execution and report generation.

If you run TestNG tests on BrowserStack Automate, you will increase scalability and reduce test case maintenance. BrowserStack provides access to a wide range of devices, browsers, and operating systems to test your applications across different environments without maintaining complex infrastructure.