Assert and Verify Methods in Selenium

Selenium is a robust browser automation tool. However, it doesn’t have powerful assertion methods. Assertions are necessary to verify test results against expected outcomes. So, they are vital for test automation. In Selenium, since there is no built-in assert method. The workaround to perform assertions in Selenium is to use a testing framework such as TestNG, JUnit or Python’s unittest.

In this tutorial, we will learn the difference between assert and verify and the why, when, and how of using these methods to make Selenium testing more efficient.

Difference between Assert and Verify in Selenium

• In the case of assertions, if the assert condition is not met, test case execution will be aborted. The remaining tests are skipped, and the test case is marked as failed. These assertions are used as checkpoints for testing or validating business-critical transactions.
• Tests will continue to run in case of verification until the last test is executed, even if assert conditions are not met.
• Verify or Soft Asserts will report the errors at the end of the test. Simply put, tests will not be aborted if any condition is not met.
• Testers need to invoke the assertAll() method to view the results.

Assertions (also known as Asserts)

The word Assert means to state a fact or belief confidently or forcefully. In Selenium, Asserts are validations or checkpoints for an application. Assertions state confidently that application behavior is working as expected. Asserts in Selenium validate the automated test cases that help testers understand if tests have passed or failed.

Types of Assertions

• Hard Assertions
• Soft Assertions (Verify Method)

Hard Asserts vs Soft Asserts

Hard Assertions

Hard Assertions are ones in which test execution is aborted if the test does not meet the assertion condition. The test case is marked as failed. In case of an assertion error, it will throw the “java.lang.AssertionError” exception.

Let’s explore different types of hard assertions with examples.

assertEquals() is a method that takes a minimum of 2 arguments and compares actual results with expected results. If both match, the assertion is passed, and the test case is marked as passed. assertEquals() can compare Strings, Integers, Doubles, and many more variables, as shown in the image below.

Below is an example of assertEquals().

package com.tests;



import static org.testng.Assert.assertEquals;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import org.testng.annotations.Test;



public class BrowserStackTutorials {

    @Test

    public void testAssertEquals() {

        System.setProperty("webdriver.chrome.driver", "C:\\I2EWebsiteTest\\Driver\\chromedriver.exe");

        WebDriver driver = new ChromeDriver();

        driver.navigate().to("https://www.browserstack.com/");

        String actualTitle = driver.getTitle();

        String expectedTitle = "Most Reliable App & Cross Browser Testing Platform | BrowserStack";

        assertEquals(actualTitle, expectedTitle, "Page title does not match the expected value");

        driver.quit();

    }

}

assertEquals(actualTitle, expectedTitle, “Page title does not match the expected value”);:

• This assertion compares the actual title (actualTitle) retrieved from the webpage with the expected title (expectedTitle).
• If the two titles match, the test case proceeds.
• If the titles do not match, the assertion fails and halts further execution of the test case.
• The third parameter, “Page title does not match the expected value“, is a custom error message displayed when the assertion fails.

assertNotEquals()

assertNotEquals() is a method that does the opposite of the assertEquals() method. In this case, the method compares the actual and expected result. But if the assertion condition is met if the two are not identical. The test case is marked as passed if actual and expected results are not the same.

Code Snippet for assertNotEquals() in Selenium

package com.tests;



import static org.testng.Assert.assertNotEquals;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import org.testng.annotations.Test;



public class BrowserStackTutorials {

    @Test

    public void testAssertNotEquals() {

        System.setProperty("webdriver.chrome.driver", "C:\\I2EWebsiteTest\\Driver\\chromedriver.exe");

        WebDriver driver = new ChromeDriver();

        driver.navigate().to("https://www.browserstack.com/");

        String actualTitle = driver.getTitle();

        String expectedTitle = "Incorrect Title";

        assertNotEquals(actualTitle, expectedTitle, "Titles should not match, but they do");

        driver.quit();

    }

}

assertNotEquals(actualTitle, expectedTitle, “Titles should not match, but they do”);:

• This assertion checks that the actual title (actualTitle) retrieved from the webpage does not match the expected title (expectedTitle).
• If the titles are different, the assertion passes, and the test continues.
• If the titles match, the assertion fails, and the custom error message “Titles should not match, but they do” is displayed.

assertTrue()

This Assertion verifies the Boolean value returned by the condition. If the Boolean value is true, then the assertion passes the test case.

Code Snippet for assertTrue() in Selenium

package com.tests;



import static org.testng.Assert.assertTrue;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import org.testng.annotations.Test;



public class BrowserStackTutorials {

    @Test

    public void testAssertTrue() {

        System.setProperty("webdriver.chrome.driver", "C:\\I2EWebsiteTest\\Driver\\chromedriver.exe");

        WebDriver driver = new ChromeDriver();

        driver.navigate().to("https://www.browserstack.com/");

        boolean verifyTitle = driver.getTitle().equalsIgnoreCase("Most Reliable App & Cross Browser Testing Platform | BrowserStack");

        assertTrue(verifyTitle, "The page title is not as expected");

        driver.quit();

    }

}

assertTrue(verifyTitle, “The page title is not as expected”);:

• This assertion checks whether the condition in verifyTitle evaluates to true.
• verifyTitle is a boolean variable that compares the actual title of the webpage with the expected title in a case-insensitive manner.
• If verifyTitle is true, the assertion passes, and the test proceeds.
• If verifyTitle is false, the assertion fails, and the custom error message “The page title is not as expected” is displayed.

assertFalse()

This method works the opposite of assertTrue(). The Assertion verifies the Boolean value returned by the condition. If the Boolean value is false, the assertion passes the test case.

Code Snippet for assertFalse() in Selenium

package com.tests;



import static org.testng.Assert.assertFalse;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import org.testng.annotations.Test;



public class BrowserStackTutorials {

    @Test

    public void testAssertFalse() {

        System.setProperty("webdriver.chrome.driver", "C:\\I2EWebsiteTest\\Driver\\chromedriver.exe");

        WebDriver driver = new ChromeDriver();

        driver.navigate().to("https://www.browserstack.com/");

        boolean verifyTitle = driver.getTitle().equalsIgnoreCase("Incorrect Title");

        assertFalse(verifyTitle, "The title matched an incorrect value");

        driver.quit();

    }

}

assertFalse(verifyTitle, “The title matched an incorrect value”);:

• This assertion checks whether the condition in verifyTitle evaluates to false.
• verifyTitle is a boolean variable that compares the actual title of the webpage with an incorrect title (“Incorrect Title“) in a case-insensitive manner.
• If verifyTitle is false, meaning the actual title does not match the incorrect title, the assertion passes, and the test continues.
• If verifyTitle is true, meaning the actual title matches the incorrect title, the assertion fails, and the custom error message “The title matched an incorrect value” is displayed.

Code Tip: Want to run a quick test of the above code on our cloud infrastructure?

Talk to an Expert

assertNull()

This method verifies if the expected output is null. If not, the value returned is false.

Code Snippet for assertNull() in Selenium

package com.tests;



import static org.testng.Assert.assertNull;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import org.testng.annotations.Test;



public class BrowserStackTutorials {

    @Test

    public void testAssertNull() {

        System.setProperty("webdriver.chrome.driver", "C:\\I2EWebsiteTest\\Driver\\chromedriver.exe");

        WebDriver driver = new ChromeDriver();

        String verifyAssertNull = null;

        assertNull(verifyAssertNull, "Expected value is not null");

        driver.quit();

    }

}

assertNull(verifyAssertNull, “Expected value is not null”);:

• This assertion checks if the value of the variable verifyAssertNull is null.
• In this case, verifyAssertNull is explicitly set to null.
• If verifyAssertNull is indeed null, the assertion passes, and the test continues.
• If verifyAssertNull is not null, the assertion fails, and the custom error message “Expected value is not null” is displayed.

assertNotNull()

This method works opposite to the assertNull() method. The assertion condition is met when the method validates the expected output to be not null.

Code Snippet for assertNotNull() in Selenium

package com.tests;



import static org.testng.Assert.assertNotNull;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import org.testng.annotations.Test;



public class BrowserStackTutorials {

    @Test

    public void testAssertNotNull() {

        System.setProperty("webdriver.chrome.driver", "C:\\I2EWebsiteTest\\Driver\\chromedriver.exe");

        WebDriver driver = new ChromeDriver();

        driver.navigate().to("https://www.browserstack.com/");

        String actualTitle = driver.getTitle();

        assertNotNull(actualTitle, "Page title should not be null");

        driver.quit();

    }

}

assertNotNull(actualTitle, “Page title should not be null”);:

• This assertion verifies that the value of the actualTitle variable is not null.
• The actualTitle is obtained by calling driver.getTitle(), which retrieves the title of the webpage at https://www.browserstack.com/.
• If actualTitle is not null, the assertion passes, and the test continues.
• If actualTitle is null (i.e., if the page did not load or no title was retrieved), the assertion fails, and the message “Page title should not be null” is displayed.

Soft Assertions

In a hard assertion, when the assertion fails, it terminates or aborts the test. If the tester does not want to terminate the script, they cannot use hard assertions. To overcome this, one can use soft assertions.

Code Snippet for Soft Assertions in Selenium

package com.tests;



import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import org.testng.annotations.Test;

import org.testng.asserts.SoftAssert;



public class BrowserStackTutorials {

    @Test

    public void testSoftAssert() {

        System.setProperty("webdriver.chrome.driver", "C:\\I2EWebsiteTest\\Driver\\chromedriver.exe");

        WebDriver driver = new ChromeDriver();

        SoftAssert softAssert = new SoftAssert();

        driver.navigate().to("https://www.browserstack.com/");

        String actualTitle = driver.getTitle();

        softAssert.assertEquals(actualTitle, "Most Reliable App & Cross Browser Testing Platform | BrowserStack", "Title does not match");

        softAssert.assertNotEquals(actualTitle, "Incorrect Title", "Title matches an incorrect value");

        softAssert.assertNotNull(actualTitle, "Page title should not be null");

        boolean titleMatch = actualTitle.equalsIgnoreCase("Most Reliable App & Cross Browser Testing Platform | BrowserStack");

        softAssert.assertTrue(titleMatch, "Title does not match in case-insensitive comparison");

        softAssert.assertAll();  // Reports all assertion results

        driver.quit();

    }

}

• SoftAssert softAssert = new SoftAssert(); creates an instance of SoftAssert to perform soft assertions.
• softAssert.assertEquals(…): Verifies if the actual title matches the expected title. Fails with a custom message if the condition is false.
• softAssert.assertNotEquals(…): Ensures that the actual title does not match an incorrect value.
• softAssert.assertNotNull(…): Confirms that the retrieved title is not null.
• softAssert.assertTrue(…): Validates that the title matches the expected value in a case-insensitive manner.
• softAssert.assertAll(): Collects and reports all the assertion results. Without this line, failed assertions would not be reported.

Implementing Soft Assertions Without a Library

It is possible to implement soft assertions without relying on libraries like TestNG or JUnit. The idea here is to manually capture assertion results during test execution to report all of them at the end. Now, even after an assertion fails, you must continue test execution. At the end of the test, check all the assertions and report them.

Here is how soft assertions can be implemented manually:

• Create a collection to store assertion results: This could be a list of custom assertion results that track whether each assertion passed or failed.
• Perform assertions and collect results: Instead of throwing an exception immediately when an assertion fails, store the result (pass or fail) and continue the test.
• At the end of the test, report all assertions: After the test has finished executing, iterate through the collection and report all assertion results.

Example of Manual Soft Assertion Implementation

package com.tests;



import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import org.testng.annotations.Test;

import java.util.ArrayList;

import java.util.List;



public class BrowserStackTutorials {



    // Custom class to hold assertion result information

    static class AssertionResult {

        boolean passed;

        String message;

        
        AssertionResult(boolean passed, String message) {

            this.passed = passed;

            this.message = message;

        }

    }



    @Test

    public void testManualSoftAssert() {

        System.setProperty("webdriver.chrome.driver", "C:\\I2EWebsiteTest\\Driver\\chromedriver.exe");

        WebDriver driver = new ChromeDriver();

        driver.navigate().to("https://www.browserstack.com/");
        

        // List to collect assertion results

        List<AssertionResult> assertionResults = new ArrayList<>();



        // Manually perform assertions and collect results

        String actualTitle = driver.getTitle();

        String expectedTitle = "Most Reliable App & Cross Browser Testing Platform | BrowserStack";



        // Check if title is as expected (assertion)

        if (actualTitle.equals(expectedTitle)) {

            assertionResults.add(new AssertionResult(true, "Title matches as expected"));

        } else {

            assertionResults.add(new AssertionResult(false, "Title does not match the expected value"));

        }



        // Check if title is not null (assertion)

        if (actualTitle != null) {

            assertionResults.add(new AssertionResult(true, "Title is not null"));

        } else {

            assertionResults.add(new AssertionResult(false, "Title is null"));

        }



        // Additional checks can be added as needed

        boolean isTitleContainsKeyword = actualTitle.contains("BrowserStack");

        if (isTitleContainsKeyword) {

            assertionResults.add(new AssertionResult(true, "Title contains 'BrowserStack'"));

        } else {

            assertionResults.add(new AssertionResult(false, "Title does not contain 'BrowserStack'"));

        }



        // At the end of the test, report all assertion results

        for (AssertionResult result : assertionResults) {

            if (result.passed) {

                System.out.println("PASSED: " + result.message);

            } else {

                System.out.println("FAILED: " + result.message);

            }

        }



        driver.quit();

    }

}

• Custom AssertionResult class: This class stores whether the assertion passed or failed and the associated message.
• Storing results in a list: For each assertion, we create an AssertionResult and add it to the assertionResults list.
• No immediate test failure: Instead of throwing an exception when an assertion fails, we continue testing and store the results.
• Report assertions at the end: After all assertions have been executed, we loop through the collected results and print whether each assertion passed or failed.

Here, you must manually implement each assertion and result collection, which increases code complexity. Also, if you forget to check the results at the end, the failed assertions may not stop the test. So, review the results carefully.

Conclusion

• Hard and Soft Assertions are very important for designing and running Selenium Webdriver tests.
• They are instrumental in verifying application behavior at critical stages.
• By using assertions, testing teams can determine if an application is working as expected.
• They can also save teams the trouble of running tests that don’t need to be run if a condition is not met.

It is recommended to run tests on real devices for better accuracy as it takes real user conditions into account. BrowserStack gives you access to 3000+ real devices and browsers to test on. It allows you to run selenium browser automation tests on multiple browser-device combinations simultaneously using parallel testing on its Cloud Selenium Grid.

Try BrowserStack Now