AutomateView and manage test resultsMark tests as pass/fail

Mark tests as passed or failed

Mark your Automate tests as passed or failed from within the test script or after the test has completed.

You are currently viewing the documentation for the old dashboard. We’ve revamped the dashboard experience. Explore the new dashboard now!

BrowserStack cannot identify if your test’s assertions have passed or failed. Therefore, based on the assertions in your test script, you have to explicitly instruct BrowserStack whether your tests have passed or failed.

This guide will provide detailed information on how to mark your tests as passed or failed.

Once you have completed your test run, you will be able to view the status of your tests on the Automate Dashboard as shown below:

Mark tests as passed or failed

You can mark the status of your test along with the reason using the following methods:

During the test using JavascriptExecutor
After the test has completed using REST API

You can also edit the test status and reason from the Automate Dashboard, check out how to edit your test session details.

Mark test status from the test script using JavascriptExecutor

Your test script can contain many assertions. You may choose to mark the status of the test based on these assertions using JavascriptExecutor.

Used the following arguments in the JavaScript method to set the status and the corresponding reason for the test:

Argument	Description	Accepted values
`status`	Status of the test	A string. `passed` or `failed`
`reason`	Test Pass or Fail reason	A string.

Use the following code snippet to set the status of the test:

final JavascriptExecutor jse = (JavascriptExecutor) driver;
JSONObject executorObject = new JSONObject();
JSONObject argumentsObject = new JSONObject();
argumentsObject.put("status", "<passed/failed>");
argumentsObject.put("reason", "<reason>");
executorObject.put("action", "setSessionStatus");
executorObject.put("arguments", argumentsObject);
jse.executeScript(String.format("browserstack_executor: %s", executorObject));

If you are using BrowserStack SDK, setSessionStatus is managed autonomously by the SDK. However, this is applicable only for language frameworks. For vanilla flavours, Session status needs to be set manually using JavaScript Executor.

The following sample test script shows the test status and the associated reason:

Selenium 4 W3C
Selenium Legacy JSON

      Java
    

      Node.js
    

      C#
    

      PHP
    

      Python
    

      Ruby
    
package bs_automate;
import java.net.URL;
import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.MutableCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;

public class bs_automate_class {
  public static final String USERNAME = "YOUR_USERNAME";
  public static final String AUTOMATE_KEY = "YOUR_ACCESS_KEY";
  public static final String URL = "https://" + USERNAME + ":" + AUTOMATE_KEY + "@hub-cloud.browserstack.com/wd/hub";
  public static void main(String[] args) throws Exception {

    // Input capabilities
    MutableCapabilities capabilities = new MutableCapabilities();
    capabilities.setCapability("browserName", "Chrome");
    capabilities.setCapability("browserVersion", "103.0");
    HashMap<String, Object> browserstackOptions = new HashMap<String, Object>();
    browserstackOptions.put("os", "Windows");
    browserstackOptions.put("osVersion", "10");
    browserstackOptions.put("resolution", "1920x1080");
    browserstackOptions.put("projectName", "Mark test as pass fail using JS Executor"); // test name
    browserstackOptions.put("buildName", "Sample Build"); // CI/CD job or build name
    browserstackOptions.put("seleniumVersion", "4.0.0");
    capabilities.setCapability("bstack:options", browserstackOptions);


    // Searching for 'BrowserStack' on google.com
    driver.get("https://www.google.com");
    WebElement element = driver.findElement(By.name("q"));
    element.sendKeys("BrowserStack");
    element.submit();
    System.out.println(driver.getTitle());

    // Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
    if (driver.getTitle().equals("BrowserStack - Google Search")) {
      jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"passed\", \"reason\": \"Title matched!\"}}");
    }
    else {
      jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \"Title not matched\"}}");
    }

    driver.quit();
  }
}
var webdriver = require('selenium-webdriver');

// Input capabilities
var capabilities = {
	'bstack:options' : {
		"os" : "Windows",
		"osVersion" : "10",
		"resolution" : "1920x1080",
		"projectName" : "Mark test as pass fail using JS Executor", // test name
		"buildName" : "Sample Build", // CI/CD job or build name
		"seleniumVersion" : "4.0.0",
		"userName" : "YOUR_USERNAME",
		"accessKey" : "YOUR_ACCESS_KEY",
	},
	"browserName" : "Chrome",
	"browserVersion" : "103.0",
}

var driver = new webdriver.Builder().
  usingServer('https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub').
  withCapabilities(capabilities).
  build();
  
// Searching for 'BrowserStack' on google.com
driver.get('https://www.google.com').then(function(){
  driver.findElement(webdriver.By.name('q')).sendKeys('BrowserStack\n').then(function(){
    driver.getTitle().then(function(title) {
      console.log(title);

      // Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
      if(title === "BrowserStack - Google Search"){
      	driver.executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed","reason": "Title matched!"}}');
      } else {
      	driver.executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed","reason": "Title not matched!"}}');
      }
	  driver.quit();
    });
  });
});
using System;
using OpenQA.Selenium;
using OpenQA.Selenium.Remote;
namespace cs_testing
{
    class Program
    {
        static void Main(string[] args)
        {
            IWebDriver driver;

            // Input capabilities
            ChromeOptions capabilities = new ChromeOptions();
            capabilities.BrowserVersion = "103.0";
            Dictionary<string, object> browserstackOptions = new Dictionary<string, object>();
            browserstackOptions.Add("os", "Windows");
            browserstackOptions.Add("osVersion", "10");
            browserstackOptions.Add("resolution", "1920x1080");
            browserstackOptions.Add("projectName", "Mark test as pass fail using JS Executor"); // test name
            browserstackOptions.Add("buildName", "Sample Build"); // CI/CD job or build name
            browserstackOptions.Add("seleniumVersion", "4.0.0");
            browserstackOptions.Add("userName", "YOUR_USERNAME");
            browserstackOptions.Add("accessKey", "YOUR_ACCESS_KEY");
            capabilities.AddAdditionalOption("bstack:options", browserstackOptions);


            // Searching for 'BrowserStack' on google.com
            driver.Navigate().GoToUrl("https://www.google.com");
            IWebElement query = driver.FindElement(By.Name("q"));
            query.SendKeys("BrowserStack");
            query.Submit();
            Console.WriteLine(driver.Title);

            // Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
            string str = "BrowserStack - Google Search";
            if (string.Equals(driver.Title, str))
            {
                ((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"passed\", \"reason\": \" Title matched!\"}}");
            }
            else
            {
                ((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \" Title not matched \"}}");
            }
            driver.Quit();
        }
    }
}
<?php
  require_once('vendor/autoload.php');
  use Facebook\WebDriver\Remote\RemoteWebDriver;
  use Facebook\WebDriver\WebDriverBy;

  # Input capabilities
$caps = array(
	'bstack:options' => array(
		"os" => "Windows",
		"osVersion" => "10",
		"resolution" => "1920x1080",
		"projectName" => "Mark test as pass fail using JS Executor", #test name
		"buildName" => "Sample Build", #CI/CD job or build name
		"seleniumVersion" => "4.0.0",
	),
	"browserName" => "Chrome",
	"browserVersion" => "103.0",
)

  $web_driver = RemoteWebDriver::create("https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub", $caps);
  
  # Searching for 'BrowserStack' on google.com
  $web_driver->get("https://google.com");
  $element = $web_driver->findElement(WebDriverBy::name("q"));
  if($element) {
      $element->sendKeys("BrowserStack");
      $element->submit();
  }
  print $web_driver->getTitle();

  # Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
  if ($web_driver->getTitle() == "BrowserStack - Google Search"){
      $web_driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}' );
  }  else {
      $web_driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched!"}}');
  }

  $web_driver->quit();
?>
import requests
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

# Input capabilities
desired_cap = {
	'bstack:options' : {
		"os" : "Windows",
		"osVersion" : "10",
		"resolution" : "1920x1080",
		"projectName" : "Mark test as pass fail using JS Executor", #test name
		"buildName" : "Sample Build", #CI/CD job or build name
		"seleniumVersion" : "4.0.0",
	},
	"browserName" : "Chrome",
	"browserVersion" : "103.0",
}
driver = webdriver.Remote(
    command_executor='https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub',
    desired_capabilities=desired_cap)
    
# Searching for 'BrowserStack' on google.com
driver.get("https://www.google.com")
if not "Google" in driver.title:
    raise Exception("Unable to load google page!")
elem = driver.find_element_by_name("q")
elem.send_keys("BrowserStack")
elem.submit()
print(driver.title)

# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if (driver.title=="BrowserStack - Google Search"):
	driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}')
else:
	driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched"}}')
driver.quit()
require "rubygems"
require "selenium-webdriver"

# Input capabilities
capabilities = {
	'bstack:options' => {
		"os" => "Windows",
		"osVersion" => "10",
		"resolution" => "1920x1080",
		"projectName" => "Mark test as pass fail using JS Executor", #test name
		"buildName" => "Sample Build", #CI/CD job or build name
		"seleniumVersion" => "4.0.0",
	},
	"browserName" => "Chrome",
	"browserVersion" => "103.0",
}

driver = Selenium::WebDriver.for(:remote,
  :url => "https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub",
  :desired_capabilities => caps)

# Searching for 'BrowserStack' on google.com
driver.navigate.to "https://www.google.com"
element = driver.find_element(:name, "q")
element.send_keys "BrowserStack"
element.submit

puts driver.title

# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if driver.title=="BrowserStack - Google Search"
  driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}')
else
  driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched"}}')
end

driver.quit

Mark test status after test completion using REST API

You can also mark your test as as passed or failed on BrowserStack after the test script has completed its run, however, you will not be able to follow the above-mentioned approach. In such cases, it is recommended that you mark your tests using our REST API endpoint to mark the status of test as passed or failed. You can also provide a reason to explain why you’re marking a test as failed (or passed).

Use the following arguments in the REST API call to set the test status, and the corresponding reason:

Argument	Description	Accepted values
`status`	Status of the test	A string. `passed` or `failed`
`reason`	Test Pass or Fail reason	A string.

The following REST API can be used to achieve this:

Command Line

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" -X PUT -H "Content-Type: application/json" \
-d "{\"status\":\"<new-status>\", \"reason\":\"<reason text>\"}" \
 https://api.browserstack.com/automate/sessions/<session-id>.json

Read more about how to fetch the <session-id> of your session which you want to mark and use it in the REST API call.

Check this page for a list of various JavaScript Executors that BrowserStack offers.

Did this page help you?

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked

This page has exactly what I am looking for
This content & code samples are accurate and up-to-date
The content on this page is easy to understand
Other (please tell us more below)

Any additional feedback?

Thank you for your valuable feedback

Is this page helping you?

Yes

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked

This page has exactly what I am looking for
This content & code samples are accurate and up-to-date
The content on this page is easy to understand
Other (please tell us more below)

Any additional feedback?

Thank you for your valuable feedback!