App AutomateView and manage test resultsMark tests as passed or failed

Mark tests as passed or failed

Mark your tests as passed or failed from within the test script or after the test has completed.

If you are still using the older dashboard, see the documentation for the old dashboard.

BrowserStack does not know whether your test’s assertions have passed or failed. Therefore, based on your test script’s assertions, you can explicitly inform BrowserStack whether your tests have passed or failed, and the test status can be marked accordingly. This document will help you in updating your test status.

Once you have executed the steps given below, the status and reason are visible on the test session details page of the App Automate dashboard as shown below:

Status reason passed or failed visible on the session details page of App Automate dashboard

You can mark the status of your test along with the reason using the following methods:

From within test script using JavascriptExecutor
After the test has completed using REST API

Mark test status from within test script using JavascriptExecutor

Your test script might contain a lot of assertions and you may choose to mark the status of the test based on any/all assertions, as per your test logic using a JavascriptExecutor during test execution.

The following arguments are required to be passed while using the JavascriptExecutor:

status accepts either passed or failed as the value
reason accepts a value in string datatype. This can be useful to record a reason for test failure.

Use the code snippet below to mark the status of the test using JavascriptExecutor.

JavascriptExecutor jse = (JavascriptExecutor)driver;
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"<passed/failed>\", \"reason\": \"<reason>\"}}");

If you are using BrowserStack SDK, setSessionStatus is managed autonomously by the SDK. However, this is applicable only for language frameworks. For vanilla flavours, Session status needs to be set manually using JavaScript Executor.

Following is a sample test script shows the marking of test status and the associated reason:

W3C Protocol
JSON Wire Protocol

      Java
    

      Node.js
    

      C#
    

      PHP
    

      Python
    

      Ruby
    
package android;

import java.net.URL;
import java.util.List;
import java.net.MalformedURLException;

import io.appium.java_client.MobileBy;
import io.appium.java_client.android.AndroidDriver;
import io.appium.java_client.android.AndroidElement;

import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.openqa.selenium.remote.DesiredCapabilities;


public class BrowserStackSample {

  public static void main(String[] args) throws MalformedURLException, InterruptedException {
    
      DesiredCapabilities capabilities = new DesiredCapabilities();

      capabilities.setCapability("platformName", "android");
      capabilities.setCapability("platformVersion", "9.0");
      capabilities.setCapability("deviceName", "Google Pixel 3");
      capabilities.setCapability("app", "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32");

        AndroidDriver<AndroidElement> driver = new AndroidDriver<AndroidElement>(
            new URL("https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub.browserstack.com/wd/hub"), capabilities);

        JavascriptExecutor jse = (JavascriptExecutor)driver;
        

        AndroidElement searchElement = (AndroidElement) new WebDriverWait(driver, 30).until(
            ExpectedConditions.elementToBeClickable(
                MobileBy.AccessibilityId("Search Wikipedia")));
        searchElement.click();
    AndroidElement insertTextElement = (AndroidElement) new WebDriverWait(driver, 30).until(
             ExpectedConditions.elementToBeClickable(
                 MobileBy.id("org.wikipedia.alpha:id/search_src_text")));
        insertTextElement.sendKeys("BrowserStack");
        Thread.sleep(5000);
        List<AndroidElement> allProductsName = driver.findElementsByClassName(
            "android.widget.TextView");

    // Setting the status of test as 'passed' or 'failed' based on the condition: if results are found for the search
    if (allProductsName.size() > 0) {
      jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"passed\", \"reason\": \"Results found!\"}}");
    }
    else {
      jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \"Results not found\"}}");
    }

        driver.quit();
    
  }

}
let wd = require('wd');
let assert = require('assert');
let asserters = wd.asserters;

var Caps = {
  "platformName" : "android",
  "platformVersion" : "9.0",
  "deviceName" : "Google Pixel 3",
  "app" : "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32",
};

driver = wd.promiseRemote("https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub");

driver.init(desiredCaps)
  .then(function () {
    return driver.waitForElementByAccessibilityId(
      'Search Wikipedia', asserters.isDisplayed 
      && asserters.isEnabled, 30000);
  })
  .then(function (searchElement) {
    return searchElement.click();
  })
  .then(function () {
    return driver.waitForElementById(
      'org.wikipedia.alpha:id/search_src_text', asserters.isDisplayed 
      && asserters.isEnabled, 30000);
  })
  .then(function (searchInput) {
    return searchInput.sendKeys("BrowserStack");
  })
  .then(function () {
    return driver.elementsByClassName('android.widget.TextView');    
  })
  .then(function (search_results) {
 
    // Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
    if(search_results.length > 0){
      return driver.execute('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed","reason": "Results found!"}}');
    } else {
      driver.execute('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed","reason": "No results available!"}}');
    
     }
  })
  .fin(function() { 
    return driver.quit(); 
  })
  .done();
using System;
using OpenQA.Selenium.Appium;
using OpenQA.Selenium.Appium.Android;
using OpenQA.Selenium.Remote;
using OpenQA.Selenium.Support.UI;
using System.Collections.Generic;
using OpenQA.Selenium;

namespace csharp_appium_first_test_browserstack
{
    class Program
    {
        static void Main(string[] args)
        {
            AppiumOptions capabilities = new AppiumOptions();

            // Set URL of the application under test
            capabilities.AddAdditionalCapability("bstack:options", browserstackOptions);
            capabilities.AddAdditionalCapability("platformName", "android");
            capabilities.AddAdditionalCapability("platformVersion", "9.0");
            capabilities.AddAdditionalCapability("appium:deviceName", "Google Pixel 3");
            capabilities.AddAdditionalCapability("appium:app", "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32");

            AndroidDriver<AndroidElement> driver = new AndroidDriver<AndroidElement>(
                    new Uri("https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub"), capabilities);

            AndroidElement searchElement = (AndroidElement)new WebDriverWait(
                driver, TimeSpan.FromSeconds(30)).Until(
                SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(
                    MobileBy.AccessibilityId("Search Wikipedia"))
            );
            searchElement.Click();
            AndroidElement insertTextElement = (AndroidElement)new WebDriverWait(
                driver, TimeSpan.FromSeconds(30)).Until(
                SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(
                    MobileBy.Id("org.wikipedia.alpha:id/search_src_text"))
            );
            insertTextElement.SendKeys("BrowserStack");
            System.Threading.Thread.Sleep(5000);

            IReadOnlyList<AndroidElement> allTextViewElements =
                driver.FindElementsByClassName("android.widget.TextView");

            // Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search

            if ((allTextViewElements.Count > 0))
            {
                ((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"passed\", \"reason\": \" Results found! \"}}");
            }
            else
            {
                ((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \" Results not found!\"}}");
            }

            driver.Quit();
        }
    }
}
<?php
  require_once('vendor/autoload.php');
    use Facebook\WebDriver\Remote\DesiredCapabilities;
    use Facebook\WebDriver\Remote\RemoteWebDriver;
    use Facebook\WebDriver\WebDriverBy;

    $caps = array(
      "platformName" => "android",
      "platformVersion" => "9.0",
      "deviceName" => "Google Pixel 3",
      "app" => "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32",
    )

    $driver = RemoteWebDriver::create("https://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub-cloud.browserstack.com/wd/hub", $caps);


  
    $driver->getTitle();

  # Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'Wikipedia'
  if ($driver->getTitle() == "Wikipedia"){
      $driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}' );
  }  else {
      $driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched!"}}');
  }

  $driver->quit();
?>
from appium import webdriver
from appium.webdriver.common.mobileby import MobileBy
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

desired_cap = {
    "platformName" : "android",
    "platformVersion" : "9.0",
    "deviceName" : "Google Pixel 3",
    "app" : "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32",
}

driver = webdriver.Remote(
    command_executor="https://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub-cloud.browserstack.com/wd/hub", 
    desired_capabilities=desired_cap
)


search_element = WebDriverWait(driver, 30).until(
    EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "Search Wikipedia"))
)
search_element.click()
search_input = WebDriverWait(driver, 30).until(
    EC.element_to_be_clickable((MobileBy.ID, "org.wikipedia.alpha:id/search_src_text"))
)
search_input.send_keys("BrowserStack")
time.sleep(5)

# Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
search_results = driver.find_elements_by_class_name("android.widget.TextView")
if (len(search_results) > 0):
	driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Results found!"}}')
else:
	driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "No results found"}}')
driver.quit()
require 'rubygems'
require 'appium_lib'


caps = {
    "platformName" => "android",
    "platformVersion" => "9.0",
    "deviceName" => "Google Pixel 3",
    "app" => "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32",
}

appium_driver = Appium::Driver.new({
    'caps' => caps,
    'appium_lib' => {
        :server_url => "https://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub-cloud.browserstack.com/wd/hub"
    }}, true)
driver = appium_driver.start_driver

wait = Selenium::WebDriver::Wait.new(:timeout => 30)
wait.until { driver.find_element(:accessibility_id, "Search Wikipedia").displayed? }
element = driver.find_element(:accessibility_id, "Search Wikipedia")
element.click

wait.until { driver.find_element(:id, "org.wikipedia.alpha:id/search_src_text").displayed? }
search_box = driver.find_element(:id, "org.wikipedia.alpha:id/search_src_text")
search_box.send_keys("BrowserStack")
sleep 5

# Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search

results = driver.find_elements(:class, "android.widget.TextView")
if results.count > 0
    driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Results found"}}')
else
   driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "No results found"}}')
end

driver.quit

Mark test status after test has completed using REST API

You can also mark a test session’s status as passed or failed using our REST API. This is particularly useful when a test successfully completed on BrowserStack (i.e. Completed status) but there was a failed assertion in the test case.

The following arguments need to be passed in the REST API call for setting the status, and its corresponding reason:

status accepts either passed or failed as the value
reason accepts a value in string datatype. This can be useful to record a reason for test failure.

Here is a REST API request to update the status and reason for an example test session :

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X PUT "https://api-cloud.browserstack.com/app-automate/sessions/f97f02bf39d592f5fc349cee419294fdfb7593a2.json" \
-H "Content-Type: application/json" \
-d '{"status":"failed", "reason":"Element not found on the login page"}'

Character Limits

A maximum of 256 characters can be passed as input value for reason argument.
Escape sequences such as \n, \t, \b, \", \r are not accepted as valid input for reason argument.

Check this page for a list of various JavaScript Executors that BrowserStack offers.

Did this page help you?

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked

This page has exactly what I am looking for
This content & code samples are accurate and up-to-date
The content on this page is easy to understand
Other (please tell us more below)

Any additional feedback?

Thank you for your valuable feedback

Is this page helping you?

Yes

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked

This page has exactly what I am looking for
This content & code samples are accurate and up-to-date
The content on this page is easy to understand
Other (please tell us more below)

Any additional feedback?

Thank you for your valuable feedback!