Mark status of your Playwright tests

Learn how to mark the status of your Playwright tests as passed or failed based on your test assertions

Why marking the status of test is a separate activity?

Your Playwright tests are run on BrowserStack using a client-server architecture where the script stays in your system and chromium.connect method is used over a WebSocket to connect to a CDP endpoint on BrowserStack, which in-turn gives you a browser instance.

Due to the architecture as explained above, there is no way for BrowserStack to know if your test assertions have passed/failed because the assertions run on the client side i.e. your system. Hence, for you to be able to see the status of the tests as passed or failed on the BrowserStack Dashboard, your test script must send an indication stating the status of the test. This document explains the methods that you can use to do the same.

Mark status of test from within the test script

At any point in your Playwright script, you can use the following page.evaluate and that would simply set the status of your test on BrowserStack.

page.evaluate(_ => {}, `browserstack_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: {status: '<passed/failed>',reason: '<the string reason goes here>'}})}`);

You have to simply pass either passed or failed in the value for the status key. You can optionally specify a reason for the same as well. It is advisable to make this call from an after hook where you would have the assertion status for the test present.

Mark the status of your Playwright test using REST API during/after the test script run

You may also choose to internally maintain the statuses of all the tests in your test suite and mark the statuses on BrowserStack only after your entire test suite has completed. You can achieve the same using our REST API for marking test status.

Every test session that runs on BrowserStack has a unique session ID and that ID is required to mark the status of the test using our REST API. You can fetch the session ID for your test session(s) using the following snippet within your test:

const resp = await JSON.parse(await page.evaluate(_ => {}, `browserstack_executor: ${JSON.stringify({action: 'getSessionDetails'})}`));
console.log(resp.hashed_id);  // This gives the session ID of the running session

Now, that you have the hashed_id (i.e. Session ID), you can use the following REST API call to set the test status:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X PUT -H "Content-Type: application/json" \
-d "{\"status\":\"<passed/failed>\", \"reason\":\"<reason text>\"}" https://api.browserstack.com/automate/sessions/<session-id>.json

The above API can be invoked at any time after the session has completed. You can learn more about the API.