Organize Playwright tests for better reporting

Learn how to organize Playwright tests by using the project, build, and name capabilities on BrowserStack Automate.

You can organize your automated Playwright tests that you run on BrowserStack by ‘projects’ and ‘builds’ by using the project, build, and name capabilities while you start your Playwright test. The following table explains the different capabilities that you can specify:

How to set the project, build and name Capabilities

For every Playwright test that you run on BrowserStack you need to connect to the BrowserStack CDP endpoint using playwright.connect. Within the CDP endpoint URL, we accept a string of capabilities at the end and you need to send all these project, build and name capabilities for your tests using this as shown below:

const { chromium } = require('playwright');

const cp = require('child_process');
const clientPlaywrightVersion = cp.execSync('npx playwright --version').toString().trim().split(' ')[1];

const caps = {
    'project': 'Your project name goes here',
    'build': 'Your specified build name goes here',  
    'name': 'Your specified test name goes here',
    'buildTag': 'Your build tag goes here',
    'browserstack.username': process.env.BROWSERSTACK_USERNAME || 'YOUR_USERNAME',
    'browserstack.accessKey': process.env.BROWSERSTACK_ACCESS_KEY || 'YOUR_ACCESS_KEY',
    'client.playwrightVersion': clientPlaywrightVersion
  };
  const browser = await chromium.connect({
    wsEndpoint: `wss://cdp.browserstack.com/playwright?caps=${encodeURIComponent(JSON.stringify(caps))}`,
  });

As shown in the snippet above, the chromium.connect actually connects to a CDP endpoint at BrowserStack and in that URL, we accept the list of params as specified in the caps JSON object that was defined above.

Best practices for setting the different capabilities

As we have seen in the above code snippet, the project, build, name and buildTag capabilities can be specified to take any value but hard coding all those values in your script is probably not a good idea.

Instead, you can follow the steps mentioned below to ensure that you get the best experience that these capabilities can provide:

1. Define environment variables for project and build names for e.g.

   export PUPPETEER_BUILD_NAME="staging-regression-9-august"
   export PUPPETEER_PROJECT_1="E2E Regression"
   

2. Once you have the environment variables, you can simply populate the build and project capabilities using the values as defined in the environment variables, as shown below:

   const caps = {
       'project': process.env.PLAYWRIGHT_PROJECT_1,
       'build': process.env.PLAYWRIGHT_BUILD_NAME
   }
   

3. The advantage of using environment variables is, you can simply change the value of the environment variable before every build run on BrowserStack and you’d get newer tests organized in new build names every time (see next section to understand the benefits for using build, project and session names)
4. You can hard code the name of each Playwright test because that will most probably not change across different build runs. Also, it is advisable to keep meaningful names of tests so that it becomes easily identifiable in the BrowserStack Dashboard.

Why should you organize your Playwright tests?

We’ve gone through the how and what part of organizing Playwright tests on BrowserStack. Now, we’ll take a look at the why part of it.

Organizing your Playwright tests by project, build, name and buildTag gives you multiple advantages when it comes to debugging your tests on the BrowserStack Dashboard. We take a look at the benefits with the help of examples:

1. Using the project capability would let you filter out only the tests that pertain to that project, in the BrowserStack Dashboard. The following snapshot of the Automate Dashboard shows the filter on the left pane that you can use to apply the project filter.
   
   
2. Using the build capability will organize all test sessions with the same build name, under the same build. You can then access any session in that build simply by clicking on the build in the left pane of Automate Dashboard.
3. Using the name capability will enable you to recognize each individual test in the Automate Dashboard by its name. You also have the status of the test on the side and hence you can quickly understand exactly which test has passed/failed.
4. Using the buildTag capability will enable you to recognize each build in the Automate Dashboard by its assigned tag. You can also filter your builds based on the buildTag.

The below snapshot of a typical Automate Dashboard shows the list of builds in the left pane. On clicking on any build name, the right side shows the list of sessions that are a part of that build (basically all sessions where the same build name was used in capability). You can also see the test name for each session and it is evident from the example that using descriptive test names is beneficial for debugging purposes later on.