Run Cucumber-based Espresso tests

App Automate supports running your Cucumber-based Espresso test. The cucumberOptions JSON object allows your Cucumber-based Espresso tests to run similar to any other Espresso test.

In this guide, you’ll learn how to:

• Filter and run test cases
• Generate test reports
• Use test sharding

Filter test cases

By default, all your test cases in your Cucumber-based Espresso test suite are executed on BrowserStack. However, you can filter test cases using the cucumberOptions JSON object to:

• Run a quick smoke test to ensure the sanity of critical functionality.
• Run only those test cases that are relevant to a new bug fix.
• Split a large test suite into multiple shards and run them in parallel.

You can filter and run test cases using the following ways:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": { "features": ["features/addition/decimal", "features/multiplication"] }, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
-H "Content-Type: application/json"

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": { "features": ["features/addition/decimal/add.feature", "features/multiplication/multiply.feature"] }, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
-H "Content-Type: application/json"

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": { "features": ["features/addition/decimal/add.feature:4:8"] }, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
-H "Content-Type: application/json"

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": {"tags":"@Sum"}, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
-H "Content-Type: application/json"

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": {"name":"addition"}, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
-H "Content-Type: application/json"

Generate test reports

A test report gives an overview of the features steps and their result. Test reports of your Cucumber-based Espresso tests are generated using the plugins parameter of the cucumberOptions JSON object when running the execute build cURL command.

The following table lists the values that the plugins parameter accepts to generate a test report:

Complete the following steps to generate and download test report:

1. Specify format of test report in command
2. Download the generated test report

Step 1: Specify format of test report in command

Specify the format of the report in the plugins parameter of the cucumberOptions JSON object, as shown in the example execute build cURL command:

  curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
  -X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
  -d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": {"plugins": ["junit"]}, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
  -H "Content-Type: application/json"

To generate test report in more than one format, specify the report format in a JSON array, as shown in the following example cURL command:

  curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
  -X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
  -d '{"app": "bs://<app_id>", "testSuite": "bs://<test_suite_id>", "cucumberOptions": {"plugins": ["html","junit","pretty"]},, "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
  -H "Content-Type: application/json"

Step 2: Download the generated test report

Specify a name for the report file, and set your build and session ID in the following cURL command:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" -X GET "https://api-cloud.browserstack.com/app-automate/espresso/v2/builds/{build_id}/sessions/{session_id}/assets" >> <filename>.zip

This command downloads a compressed file of the test report to your machine.

Generate reports for Android <= 10.0

To generate test/coverage reports for Android OS 10 and lower, make the following additional changes to the manifest.xml file:

- Add android.permission.WRITE_EXTERNAL_STORAGE
- Add android.permission.READ_EXTERNAL_STORAGE
- Set android:requestLegacyExternalStorage="true"

Use test sharding

All test cases in your Cucumber-based Espresso tests that are run using cucumberOptions are, by default, executed sequentially.

However, instead of running your test suite sequentially, you can split those test cases into different groups called shards. Each shard is then executed in parallel on one or more devices, decreasing the test execution time.

To use sharding, add the shards object to the execute build cURL command and set the strategy parameter to cucumberOptions. You can also add filters in the values parameter.

An example JSON structure to add to the shards object is as follows:

{
  "shards": {
    "numberOfShards": 2,
    "mapping": [{
      "name": "Shard 1",
      "strategy": "cucumberOptions",
      "values": {
        "tags": "TAG_EXPRESSION",
        "name": "REGEXP",
        "features": "FEATURE_FILE_PATH"
      }
    }, {
      "name": "Shard 2",
      "strategy": "cucumberOptions",
      "values": {
        "tags": "TAG_EXPRESSION",
        "name": "REGEXP",
        "features": "FEATURE_FILE_PATH"
      }
    }]
  },
  "devices": ["Samsung Galaxy S20-10.0", "Google Pixel 3-9.0"],
  "app": "<app-id>",
  "testSuite": "<test-suite-id>"
}

The following example cURL command splits the test cases into two shards, filters test cases using the tags filter, and runs the test on Samsung Galaxy S20-10.0 and Google Pixel 3-9.0 devices:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-d '{ "shards": { "numberOfShards": 2, "mapping": [{ "name": "Shard 1", "strategy": "cucumberOptions", "values": { "tags":"@first" }}, { "name": "Shard 2", "strategy": "cucumberOptions", "values": { "tags":"@second" }}]}, "devices": [ "Samsung Galaxy S20-10.0", "Google Pixel 3-9.0"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

Check out the test sharding section to learn more about sharding.