Use test sharding

Parallel testing enables you to reduce the overall test execution times, resulting in improved build times and faster releases. Test Sharding enables you to test at scale on BrowserStack.

Introduction

By default, all test cases in your XCUI test-suite are executed sequentially. Instead of running your test-suite sequentially, you can split test cases into different groups called shards. Each shard can then be executed in parallel on one or more devices. This is especially helpful when your test-suite has a large number of test cases.

For example, if your test-suite has 500 test cases. Assume each test case takes about a minute to execute. If the test-suite is executed sequentially, the overall execution time will be close to 500 mins. However, if you split your test-suite into 10 shards, each shard will consist of 50 test cases. If each shard is executed in parallel, the total test execution time will be down to 50 mins.

To enable test sharding, you need to configure the shards parameter in the REST API request to start XCUI test execution.

REST API endpoint :

POST /app-automate/xcuitest/v2/build

Example cURL request:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/xcuitest/v2/build" \
-d '{"shards": {"numberOfShards": 2, "mapping": [{"name": "Shard 1", "strategy": "only-testing", "values": ["SampleXCUITests/testAlert", "SampleXCUITests/testText"]}, {"name": "Shard 2", "strategy": "only-testing", "values": ["SampleXCUITests/testLogin"]}]}, "devices": ["iPhone X-11"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://j3c874f21852ba57957a3fdc33f47514288c4ba4"}' \
-H "Content-Type: application/json"

In this example, the test-suite will be split into 2 shards, and each of those shards will execute in parallel on iPhone X-11. This will block 2 parallel tests licenses in your App Automate subscription plan.

Sharding strategy

There are multiple strategies to control how you want to split the test-suite into different shards.

1. Use only-testing strategy

Using this strategy, you can group test cases by specifying the classes or tests (e.g. SampleXCUITestsClass/testAlert) in different shards. All the shards will be executed in parallel on the devices specified in the devices parameter.

REST API endpoint :

POST /app-automate/xcuitest/v2/build

Example cURL request :

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/xcuitest/v2/build" \
-d '{"shards": {"numberOfShards": 2, "mapping": [{"name": "Shard 1", "strategy": "only-testing", "values": ["SampleXCUITestsClass/testAlert", "SampleXCUITestsClass/testText"]}, {"name": "Shard 2", "strategy": "only-testing", "values": ["SampleXCUITestsClass/testLogin"]}]}, "devices": ["iPhone X-11","iPhone 8 Plus-11"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases will be split into 2 groups (i.e. shards). The first shard will consist of test cases testAlert and testText of class SampleXCUITestsClass. The second shard will consist of test case testLogin of class SampleXCUITestsClass. Both the shards will execute in parallel on iPhone X-11 and iPhone 8 Plus-11. This will block 4 parallel tests licenses in your App Automate subscription plan.

2. Use skip-testing strategy

Using this strategy, you can group test cases by skipping classes or tests (e.g. SampleXCUITestsClass/testAlert) in different shards. All the shards will be executed in parallel on the devices specified in the devices parameter.

REST API endpoint :

POST /app-automate/xcuitest/v2/build

Example cURL request :

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/xcuitest/v2/build" \
-d '{"shards": {"numberOfShards": 2, "mapping": [{"name": "Shard 1", "strategy": "skip-testing", "values": ["SampleXCUITestsClass/testAlert"]}, {"name": "Shard 2", "strategy": "skip-testing", "values": ["SampleXCUITestsClass/testText"]}]}, "devices": ["iPhone X-11","iPhone 8 Plus-11"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://e994db8333e32a5863938666c3c3491e778352ff"}' \
-H "Content-Type: application/json"

In the above example, the test cases will be split into 2 groups (i.e. shards). The first shard will consist of all test cases of class SampleXCUITestsClass except testAlert. The second shard will consist of all test cases of class SampleXCUITestsClass except testText. Both the shards will execute in parallel on iPhone X-11 and iPhone 8 Plus-11. This will block 4 parallel tests licenses in your App Automate subscription plan.

Device selection strategy

If you don’t want each of the shards to execute on all specified devices, you can set the deviceSelection attribute inside the shards parameter to any value. This will ensure that each shard will execute on any randomly selected device from the list of devices specified in the devices parameter. By default, its value is set to all.

REST API endpoint :

POST /app-automate/xcuitest/v2/build

Example cURL request :

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/xcuitest/v2/build" \
-d '{"shards": {"numberOfShards": 2, "deviceSelection": "any", "mapping": [{"name": "Shard 1", "strategy": "only-testing", "values": ["SampleXCUITests/testAlert", "SampleXCUITests/testText"]}, {"name": "1", "strategy": "only-testing", "values": ["SampleXCUITests/testLogin"]}]}, "devices": ["iPhone X-11", "iPhone 8 Plus-11"], "app": "bs://f7c874f21852ba57957a3fdc33f47514288c4ba4", "testSuite": "bs://j3c874f21852ba57t95g7a3o1fdc33h69t5gj471"}' \
-H "Content-Type: application/json"

In the above example, the test cases will be split into 2 groups (i.e. shards). Both the shards will execute in parallel, but each shard will execute on a single device randomly selected from the list (i.e. iPhone X-11 or iPhone 8 Plus-11). This will block 2 parallel tests licenses in your App Automate subscription plan (one for each shard).

Related topics

• Manage your uploaded app or test suite using our upload app REST API and upload test suite REST API.

• Use the get app API endpoint and list test suites API endpoint to list your recently uploaded apps and test suites.

• Delete your uploaded app or test suite using the delete app API endpoint and delete test suite API endpoint.