Test orchestration on BrowserStack Automate

Optimize your Automate tests by efficiently managing their execution, reporting, and rerunning.

Test Orchestration is designed to streamline your team’s testing processes. It enhances test execution by automating tasks such as rerunning failed tests, stopping test suites after critical failures, and prioritizing failed tests. These features reduce manual intervention, optimize resource usage, and improve the reliability of test outcomes, allowing teams to focus on resolving defects faster.

With Test Orchestration, you can accelerate feedback loops, quickly identify and address critical issues, and ensure your development and QA teams maintain high product quality throughout the testing cycle.

Supported strategies

BrowserStack Automate currently supports the following Test Orchestration strategies:

• Auto Reruns
• Fail Fast
• Prioritizing Failures

Test Orchestration is BrowserStack can be triggered using the following capability:

Key Features

Auto Reruns

Automatically rerun failed tests to identify flaky issues, reduce manual intervention, and improve test reliability. By automatically rerunning failed tests, you can address intermittent issues without manually triggering additional test cycles, ensuring more reliable results.

To enable Auto Reruns, configure the feature in your browserstack.yml file. You can control the number of reruns by setting a limit on how many times a test should be retried.

Fail Fast

Fail Fast stops the execution of a test suite when a critical failure occurs. This feature helps teams save resources and prioritize fixing high-priority issues by preventing further execution of tests that are likely to fail due to existing critical problems.

You can configure Fail Fast by setting thresholds for the number of allowable failures. Once this threshold is reached, the test suite will automatically abort, allowing the team to focus on resolving the most urgent issues.

Prioritizing Failures

Prioritizing failures involves running previously failed tests first in subsequent test cycles. This strategy is useful when faster feedback is crucial, as it helps teams quickly identify if known issues persist or if new issues have emerged.

By prioritizing fixing failures from the last test run, you can expedite feedback and address high-impact bugs earlier in the development process.

Benefits

• Faster Feedback Loops Running failed tests first helps teams get immediate insights into critical issues, reducing the time taken to detect and fix problems.
• Increased Test Efficiency By focusing on high-priority failures, teams can avoid re-running all tests unnecessarily, saving time and resources.
• Early Bug Detection Prioritizing failures ensures that critical issues are addressed sooner, minimizing the risk of issues progressing further into the development cycle.
• Improved Development Focus Identifying recurring or new failures quickly allows developers to concentrate on resolving issues that have a higher likelihood of impacting the end-user experience.

The following is a sample master-slave script for a Jenkinsfile in a Jenkins Pipeline:

BrowserStack will be adding more strategies to the above list. If you want any specific Test Orchestration feature, contact support.