View test results

Once an Espresso test execution is started using REST API, you will be able to access the test execution results on the App Automate dashboard as well as using our REST API. Every test execution has a unique build ID associated to it. This is returned in the API response to start test execution.

For each build ID, there is a corresponding test build on the dashboard. Once a build is selected, you can see the overall summary of your test execution and results for each individual test case. You can drill down into the details of a specific test case to view its execution details and debugging information such as video recording, instrumentation logs and device logs. You can also use this build ID to access test results and debugging information using REST API.

Interpreting test execution results

Each test case in the test-suite can have one of the following status :

1. Queued : A test case queued for execution. This is the default initial state.
2. Running : A test case that is being executed by the test runner.
3. Error : An erred test is one that failed due to an unanticipated issue on BrowserStack testing infrastructure.
4. Failed : Explicit assertion in the test case that marks the test as failed. This can also happen if there is an uncaught runtime exception during test execution.
5. Passed : Explicit assertion in the test case that marks the test as passed.
6. Timed out : Execution of a test case (in a running state) is halted either because:
   (a) The session got timed out (by BrowserStack) because it exceeded 2 hour limit.
   (b) The test case was “idle” for 15 mins (“Idle” is defined as no update from the test runner).
7. Skipped : A test case that was never invoked during the test-suite execution. It can happen in three different scenarios:
   (a) The session got timed out (by BrowserStack) because it exceeded 2 hour limit. All remaining test cases in the session will be marked as skipped.
   (b) Execution of a test case is skipped by the test runner. For e.g. this can happen if the test case uses an @Ignore annotation.
   (c) Test runner encountered a AssumptionViolatedException and hence skipped execution.
   

Interpreting build execution results

The status of a build depends on the status of executed tests in the build. BrowserStack sets build status as (in order of precedence):

1. Running : If the status of one or more tests is running.
2. Failed : If one or more tests has failed assertions or uncaught run-time exceptions.
3. Error : If one or more tests had errors due to unexpected BrowserStack issues.
4. Timed out : If the status of one or more tests is timed out.
5. Passed : If the status of all tests is passed (excluding skipped).
6. Queued : If the status of all tests is queued.
7. Skipped : If status of all tests is skipped.