Download custom reports or other build artifacts

Learn how to download Cypress custom reports or any other custom build artifacts that you might generate during your Cypress tests.

In this guide you will learn the following:

• Understand about Cypress custom reporters
• Download custom reports on your system
• Download any other build artifacts that you generate
• Download Allure reports
• Understand the downloaded report/artifact directory structure
• Learn how to generate a single report file

Cypress custom reporters

Cypress offers a feature to use any custom reporter like mochawesome, JUnit reporter, etc. These reporters and their associated reporter options have to be specified either in CLI options or on your cypress.config.js or cypress.json file. You can read more in Cypress documentation for custom reporters.

Now, once you have specified the reporter and reporter options, the reports would be generated in a particular folder within your project directory, for e.g. ./results. If your tests would have been running locally on your laptop then you could have accessed the reports directly from the download directory but since the tests are running on BrowserStack machines, so the reports would be generated in BrowserStack machines and through this feature outlined in this document, we make it available to you.

Download custom reports

Edit your browserstack.json file to specify the download directory where you want BrowserStack to automatically download the reports after the build completes.

To download custom reports:

1. Specify custom reporter and reporter options in cypress.json or in CLI browserstack-cypress run --reporter <specify-reporter-here> --reporter-options <reporter-options-go-here>. Learn more about this step in Cypress documentation.
2. Specify the download directory in browserstack.json as shown below:
   browserstack.json

   "run_settings": {
    "downloads": ["./results", "./any_folder/any_other_folder"]
   }
   

   The download directory path has to be specified relative to the path where your browserstack.json resides.

3. Run your Cypress build with --sync argument and we would automatically create the download directory after your build completes. If Cypress is not running in the --sync mode, then you’d be able to download the directory by running the command: browserstack-cypress generate-downloads <build-id>

Download Allure reports

The Allure framework is a flexible, lightweight, multi-language test reporting tool that not only provides a concise representation of what has been tested but also presents it in a neat web report format.

Prerequisite

To use Allure with Cypress, ensure that you have updated the Allure-specific information in your repository, including any changes in the cypress.config.js and e2e.js files. For more information on getting started with the setup of Cypress Allure reports, refer to the README of the cypress-allure-plugin Git repository.

To download Allure reports in Cypress integrated with BrowserStack, follow the steps below:

1. Specify the download directory in browserstack.json and add the following information in npm_dependencies:

   browserstack.json

   "run_settings":
     "downloads": [
       "./allure-results"
     ],
   "npm_dependencies": {
   "@shelex/cypress-allure-plugin": "^2.32.1" }
   
   

   You can choose the name of the directory; in this example, we used allure-results.

2. From the root folder, execute the command in your terminal to start the test:

     browserstack-cypress run --env allure=true
   

   A new folder named allure-results is generated inside the build_artifacts/ build_id/session_id path. The folder contains the build artifacts, which are JSON files. After running your test, go to the Automate dashboard to view your test results. Your folder structure looks something like this:

3. To download allure reports, you need to install Allure and then provide the path to the JSON file that is present the allure-results folder. Run the following commands:
   brew install allure
allure serve <path to json file>
   
   

To locate the JSON file path, consider the following example (based on the folder structure in the above image):
The JSON file, named 69b836fc-09bc-443a-9e21-1f766201dd68-result.json, can be found within the allure-results folder at the following location:

build_artifacts/67b215d3d30758c781cc35e8b610081bb5bd980e/chrome-116-OS X-Ventura/aea340d9f01d570afd1d28a46abf96ec85d756f1/allure-results/69b836fc-09bc-443a-9e21-1f766201dd68-result.json"

Download any other build artifact

As you saw on the above section, we don’t really distinguish between your custom reports or any other artifact that you might generate during your Cypress tests. For e.g. if you want to take screenshots at specific points in time during your test and save it in a particular directory or if you want to generate any other log file during the run, then you can download all those artifacts as well using this feature.

You simply need to follow steps 2 and 3 of the above section i.e. you still need to specify the comma-separated list of download directories that you needs us to download onto your system. The example browserstack.json file is shown below:

"run_settings": {
    "downloads": ["./my-custom-screenshot-folder", "./any_folder/any_other_folder"]
}

As can be seen in the example above, you need to ensure that you provide the correct directory structure where the artifacts are supposed to be generated if this build were to run locally on your laptop. Also note, the directory path needs to be specified relative to the path where your browserstack.json resides.

How to navigate the downloaded artifacts

When you run Cypress on your laptop, you just run a single instance of Cypress generally and the platform is also a single one i.e. your laptop. But, when your tests run on BrowserStack across multiple browsers/OSes and in parallels, so we are basically running multiple instances of Cypress across different machines in parallel. So, if you’ve specified a reporter then reports are actually generated across multiple machines on BrowserStack.

The downloaded directory will contain multiple sub-directories consisting of the various reports that were generated across multiple machines where your Cypress tests were run. Considering that you had specified 2 download folders results and my-custom-screenshot-folder, the downloads would take place in the following directory structure:

• ./build_artifacts
  • <build-id-whose-reports-are-downloaded-under-this-folder>
    • chrome-91-win-10
      • session-id-for-machine-1
        • results
        • my-custom-screenshot-folder
      • session-id-for-machine-2
        • results
        • my-custom-screenshot-folder
      • session-id-for-machine-3
        • results
        • my-custom-screenshot-folder
    • firefox-88-macOS-catalina
      • session-id-for-machine-1
        • results
        • my-custom-screenshot-folder
      • session-id-for-machine-2
        • results
        • my-custom-screenshot-folder
      • session-id-for-machine-3
        • results
        • my-custom-screenshot-folder

As you can see in the above structure, the contents of the results folder and any other folder that you might have specified, will be generated in multiple machines and hence we put the contents inside multiple folders pertaining to each machine where your tests have run.

Merge custom reports

If you’ve specified a custom reporter like mochawesome or JUnit and you get the generated files across different folders as shown in the previous section, you might want to combine the files to generate one single report for easier consumption.

You need to ensure that you have appended -[hash] to the specified file name in --reporter-options (read more in Cypress docs) so that different files get generated for different spec files running on the same machine. Once that is sorted, you can follow the below steps to create a combined report:

1. If you’re using mochawesome, then you need to use mochawesome-merge utility to generate one combined JSON report.
2. You can install the following modules to achieve the above objective:

   npm install --save-dev mochawesome mochawesome-merge mochawesome-report-generator
   

3. Now, lets assume that you had specified ./results within the downloads settings of browserstack.json, then the folder structures have been downloaded after your build finishes on BrowserStack. You need to run the mochawesome-merge utility now using the following command:

   npx mochawesome-merge "build_artifacts/<build-id>/*/results/*.json" > mochawesome.json
   

4. Once the merged JSON is created, you can run the next command to create the single HTML report:

   npx marge mochawesome.json
   

5. If you’re using JUnit reporter, you can use the junit-report-merger to achieve the same as above.