Integrate your Quality Gate result with your CI/CD

Learn how to integrate Quality Gate with your CI/CD tools

In order to automate workflows like blocking or allowing deployments/PRs automatically, you can integrate via the Quality Gates API.

You can get started in minutes with the following recipes that you can use out of the box or modify for your specific workflows.

Examples

Generic Shell Script to fetch and interpret Quality Gate Result

Get started in minutes by using this shell script as a starter!

How it works:

Plug in variables like project_name, build_name, username, access_key which you are using to set up your builds (via your framework config files or browserstack.yml files) on Test Observability. This will help retrieve the Quality Gate analysis for the latest build matching these criteria.
The script will automatically get the Quality Gate result for your build, assert the response and throw the right exit code.
This script also implements a polling loop to retry the API call for a set number of times while Test Observability completes the Quality Gate analysis.

Note 1: This sample script uses the jq tool to be installed to process the Quality Gate response and throw the right exit code. You can install jq from brew or the jq website.

Note 2: If you use the BrowserStack Jenkins Plugin, ensure you specify the $BROWSERSTACK_BUILD_NAME environment variable in the shell script where you have to enter your build_name. This will ensure the correct build’s Quality Gate result is fetched.

If you wish to modify the script, please keep the following in mind:

The Quality Gate Status API endpoint requires you to provide the UUID of a build - a unique identifier to distinguish builds. This script leverages another API endpoint - the Get Latest Build Details API to help get you the UUID automatically, and plug it in to the API.

```bash
#!/bin/bash

# Update values here
project_name="{Insert Value Here}"  # Replace with your project name
build_name="{Insert Value Here}"  # Replace with your build name use $BROWSERSTACK_BUILD_NAME if you use the BrowserStack Jenkins Plugin
username="{Insert Value Here}" # Replace with the username of the user who ran the build (Hint: Check your Browserstack.yml / Framework Config Files)
access_key="{Insert Value Here}"  # Replace with the access key of the user who ran the build (Hint: Check your Browserstack.yml / Framework Config Files)
max_attempts=20 # Replace with your max retry attempt
build_tags="{Insert Value Here}" # Optional - Replace with custom build tags if any


# Script Functions - REFRAIN FROM MODIFYING THE SCRIPT BELOW 

# Function to sanitize project and build names
sanitize_name() {
local name="$1"
sanitized_name=$(echo "$name" | sed 's/ /%20/g')
echo "$sanitized_name"
}

# Function to get the UUID of the latest build
get_latest_build_uuid() {
local sanitized_project_name=$(sanitize_name "$project_name")
local sanitized_build_name=$(sanitize_name "$build_name")
local sanitized_build_tags=$(sanitize_name "$build_tags")

# Fetch and sanitize the JSON response

if [ -n "$sanitized_build_tags" ]; then
  response=$(curl -s --retry 3 --connect-timeout 10 -u "$username:$access_key" "https://api-observability.browserstack.com/ext/v1/builds/latest?project_name=$sanitized_project_name&build_name=$sanitized_build_name&user_name=$username&build_tags=$sanitized_build_tags")
else
  response=$(curl -s --retry 3 --connect-timeout 10 -u "$username:$access_key" "https://api-observability.browserstack.com/ext/v1/builds/latest?project_name=$sanitized_project_name&build_name=$sanitized_build_name&user_name=$username")
fi

response=$(echo "$response" | perl -pe 's/([[:cntrl:]])/sprintf("\\x{%02X}", ord($1))/eg')

local build_uuid=$(echo "$response" | jq -r '.build_id')
echo "$build_uuid"
}

# Function to hit Quality Gates API and get the result
get_quality_gate_result() {
quality_gate_result=$(curl -s -H --retry 3 --connect-timeout 10 -u "$username:$access_key" https://api-observability.browserstack.com/ext/v1/quality-gates/$build_uuid)
echo "$quality_gate_result"
}

# Function to poll the API until the results of the Quality Gate are received
poll_quality_gate_api() {
local attempt=0
local max_time=600
local total_time=0

while [[ $attempt -lt $max_attempts ]] && [[ $total_time -lt max_time ]]; do
quality_gate_result=$(get_quality_gate_result "$1" "$2" "$build_uuid")
local result=$(echo "$quality_gate_result" | jq -r '.status')
if [ "$result" != "running" ]; then
echo "$quality_gate_result"
exit 0
fi
sleep 30 # Poll every 30 seconds
((attempt++))
total_time=$((attempt * 30))  # Total elapsed time in seconds
done

echo "Timed out waiting for Quality Gate results"
exit 1
}

# Function to assert the API response and throw pass/fail exit code
assert_quality_gate_result() {
local result=$(echo "$quality_gate_result" | jq -r '.status')
# Replace the condition below with the actual condition to determine pass/fail
if [ "$result" == "passed" ]; then
echo "Quality Gate passed"
exit 0
else
echo "Quality Gate failed"
exit 1
fi
}

# Quality Gates API Poll
build_uuid=\$(get_latest_build_uuid)
sleep 20
quality_gate_result=\$(poll_quality_gate_api)
echo "Quality Gate Result: \$quality_gate_result"

assert_quality_gate_result
```

Copy

Pipeline Example for Jenkins

In order to integrate this into Jenkins, you can refer to the example below of how you can implement this in a multi-stage pipeline.

How it works:

The pipeline script uses the catchError capability in Jenkins to ensure that if a pipeline stage fails, it doesn’t stop the subsequent stages from executing.
The pipeline script then marks the overall build status using a post stage after the Quality Gate stage completes.
While you can use other approaches to catchError, ensure you allow the Quality Gate to execute after your tests complete - this will help you prevent false negatives from blocking deployments - as your Quality Gate could pass despite test failures.
Note: Insert the shell script listed above within this pipeline example. It has been omitted in this example for brevity.

Pipeline

pipeline {
    agent any

    stages {
        stage('Build / Test Step') {
            // Place your build and test steps in separate stages with the catchError handling like below
            steps {
                catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
                    script {
                        // Build and Test Steps Go Here
                    }
                }
            }
        }

        stage('Quality Gate Stage') {
            // Implement a new Quality Gate Stage like below
            steps {
                catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
                    script {
                        echo 'Performing Quality Gate Check...'
                        def qualityGateResult = sh(script: '''
                        # <paste sample shell script from above here>
                        ''', returnStatus: true)
                        
                        if (qualityGateResult == 0) {
                            echo 'Quality Gate passed.'
                            currentBuild.result = 'SUCCESS'
                        } else {
                            echo 'Quality Gate failed.'
                            currentBuild.result = 'FAILURE'
                        }
                    }
                }
            }
            
            post {
                always {
                    script {
                        // Mark the stage based on currentBuild.result
                        if (currentBuild.result == 'SUCCESS') {
                            echo 'Quality Gate passed.'
                        } else {
                            echo 'Quality Gate failed.'
                            error 'Quality Gate failed.'
                        }
                    }
                }
            }
        }
    }
}

Did this page help you?

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked

This page has exactly what I am looking for
This content & code samples are accurate and up-to-date
The content on this page is easy to understand
Other (please tell us more below)

Any additional feedback?

Thank you for your valuable feedback

Is this page helping you?

Yes

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked

This page has exactly what I am looking for
This content & code samples are accurate and up-to-date
The content on this page is easy to understand
Other (please tell us more below)

Any additional feedback?

Thank you for your valuable feedback!