How to perform Visual Regression Testing using Cypress

Home Guide How to perform Visual Regression Testing using Cypress

How to perform Visual Regression Testing using Cypress

Visual regression testing in Cypress compares snapshots of web pages to help identify unintended UI changes.

Overview

What is Visual Regression Testing?

A visual regression test checks what the user will see after any code changes have been executed. This is done by comparing screenshots taken before and after the code changes.

Visual Regression Testing using Cypress

Cypress is a popular test automation framework that allows a lot of extensibility. Visual Regression Testing is one of the Extensible features of Cypress. Cypress offers a lot of plugins that can be used to capture visual images and compare them.

Benefits of Cypress Visual Regression Testing

Identify unintended visual changes before they reach production
Automated Visual Comparisons by integrating with tools like Percy
Seamless CI/CD Integration
Easy to manage baseline images
Cross-Browser Compatibility

Learn all about visual regression testing in Cypress, how to perform it and the tools that make the process easier.

What is Regression Testing?

Regression Testing is the type of software testing process conducted to ensure that the recent code changes have not negatively affected the existing features or introduced new issues.

All or part of impacted features is tested as part of the regression testing process. Any issues which come into existence due to the latest code changes should be caught in this type of testing.

Role of Automation in Regression Testing

Unlike Sanity and Smoke, the scope of regression testing is vast, since the QA has to test the entire application or subset of features in regression testing. Considering that Manual Testing for Regression involves a lot of time and effort, also it is not cost-effective. This is the reason, why most organizations are opting for automated regression testing. Regression Testing in Agile Teams often works in Risk-based approach, opting for test automation frameworks for a streamlined test process.

Types of Regression Testing

Regression Testing can be classified into Functional Regression Testing and Visual Regression Testing. Fundamentally both work differently as explained below.

Functional Regression Testing

Function Regression Testing is the type of testing where workflow or use-cases will be tested using manual or automation tools. It involves entering some values by fetching selectors from browser DOM. This also includes validating text-based values by getting them either programmatically or manually. Most of the modern testing in practice is based on this approach.

Visual Regression Testing

A visual regression test checks what the user will see after any code changes have been executed. This is done by comparing screenshots taken before and after the code changes. This is why visual regression tests are also sometimes called visual snapshot tests. Visual test highlights any visual changes that occur after the code update.

Visual tests generate, analyze, and compare browser snapshots to detect if any pixels have changed. These pixel differences are called visual diffs (sometimes called perceptual diffs, pdiffs, CSS diffs, UI diffs). You can set the threshold values, which give you the flexibility to compare images with small differences or ignore small differences.

Visual Regression Testing using Cypress

Cypress is a popular test automation framework, it allows a lot of extensibility. Visual Regression Testing is one of the Extensible features of Cypress. There are a lot of plugins available in Cypress that can be used to capture visual images and compare them. This tutorial, explains Cypress Visual Testing using Cypress Image Diff plugin.

Step by Step Tutorial of Cypress Visual Regression

To set up Cypress Visual Regression environment, you need to install the pre-requisites listed below.

Step 1: Install the cypress image diff npm package

Navigate your project Root folder (the directory where package.json is located). Enter the below command

npm i -D cypress-image-diff-js

Step 2: Configure Image Diff Plugin

Navigate to cypress/plugin/index.js and enter the commands written below

// cypress/plugin/index.js
module.exports = (on, config) => {
const getCompareSnapshotsPlugin = require('cypress-image-diff-js/dist/plugin')
getCompareSnapshotsPlugin(on, config)
}

Step 3: Import and add the Cypress image command

Navigate to cypress/support/commands.js and enter the code written below

// cypress/support/commands.js
const compareSnapshotCommand = require('cypress-image-diff-js/dist/command')
compareSnapshotCommand()

Step 4: Configure Reporter

Navigate to cypress/support/index.js and enter the code written below

after(() => {
cy.task('generateReport')
})

Step 5: Write your first Cypress Visual Test

Navigate to cypress/integration folder add new file example visual-test.js. Enter the code snippet written below

// visual-test.js
describe('Cypress Visual Testing', () => {
it('Compare fullpage of Google page', () => {
cy.visit("https://www.google.com/?hl=hi");
cy.compareSnapshot('google-page');
})
})

The above code, navigates to the Google Home page and compares if it is visually fine.

Note: When you first time run this script, it takes the base image and subsequent runs are compared with that base image.

Step 6: Run your first Visual Regression Test with Cypress

Run your Cypress Visual Test using the below command.

npx cypress run --spec "cypress/integration/visual-test.js"

Note: First Time when you run the test, it will be passed automatically as both base, and comparison images are fresh, and you might not see any difference.

Step 7: View Report

One of the key features of this plugin is that it generates the good HTML report. Once you run the test, two folders will be created i.e.:

cypress-visual-report: It contains, an HTML report, if the test passes there will not be any images shown if the test fails it will show baseline, comparison, and difference in that image.
cypress-visual-screenshots: This folder contains three subfolders namely baseline, comparison, and diff, where each folder contains the respective image files.

Passed Test Report looks like below:

Failed Test Report Looks like the below:

How to perform Cypress Visual Testing of an element

Cypress can be used to perform visual tests on a specific element, where it compares the before and after screenshots of the specified elements as seen in the example below.

describe('Visuals', () => {
it('should compare screenshot from a given element', () => {
cy.visit('www.google.com')
cy.get('#report-header').compareSnapshot('search-bar-element')
})
})

This code snippet takes a snapshot of element #report-header and compares visually if it is same or not.

Adding Threshold to Visual Test

By default, the value of the threshold will be zero, which means it should match exactly with the base image. However, you can make it flexible by applying other threshold values.

Check out other useful functions of cypress-image-diff plugin

Executing Cypress Visual Comparison Test using BrowserStack Automate

To run Cypress Visual Comparison Tests on Real Device Cloud like BrowserStack follow the steps mentioned below.

Step 1: Install BrowserStack Cypress plugin

Step 2: Create browserstack.json file using browserstack-cypress init command

Step 3: Enter the below code

{
"auth": {
"username": "<my_username>",
"access_key": "<my_access_key>"
},
"browsers": [
{
"browser": "chrome",
"os": "Windows 10",
"versions": [
"latest",
"latest - 1"
]
}
],
"run_settings": {
"cypress_config_file": "./cypress.json",
"cypress_version": "9",
"project_name": "My sandbox project",
"build_name": "Build no. 1",
"parallels": "2",
"npm_dependencies": {
"cypress-image-diff-js": "^1.18.0"
}
}
}

Note:

You can find the username and access key by logging into the browserstack website
You can also change the browser settings and platform settings from browserstack.json file

Step 4: Configure cypress.json file to include the .js files.

{
"testFiles":["*.js"]
}

Step 5: Execute your Browserstack Test

Use the below command to execute Cypress Visual Test in Browserstack

browserstack-cypress run –sync

Run Cypress Tests on Real Device Cloud

Executing Cypress Visual Regression Test with Percy

Percy is a visual testing tool that helps in visual testing of an application. Since Percy is now a part of BrowserStack, you can access Percy Dashboard with BrowserStack credentials. You can use Percy with Cypress to perform a visual test by following the steps written below:

Step 1: Install Percy using the following command

npm install --save-dev @percy/cli @percy/cypress

Step 2: To import Percy to Cypress, navigate to cypress/support/index.js File and enter the following command

import '@percy/cypress'

Step 3: Write your first Percy Visual Test Script as shown below

describe('Cypress Visual Testing', () => {
it('Compare fullpage of Google page', () => {
cy.visit("https://www.google.com/?hl=hi");
cy.percySnapshot('Google');
})
})

This example does a visual comparison of the Google.com Page using Percy and Cypress

Run your Percy Test

Go to http://percy.io create a sample project. Once you create a Sample Project Percy will generate API key. Copy the Percy Token from there.

Talk to an Expert

Enter the Percy Token using the following command for MacOS

export PERCY_TOKEN=<your_token>

For Windows OS, enter the Percy Token using the command written below

set PERCY_TOKEN=<your_token>

For powershell use below command to enter Percy Token

$env:PERCY_TOKEN ==<your_token>

Once the Percy Token is entered use below command to run your Percy Cypress Tests

npx percy exec -- cypress run

Cypress Visual Tests Starts Running and it gives you following result in command line

Best Practices for Visual Regression Testing in Cypress

While running visual regression tests using Cypress, you can follow the given best practices to run efficient and accurate tests:

Leverage a reliable visual testing library: Cypress does not come with in-built visual testing capabilities, so it is best to utilize third-party libraries like cypress-image-snapshot and percy-cypress as per your project needs.
Baseline images should be set up properly: Once the stable state of UI is confirmed, you can capture baseline snapshots. Store the baseline images in version control to ensure consistency.
Avoid flaky visual tests: In order to reduce flaky tests you can wait for UI elements to fully load by using cy.wait() or cy.intercept() where necessary. Also, ensure environment consistency when it comes to browser, viewport size, resolution etc.
Test stable elements only: You should avoid testing highly dynamic elements like timestamps, carousels, or live data feeds. Exclude untable regions in snapshots if necessary.
Execute tests in controlled environment: Obtain consistent results across different runs by using Docker or a CI/CD pipeline. For better accuracy, you can run tests in headless mode.
Manage Tolerance level: It is important to set small tolerance levels to catch major UI changes while allowing minor pixel differences .

Why perform Cypress Visual Testing on BrowserStack Percy?

Performing Cypress Visual Testing on BrowserStack Percy offers several key advantages:

Cross Browser Testing: BrowserStack Percy allows you to run visual tests across multiple browsers (desktop and mobile browsers) and devices, ensuring consistent UI across platforms.
Automated Visual Regression: Automatically compares visual snapshots of your app against baseline images, detecting any unintended visual changes or regressions.
Seamless Cypress Integration: BrowserStack Percy integrates effortlessly with Cypress, enabling you to incorporate visual testing into your existing Cypress test suite without complex setups.
Real Device Testing: With BrowserStack’s real device cloud, you can test your application on actual devices to ensure the UI looks perfect on different screen sizes and resolutions.
Parallel Testing and Speed: Run tests in parallel, significantly speeding up the testing process and allowing for faster feedback and debugging.
Comprehensive Reporting: BrowserStack Percy provides clear visual diff reports, making it easier to identify and fix visual discrepancies in your UI.
CI/CD Integration: BrowserStack Percy integrates with your CI/CD pipeline, ensuring visual tests run automatically with every code push or deployment.
Collaborative Feedback: It allows team members to provide feedback on visual differences, enhancing collaboration between developers and designers.

Conclusion

Visual regression testing in Cypress helps identify and debug unexpected visual changes and thus ensures UI consistency. Combining Cypress visual regression testing with tools like Percy further simplifies this process by offering automated visual comparisons, baseline management and smooth CI/CD integration and facilitates efficient tests and accurate results.

Try Percy Now

Useful Resources for Visual Testing

Run Visual Regression Tests effortlessly

Try BrowserStack Percy for seamless Visual Regression Testing using Visual Diff for smart pixel-to-pixel comparison & accurate test results

How to perform Visual Regression Testing using Cypress