Complete Guide To Cypress Visual Regression Testing

Sometimes referred to as automated UI testing or visual regression testing, VRT checks software from a purely visual standpoint (taking a screenshot and comparing it against another approved screenshot). Cypress is an emerging test automation framework that enables teams to ship high-quality products faster.

Cypress utilizes a modern architectural approach and is fully equipped with built-in screenshot functionality and also video recording capabilities, making it one of the best test automation frameworks for Cypress visual regression testing.

For those Automation Engineers validating a UI or performing an HTML canvas test, the UI look and feel of the application is vital for our end-users. Companies spend a lot on strategies to acquire more customers through advertisements, campaigns, and more. A missing visual error in production can jeopardize their reputation. My recommendation is to start learning some Visual Testing tools.

Visual Testing is about which parts of your UI are most important to test (Visual Coverage). Perhaps visual or HTML canvas test is similar to functional testing in that it’s designed to be an automated process that runs alongside code reviews. However, unlike functional testing, the HTML canvas test results cannot be considered as Passed or Failed.

Starting your journey with Cypress UI Automated Testing? Check out how you can test your Cypress automation test scripts on LambdaTest’s online cloud.

Now execute test scripts faster than any other automation testing grid on LambdaTest’s Automated UI testing cloud.

Let’s deep dive into performing visual regression testing with the Cypress test automation framework.

Understanding Visual Regression Testing

Before talking about visual regression testing, we need to understand the current state of visual testing. Functional test scripts can validate the size, position, and color scheme of visual elements.

As Automation Engineers, we perform ad hoc and exploratory testing to catch edge cases and visual bugs regarding manual checkpoints. But, to be honest, it could be challenging from a manual standpoint to verify an entire screen or different screens and detect those visual diffs.

Some years ago, there was a good approach called pixel matching. It was related to a bitmap of a screen capturing various points, and its pixels were compared to a baseline bitmap. Pixel matching tools can spot pixel differences quickly and consistently and highlight the visual differences based on pixels.

If we had a great approach, what went wrong? Well, various things were wrong with snapshot or screenshot testing; you can also test on Puppeteer Visual Testing to capture and compare screenshots and deliver seamless user experience. we can enumerate some of the challenges using this approach:

• Variation of screen resolutions
• Image hardware rendering bitmaps differently
• Variations in browser font size and other elements
• Flakiness on dynamic content (Capturing screenshots by the element, we can skip flaky visual tests)

Also Read – How to capture Screenshots in Selenium

With this Screenshot Testing Tutorial, you will learn how to perform Automated Screenshot Testing on the LambdaTest platform.

Actual tools currently combine computer vision as the core of visual comparison and visual diffs when discussing visual regression testing. Unlike pixel matching (snapshot testing), VRT tools do not need unique environments that remain static to ensure accuracy. By blending test automation and image processing, visual regression testing checks that our application looks as it should. With functional checks alone, multiple of the challenges of frontend testing persists.

Online Comparison Tool

Cypress Visual Regression Testing

In this cypress tutorial for cross browser testing with Cypress, we discussed the difference between snapshot testing and visual regression testing; typically, we can find some plugins in Cypress to compare images and follow the same process of performing snapshot testing; for example, we can use the cypress-plugin-snapshots.

This open-source plugin compares the baseline and the side of the current image by the side within the Cypress Test Runner if the pixel difference is above the threshold.

Check out our UI comparison tool to easily capture and compare UI screenshots.

Cypress Version 9.0.0

Before talking about Cypress plugins, we should consider the new Cypress 9.0.0, as I updated to that version; if you want to update, please consider the following changes below; my recommendation is to check the Release notes before updating any project to the latest version.

• The nodeVersion configuration option now defaults to the system. The behavior of the system option has always changed to use the Node.js binary/version that launched Cypress. If Cypress was not launched via the terminal, Cypress would use the bundled Node.js version. It could change the code behavior within your plugins file since it may be run in your system Node.js version #18684.
• Windows 32-bit is not supported in Cypress #17962.
• cy.contains() no longer yields the <body> element when it matches the content of <script> or <style> tags #14861.
• Attempting to add an existing built-in Cypress command using Cypress.Commands.add() will now throw an error, indicating that Cypress.Commands.overwrite() should be used instead to overwrite the behavior of existing commands #18572.
• Custom command implementations are now typed based on the declared custom chainable #17496.
• The bundled Node.js version was upgraded from 14.17.0 to 16.5.0. This could change the code behavior within the plugins file when using the bundled Node.js version of Cypress #18317.
• When null is passed as the encoding to cy.readFile() or cy.fixture(), the file is treated as binary and read as a Buffer. Similarly, null passed as the encoding to cy.writeFile() allows direct writing of buffers. If the encoding is unspecified, the default remains utf8, matching the current behavior #18534.
• Sticky elements within a fixed container will now be adequately scrolled to during action commands #4233.
• document.referrer will now correctly reflect the correct value from the application under test after cy.visit() #4295.

Also read: Now perform Cypress automation testing on Cypress version 9.0.0

If you are new to the world of Cypress or don’t know what is Cypress in detail, you can go through this complete Cypress Tutorial for Beginners, which will walk you through the basics of what is Cypress, including the Cypress Installation & Project Setup, and help you create your first test script using Cypress.

However, you can visit the LambdaTest YouTube channel for more videos on Cypress Testing, Selenium testing, etc., so there’s never a shortage of learning opportunities.

Cypress Visual Regression Plugin

I recommend one plugin (cypress-visual-regression) for Cypress visual regression testing, which is really easy to install within Cypress.

Setting up our Cypress Project

Step 1: Installation

We should add the following to your cypress.json file:

Now, let’s add the cypress-regression-testing plugin to cypress/plugins/index.js:

And add the command to cypress/support/commands.js:

Note: Please ensure import commands.js in cypress/support/index.js

Once the configuration is ready, we can add our visual testing checkpoints into our existing cypress tests, let’s see an example:

The code above will visit a web page, take a full page screenshot, and validate zero difference between the base and the actual image comparison. You will wonder about the image baseline, as we don’t have anything in our code. To run basic comparisons, we need to have a baseline image; let’s explore that.

Let’s run the following command:

Once the command is completed, we should see a new couple of folders like the following:

Once the baseline is created, we’re ready to compare and run a test; first, we need to run the following command:

After the command runs successfully, we will see the folder structure like the following.

As we can see from the image above, we have three folders created, one for the actual result, one for the base image, and another for spotting the difference between the images. So basically, we are done with our basic configuration; next time we run, we can spot any discrepancies in our command line.

Implementing Cypress Plugin Snapshots

Want to compare your screenshots against the baseline to identify visual changes and deliver pixel-perfect websites? Check out our visual comparison tool.

There is another exciting plugin for snapshot tests in Cypress. Same API as Jest that we can use with the Cypress GUI. To install the plugin, we need to run:

If you are using Cypress 9.0.0, which in my case, then you need to add an extra parameter like this:

Just avoid version conflicts and, in many cases, break the installation process. The –legacy-peer-deps flag was introduced with v7 to bypass peerDependency auto-installation; it tells NPM to ignore peer deps and proceed with the installation anyway. It is how things used to be with NPM v4 thru v6.

As we did for the previous plugin, we need to do the same for cypress-plugin-snapshot; add this to your cypress.json configuration file:

Then add to your cypress/plugins/index.js file and change it to look like this:

Similar to your cypress/support/index.js file and add the following line:

After installation is done, we are ready to create some tests; my suggestion is to create a new test spec under your integration folder ‘cypress/integration/.’

Code Walkthrough

Step 1: We first import config.json since it contains the test links located in /cypress/integration/e2e_tests/config.json

Step 2: We need to import our MainPage component; we use another component to reuse our HTML elements.

Step 3: We visit the URL. The base URL is stored in Cypress.json to ensure better portability and maintainability; also, we use the Before Fixture.

Step 4: In our first test, we take a full-page screenshot to validate that our web page is displayed correctly and there are no UI errors.

Full page screenshot

Step 5: In our second test, we want to validate multiple elements to make sure they are displayed correctly; this is useful in case of any specific element like a banner or footer on our webpage. For this case, we have our elements inside of our MainPage component located in cypress/page-objects/components/Mainpage.js

If we want to avoid flakiness related to dynamic content or other factors, it is better to capture elements to compare discrepancies; we want to ignore full-page screenshots and focus on our web pages’ specific UI instability areas.

Step 6: For the last test, we want to validate specific search results; in this case, we are searching in our blog for Cypress-related material and taking a full-page screenshot. Last but not least, we are renaming the screenshot to customize our image comparison.

A critical aspect of this plugin is the screenshot location, and the plugin saves the actual image in cypress/integration/__image_snapshots__. The snapshot compares against another picture of the current interface on subsequent runs, and if the tool finds any difference, the test fails. But what if we want to change our current image baseline? Well, in that case, as there is no way to run a baseline command, the easiest way to do that is to remove the previous images saved on cypress/integration/__image_snapshots__ and rerun our tests to get a new baseline.

Open Cypress test runner and click on the corresponding test to execute the same.

And here is the test execution, which indicates that we have a new baseline:

One more time to see that the image is correct:

You can download the code from the GitHub repository.

Also Read: How To Find HTML Elements Using Cypress Locators

Take this certification to showcase your expertise with end-to-end Cypress testing and stay one step ahead.

Here’s a short glimpse of the Cypress 101 certification from LambdaTest:

How to use Visual Regression Plugin on Cypress Cloud Grid

We can use a Cloud Cypress Grid like LambdaTest, which provides automated cross browser testing on 40+ browsers and operating systems, and Parallel testing to expedite the test execution in a scalable way. In addition, it will help improve our overall visual test coverage by resulting in better visual standpoint product quality as we can cover different combinations using the same test scripts. Other than that, LambdaTest also offers visual regression testing with Selenium.

Starting your journey with Cypress Parallel Testing? Check out how you can test your Cypress test scripts on LambdaTest’s online cloud.

Now execute test scripts faster than any other automation testing grid on LambdaTest’s Automated UI testing cloud.

To get started, you have to install LambdaTest Cypress CLI on your machine. Trigger the following command to install the same:

After installation is completed, set up the configuration using the below command:

Once the command is completed, lambdatest-config.json is created in the project folder. Next, enter the LambdaTest credentials from the LambdaTest Profile Section.

Here is how you can configure the required browser & OS combinations in lambdatest-config.json:

The run_settings section in the JSON file contains the desired Cypress test suite capabilities, including Cypress_version, build_name, visual feedback settings, number of parallel sessions, etc.

Tunnel_settings in the JSON file lets you connect your local system with LambdaTest servers via an SSH-based integration tunnel. Once this tunnel is established, you can test locally hosted pages on all the browsers currently supported by Cypress on LambdaTest.

Now that the setup is ready, it’s time to run the tests; remember that our run_settings file displays the parallels field as five once we trigger our execution in parallel without any extra parameter.

Shown below is the test execution status from the LambdaTest Automation Dashboard.

Here is the execution snapshot, which indicates the progress of test execution:

To summarize, the significant benefit of Cypress visual regression testing on the cloud is achieving optimal visual test coverage without modifying the core logic of the test code to incorporate multiple OS and Browsers.

To view test performance metrics, navigate to the LambdaTest Analytics Dashboard. The Test Overview will provide a snapshot of tests consistent with stable behavior. Meanwhile, the Test Summary will display the total number of tests passed or failed, as well as any completed tests and pending tests.

Also read: How To Perform cross browser testing with Cypress Testing at LambdaTest

With this Visual Testing Tutorial, you will learn how to perform Smart Visual UI Testing on the LambdaTest platform.

You can also use Lambdatest to perform Cypress Visual Testing on the Cloud. Lambdatest compares screenshots pixel by pixel to detect visual bugs and provide a consistent user experience.

Final Thoughts

Visual testing does the heavy lifting to contribute to our UI validation; no matter if you use open source tools or paid tools, visual inconsistencies deeply influence the user journey, which means they need to be eliminated. Performing Cypress Visual Regression Testing can help us validate quickly, correctly, and comprehensively, making it indispensable for any enterprise looking to have an online presence.

“Just because your test automation is working doesn’t mean that it adds value.” — Enrique A Decoss

Remember, we need to change our test automation landscape positively. If you are not including visual testing in your automation test scripts, this is an excellent opportunity to make your test scripts more robust; at the end of the day, automation testing must add value to our customers. Lambdatest allows you to complete automated visual testing more quickly. Compare screenshots using automated visual testing to ensure bug-free delivery! . On a side note you can highlight visual changes to identify errors and deliver pixel-perfect websites through our Screenshot Comparison Tool.

Happy Visual Testing!