Websites today run into hundreds of pages and components. Any minor code change can potentially cause a massive visual bug in a part of the website you might have missed.

In this webinar, Mike Fotinakis, Director of Visual Testing at BrowserStack, and David Jones, Head of Engineering at Percy, discuss how to automate visual testing for your Netlify websites and make code changes confidently. 

Mike and David received several interesting questions during the webinar, but there just wasn't enough time to answer them all. So here's a roundup of questions that we couldn't address during the webinar:

Is it better to club visual validation with functional validation in one automated test case, or is it better to separate them?

There is no correct answer here; both approaches are valid as we have seen users successfully follow both methods. However, if you run them in parallel, you can maintain your test automation cases much more efficiently. 

Can visual testing 'learn' its margin for error? Suppose I typically ignore minor pixel changes on 1000 pages, can Percy adapt to match my tolerance for error? Alternatively, does Percy allow adjustments in pixel tolerance?

Currently, Percy does not use Machine Learning to adjust the margin of error. It makes a pixel-by-pixel comparison of pages and components. However, you can adjust the sensitivity of diffs for your projects to match your error tolerance. 

Can BrowserStack automate the web or mobile app or the browser? Or does it just help run already automated tests on different browsers and platforms?

You might want to look into Automate, a BrowserStack product that can help you run automated tests on a range of browsers, devices, and OSes.

Does visual testing work for native apps?

Unfortunately, Percy does not support visual testing for native apps currently.

Can Percy be used on any computer, or are there specific system requirements?

Since Percy is a cloud-based product, it does not have any additional processing requirements apart from your usual test automation setup. Most of the processing happens on Percy's own infrastructure.

How do you manage cross-device visual testing with Percy?

Percy supports visual testing on all 4 major browsers - Chrome, Firefox, Edge, and Safari. To manage browsers, visit the Browser Management page to toggle browsers on/off. While reviewing diffs, Percy also groups similar changes together so that you don't have to review the same changes again. 

Can you get URLs from a customized and categorized sitemap?

Yes, you can use our CLI to take snapshots from a static directory, snapshots file, or sitemap URL. Please refer to the docs for more details: Percy CLI Snapshot.

Can Percy capture the hover state of the element? Or, for instance, clicking on a button to open a dropdown menu?

Yes, you can recreate the hover state, dropdown menus, etc., in the framework you are using (say Cypress/Selenium/others). You can then capture a snapshot using the Percy Snapshot command.

Does Percy load every page on the website based on the URL provided, or do you have to load them manually? How does the website get imported? 

Both are possible - you could do it manually or add visual testing to your existing test cases in Cypress, Selenium, or other frameworks. You can also snapshot the sitemap using Percy CLI. Please refer to the docs for more details: Percy CLI Snapshot.

What's the workflow like with WordPress?

The documentation on Percy CLI Snapshot will give you a better understanding of how Percy works with a CMS like WordPress. 

If you have a static site generated with Gatsby and React, can Percy predict which pages will be affected if a given component is changed without reviewing every page for every code change? Can you set a custom screen size for checking diffs?

As of now, Percy doesn't have predictive capabilities. It can highlight changes only after running the build across all pages. And yes, you can add up to 10 responsive screen widths for checking diffs; more information on this can be found here: Responsive Visual Testing.

Can a Zeplin screen from the design team be used as the 'old' image for comparison against an implemented page?

No, a Zeplin screen cannot be used as a baseline. Percy is meant for pre-production testing, so it cannot compare between what design and front-end teams have created. 

How do you handle the document ready state when you have async components loading?

Percy waits for all components to load before taking a screenshot. However, if components need a specific amount of time to load, you can add an explicit wait to account for this delay before taking the snapshot. 

Yes, Percy is designed to fit seamlessly into your development workflows and integrates with tools you're already using. Check out the documentation for more details: Percy CI Setup.

What CI integrations does Percy support?

You can leverage your existing CI suite to add visual coverage by simply integrating with your continuous integration service. We support all the leading CI/CD platforms and have built-in support for complex test suites even if your tests run in different test processes or on different machines. Please refer to the docs for more details: Percy CI Setup.

How do you recommend limiting your tests so they are not running all the time and costing too much?

Percy’s per screenshot prices have been optimized so you can focus on visual coverage without worrying about costs. Also, Percy is one of the fastest visual testing products in the market, so speed shouldn’t be an issue with Percy. However, re-evaluating at your test cases frequently for optimization is a must.

Can I sync the baselines with an application's feature branch in my CI/CD tool?

Yes, you can do this. We have extensive documentation on this topic. For baseline selection logic, see Base Build Selection and for setting environment variables, see Environment Variables.

Can you run Percy in the development environment and then share with a QA that doesn't have GitHub?

Yes, you can. Simply share the Percy dashboard with your QA team, and they can review builds directly on the dashboard. You can also integrate Percy with Slack so they are notified each time you push a change.

Does this work with iFrames?

Yes, Percy works with iFrames.

Can you run Percy without CI/CD?

Yes, you can manually trigger Percy builds using Percy CLI. Please refer to these docs for more details: Percy Script Tutorial. However, we strongly recommend integrating Percy with a CI/CD platform for best results.

How much does Percy cost?

Percy has several plans to cover your teams' needs. You can check out our Pricing Page for more details.

Still got questions? If you don’t see your question featured here, contact us today. We’re here to help!