Responsive web design lets you make “one website for every screen size”. Any changes you make get deployed across the board, all at once. It’s elegant, practical—and a nightmare to test.

That’s because you’re writing one set of scripts to abstract the differences between:

  • 9,000+ mobile devices.
  • 8 Android and 4 iOS versions.
  • 6 browser engines that power hundreds of browsers.
  • 4 separate versions of Open Graphics Library—and the Metal API for Apple devices (post-2018).

Fragmentation problem gets worse when you add desktops/laptops to the mix.

You’ll need a troop of developers to account for these differences. You’ll also need a large army of testers to find out a.) what’s breaking, b.) on which devices/browsers, c.) why, and d.) how to fix it without breaking the design on a different browser/device.

Through BrowserStack, over 2 million developers run more than 60 million tests every month. We spoke to some of these users to understand how they test responsive websites. Here’s what we found—compiled in a quick checklist for your design, development, and QA teams.

Setting the scope of responsive design testing

If your primary objective is to “render on all browsers/devices”, you might never finish testing your responsive website. Here’s how you create a set of constraints to test responsive websites within a realistic timeframe:

  1. Establish design baselines: You will need to visually compare a web page against its intended design (for every breakpoint). To do this, take screenshots of your website in the browser/platform it was designed and developed on. Or if you have them, keep the design files at hand. You can try this on BrowserStack Screenshots.
  2. Find the most critical browsers and devices for your website: Because you can’t be completely device/platform agnostic, pick the next best option. Build your responsive design for the maximum number of users. To do this, go through the Test on The Right Devices page to find the most popular browser, platforms, and devices in a region. Then pick the devices and browsers that are popular with your site visitors. The aim is to lock down 10-20 device/browser-OS combinations that are popular among your existing users and your potentials. Tip: Prioritize cross browser testing on any browser-OS that gets over 5% share of traffic.
  3. Identify the ‘core’ features that must render uniformly on all of those browsers/devices: An online apparel store won’t launch with a checkout page that’s illegible on smaller screens. But a ‘referrals’ page that breaks on smaller screens wouldn’t block the release. Triage and test the most important pages and features first, then move on to the rest.

Responsive Design Testing: The Ultimate Checklist

With a set of constraints in place, manually test your website on the right browsers/devices. The goal is to find visual differences from the intended design.

Note: Browsers' rendering engines, combined with device capabilities (display resolution, GPU, graphics API, etc.) affect how your website really looks to end-users. To find major inconsistencies before your users do, test your website on real mobile devices before deploying to production.

Responsive design testing: the ultimate checklist

With a set of constraints in place, manually test your website on the right browsers/devices. The goal is to find visual differences from the intended design.

Note: Device capabilities (display resolution, GPU, graphics API) and browsers’ rendering engines—each of these affect how your website looks to end-users. To find these differences before your users do, test on real devices pre-release.

Design assets

When you hear Coca-Cola, you visualize “white cursive letters on red background”. That’s the power of consistency in design.

Color, typography, icons—they are all tied to your brand’s visual identity. But these assets will look different when scaled and resized to fit different viewports. Designers can ensure that this difference doesn’t become jarring for users by testing:

  • Fonts: Some fonts will look jagged when scaled up or down depending on the browser and device. For instance, Windows doesn't scale font-weight uniformly but Mac OS does. Fonts also look a little 'thinner' on Safari as compared to Chrome. Kerning and line spacing might also affect the visual output in different viewports. Test your fonts before incorporating them in your design. Zoom in and out, take screenshots, and compare.

  • Media files: You will need multiple versions of every product or background image to display conditionally (based on device resolution, browser, and viewport). Test these images on slow network connections (with Chrome Developer Tools or similar) to make sure they don't affect website performance (and SEO). Tip: It's also a good idea to check for color inconsistencies in different browsers/devices.

The next section talks about bugs that creep in when you’re converting a responsive design into a functional user interface.

Code

Awesome frameworks like Bootstrap help you quickly build responsive websites. Despite that, there is almost always a button that refuses to align or a widget that breaks upon resizing. You can chalk it up to human error or to browser vendors that are yet to adopt one consistent mechanism for rendering web pages.

Here are some tools that help you build responsive websites for a fragmented browser landscape:

  • Can I use: To see which browsers/device matrices that support a given HTML, CSS, or JS property.

  • CSSLint and ESLint: Helps you write cleaner, more 'cross-browser compatible' CSS and JavaScript, respectively.

  • Normalize.css: Provides base rules for browser-specific default styles.

After you’ve straightened out the code, you’ll have to manually find any visual discrepancies that remain. Here’s what commonly breaks in responsive websites:

  • Layout: Start with the simplest---resize the viewport to see if the layout 'adapts' to fit. In particular, check the following:

    • Alignment: Verify that dynamic elements align and space correctly.

    • Width: Check whether each element's width adapts to the viewport.

    • Height: This is usually unchanged in responsive design, but for some cases, you might specify element height relative to the viewport (like Menu or Edit buttons) on mobile devices. Test this on different mobile browsers and orientations.

    • Spacing and padding: Verify that elements don't overlap.

    • Conditional rendering: Test that elements can appear, hide, and resize based on viewport size, dynamically.

  • Conditional scroll: Test whether input fields and sign up button are viewable/accessible behind the input keyboard on mobile screens.

  • Text: Apart from making sure your font scales and sizes correctly, you should also test the following:

    • Legibility: Your text should be large enough and clear enough to read on any viewport. Test to make sure there are no 'jagged' lines or pixelation on different browsers and devices.

    • Horizontal scroll: Long words/links can create an unintended horizontal scroll on smaller screen sizes.

    • Text input fields: Test the input font size and legibility on browsers and viewports.

  • Responsive images: Quickly test the pages and verify that:

    • the images remain within their containers,

    • right image files are displayed based on device resolution and network speed,

    • product images are expandable,

    • icon-fonts, if used, position correctly on different browsers and viewports.

  • Conditional rendering: If you're using CSS through JavaScript for conditional rendering, test the elements this behavior applies to. For instance, displaying SVG animations on desktop browsers (even if the browser window is resized).

Remember to optimize your CSS, JS, and image files for performance. Minify, compress, and inline wherever possible.

For QA engineers

The checklist above works well for manually testing a small number of responsive pages/screens. What if you have a large web app, and you’re trying to make sure every screen in it is responsive, on a number of devices and browsers, all at once? It can take anywhere between hours to days to test every last page and element, depending on the number of manual testers you have in your team. Human testers can also miss some fairly large inconsistencies (due to ‘change blindness’).

You can save time—and capture a significantly higher number of visual discrepancies—by automating responsive design tests.

This can be done with visual regression testing. You can write test scripts that compare your UI's current appearance (at specified breakpoints) against a set of design files/screenshots. The tests pass or fail if the pixel mismatch is greater than your acceptance threshold. For a quick walkthrough, here's how Hylke de Jong (Wehkamp.nl) used webdriverIO with Browserstack to run a visual regression test.

Sure, responsive design is a nightmare to test, but it allows you to deliver an awesome visual experience—uniformly—across a range of devices. It is the only practical, long-term solution to scale your reach in an increasingly fragmented device and browser landscape.

We hope this checklist helps reduce the time you spend debugging your responsive websites. If you found this helpful, here's a good strategy for cross-browser testing your websites on mobile devices.

Happy testing.

Resources:

https://david-smith.org/iosversionstats/
https://developer.android.com/about/dashboards
https://damieng.com/blog/2007/06/13/font-rendering-philosophies-of-windows-and-mac-os-x
https://www.gsmarena.com/search.php3