Accessibility Testing: An Essential Guide

“The power of the Web is in its universality. Access by everyone regardless of disability is an essential aspect.” – Tim Berners-Lee, the inventor of the World Wide Web.

Introduction to Web Accessibility

It is challenging enough to deliver flawless web and/or mobile applications, but what makes the task doubly complex is to ensure that the product is accessible to one and all. And that is where Accessibility Testing comes in.

In a recent interview, Paul Smyth, the Head of Digital Accessibility at Barclays, promoted the business case for web accessibility under three major headings:

• The Legal requirements,
• The Commercial Opportunity, and
• The Moral Obligation to provide accessibility.

The World Wide Web Consortium (W3C) defines web accessibility to indicate that websites, web tools, and web-based technology, in general, are designed and developed in a manner that “specially abled” people can use them in a frictionless manner. 

Not only is this a moral obligation, keeping in line with the philosophy of why the internet was created, but it allows a large section of the population to navigate and interact with the web and contribute in a meaningful manner to the web community as a whole.

What is Accessibility Testing?

Accessibility testing deals with testing the features of a web application in a way that ensures all users, irrespective of most disabilities, will be able to interact with the software to its full potential. This includes allowing differently-abled users to perform all key actions without external assistance and making the application intrinsically inclusive for ALL users.

How does Accessibility Testing work?

Accessibility testing can be performed both manually, as well as using test automation tools. It usually depends on checking specific application features against a set of guidelines to ensure that the relevant accessibility criteria have been met. 

For example,

• Testing that the text and labels are universally accessible – This is done by validating the application to see if users will be able to see the text in case it’s increased or decreased in size.
• Testing the contrast ratio for images and text – This validates if the users still are able to interact with the application in the case of the contrast being changed to a greater extent than currently set.
• Testing the Navigation flow of the application – This verifies if a differently-abled user, say someone with a visual impairment, will be able to navigate the full scope of the application.
• Testing the Style Sheets for the application – This helps check if the CSS of the application has a degree of tolerance to ensure that the users will still be able to access the web content if the CSS malfunctions.
• Testing hit areas of the application – This verifies that the major interaction interfaces, i.e., the “hit areas” of the application, meet the accessibility criteria for differently abled users in order for them to perform actions smoothly.

Why is Accessibility Testing important?

The web was created with a vision of removing any obstacles to communication that people face in a physical scenario, and badly designed web tools and technologies exclude a significant portion of the populace from reaping the benefits freely available to their fellow man.

The WHO in a 2020 study, stated that 15% of the entire global population has some form of disability. The Pew Research Centre states that 75% of all US citizens with disabilities use the web on a daily basis.

This market has tremendous fiscal potential as well, with the Return on Disability Group publishing a 2020 report that this segment of the population represents almost USD 13 trillion in annual disposable income when coupled with their friends and families.

Recognizing this fact, governments across the world have put legislation in place to ensure digital content is as accessible as possible. Some landmark acts in this field include:

• Americans with Disabilities Act – 1990
• Disability Discrimination Act – 1995 (UK)
• Disability Discrimination Act – 1992 (Australia)
• Disability Act of 2005 (Ireland)

It is easy for organizations to fall afoul of these acts when it comes to web accessibility and the number of lawsuits under Americans with Disabilities Act (ADA) Title III website accessibility increased to more than 2500 due to companies not providing accessible web services.

Increase in Website Accessibility Lawsuits over the years

Any organization seeking to be inclusive, tap into a relatively under-serviced market, and avoid legal penalties should focus on designing websites that do not pose an interaction barrier to people with disabilities, and this is where accessibility testing comes into the picture.

Advantages of Accessibility Testing

In the present day, there are millions of people who access the web with some form of visual, auditory, and/or mobility impairment. The ultimate success of any web application relies on how well it can be used and navigated by those with such impairments. Accessibility testing not only improves the application’s usability as a whole but acts as a crucial differentiator from the rest of the competition.

Some of the notable advantages of Accessibility Testing are as follows:

• Expansion of the user base

The introduction of accessibility features at a later stage in the software product lifecycle leads to applications containing bugs and is expensive to fix. An important example of the pitfall of ignoring Accessibility Testing is the Microsoft Edge browser. It was initially not accessible to “blind” users, and the efforts to fix accessibility issues at the end of the development cycle have been both expensive and inadequate, leading to certain sections of the general audience preferring to use older Internet Explorer versions or switch to other browsers. Testing for optimal accessibility helps expand the user base in manifold ways and avoids crippling losses in the product life cycle.

• Establishes a benchmark exceeding Compliance Standards

Several governments have established stringent requirements and guidelines for ensuring universal accessibility of ICT services. For instance, Section 255 of the Communications Act of 1934, and Section 508 were updated in 2017 to make sure that Federal US agencies comply with accessibility guidelines during the procurement, usage, maintenance, and development of ICT devices and technology.

However, in order to stay relevant in the industry and beat the competition, simply meeting compliance standards is not enough. An application can be compliant with both standards mentioned above and yet deliver a highly suboptimal user experience that negatively impacts product endorsement and adoption.

Robust accessibility testing ensures that applications exceed standard compliance protocols and put in place user-friendly accessibility options that enhance the overall UX.

•  It enables a Greater Degree of Automation
  

A simple example of a music application can be considered here. If a button isn’t properly configured for automation, it will appear on the screen simply as a “button”. However, a properly configured button will be demarcated as, say, a “play button” using alt text to add descriptive information. This will aid both ease of access and help in easier identification for automated tests.

Limitations of Accessibility Testing

Though accessibility testing is absolutely crucial from a usability and compliance viewpoint in today’s world, there are several limitations that the process suffers from.

• Accessibility tools do not guarantee that compliance regulations are met

Compliance does not guarantee true accessibility, and accessibility testing tools cannot guarantee if ALL compliance standards have been met in any foolproof manner.

It is very much possible that a website has a perfect Lighthouse score and is not optimally accessible due to unintuitive UX. Only real-device testing with actual users can confirm if a site is truly accessible.

Also, testing tools do not provide any guarantee of compliance with the WCAG regulations. Different tools use different algorithms based on the WCAG Version they test against, and even then, the tools differ on the criteria being tested by the application.
Only experts well versed in due diligence in accessibility testing can build the complete picture from different tool results and inform the organization whether they have done enough to ensure optimal accessibility. However, individual tools may not be able to give you the clean chit you so desire.

• Different tools can provide different results

As mentioned before, accessibility testing tools have been developed with different versions of the WCAG criteria in mind, and even they differ in the interpretations of those criteria.

It is important to understand what the desired threshold is and then analyze the results from that viewpoint rather than blindly depend on the results from these tools.

• Significant human intervention is required for decision-making and result validation

Accessibility testing tools help buttress our ability to make decisions on how accessible the application is, but they often lack the sense of context or human insight to make the tests watertight.

For example, an image of a tree with the alt text “tree”, and “Old Oakheart” will pass the accessibility test on any WCAG-compliant tool. However, it is a human judgment call that decides which of these alt texts is actually suitable in the context the image has been used.

How do Enterprises Benefit from Accessibility Testing

Accessibility testing allows enterprises to make sure that their application is accessible to anyone and everyone regardless of a certain set of disabilities, but also provides some other benefits as below:

• Improves overall application performance

Accessibility testing can be considered a synonym for usability testing in most contexts and can help improve the overall ease of use and UX of any product or service.

For example, one of the key WCAG standards maintains that a website needs to be usable with only keyboard commands as this aids a great section of specially-abled people. In a broader context, this leads to a website with an organized navigational information hierarchy and makes the site more intuitive for ALL users, thereby improving UX.

• Improves the overall SEO

The primary goal of SEO is to channel more traffic to the application by improving the site’s ranking on top search engines. Creating apps and websites with cleaner interfaces and navigation helps to highlight the key SEO content and helps stem the bounce rate as it is now more intuitive for visitors to navigate the site.

• Leads to a better-maintained codebase

Applications that prioritize accessibility testing have a better quality codebase in general, as these tools can also identify numerous general issues that hamper the overall usability of the product.

This also forces developers to write cleaner code with fewer bugs, a better UI, and faster load times. 

Thus, accessibility testing can be seen as a wise investment in the product development process rather than just the means to comply with a set of government regulations.

How to perform Accessibility Testing

Accessibility testing can be performed with a variety of tools depending on the criticality of the business case.

For personal sites, there can be free accessibility tools like Google Lighthouse or WAVE by WebAIM that check site content against common WCAG standards and help address the most common accessibility issues.

For more complex sites, there can be proper manual tests performed using Screen Readers that allow users to interact non-visually with digital content.

And finally, automated test tools like the Axe library allow for automated accessibility testing and self-generated reporting of these tests for an optimal experience.

The below sections delve into these methods in greater detail.

Manual Accessibility Testing using Screen Readers

Screen Readers help in converting digital text into an audio format. This helps users to interact with the web application in a non-visual manner and is especially useful for people with sight impairments.

Run Accessibility Test on BrowserStack Live

By loudly enunciating the web text visible on the screen, screen readers assist users in navigating digital content without the need to actually see it.

This is especially useful for those individuals who suffer from partial or complete blindness and/or have cognitive issues like dyslexia.

Screen Readers can work solely with the keyboard present and do not need a mouse to be used. 

On enablement, the screen reader reads out the text on the screen, and different custom commands like jump, skip, select, etc., can be used to navigate the site using the screen reader.

WCAG guidelines under the Quick Reference Guide mandate the inclusion of textual alternatives for those visually impaired, and thus it is vital to validate the scope of auditory navigation on websites using screen readers.

Below is a simple tutorial to test websites with Screen Reader on the Browserstack Live cloud:

Step 1: Sign Up to BrowserStack for a Free Trial and Enter the URL of the website to be tested on the landing page modal. This example will use https://www.browserstack.com/.

BrowserStack Landing Screen

Step 2: Once the website has loaded using the selected browser and device combination, the “switch the browser” option on the left needs to be navigated.

Opening the Left Menu to Access the Screen Reader

Step 3: Out of the list of options available, Select the “Screen Reader” option from the menu. Once the box has been checked, a green banner will appear on the top of the screen mentioning that “Screen Reader” has been enabled.

Screen Reader Activated

This step indicates the completion of the preliminary steps, and the following instructions deal with the actual tests with the screen reader. 

However, some useful commands that require familiarisation for using a screen reader are enlisted below for reference:

• TAB – To move to the next element.
• SPACEBAR – for opening dialog boxes.
• SHIFT + TAB – for navigating the next elements
• ESC – To close menus

Step 4: The next step is to use the TAB command to navigate to the Product Link option and verify if the Screen Reader gives voice to the content on the screen, and aid the tester with further command hints.

In this case, SPACEBAR is pressed to expand the Product option to display the menu as shown below.

Using Screen Reader elements

Automated Accessibility Testing using AXE

Screen Readers are a great way to execute accessibility tests manually. However, these tests may need to be executed in bulk and in a very short span of time, especially when it comes to regression test scenarios.

BrowserStack allows the leveraging of the Axe library – an automated testing tool that validates the entire web document against standard accessibility testing tools and generates automated reports.

Run Automated Accessibility Tests on BrowserStack

BrowserStack Automate allows testers to leverage a variety of scripting languages to achieve this, as shown below:

Step 1: Download the axe.min.js library file.

Step 2: Write a script to test the webpage required in a number of languages, including Java, Python, Node.js, PHP, C#, etc.

The following example uses Selenium and Java.

import org.openqa.selenium.JavascriptExecutor;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.json.Json;

import org.openqa.selenium.MutableCapabilities;

import org.openqa.selenium.remote.RemoteWebDriver;

import java.io.File;

import java.io.FileWriter;

import java.net.URL;

import java.nio.file.Files;

import java.nio.file.Path;

import java.nio.file.Paths;




import com.google.gson.Gson;

import com.google.gson.JsonObject;




public class test{

public static final String AUTOMATE_USERNAME = "YOUR_USERNAME";

public static final String AUTOMATE_ACCESS_KEY = "YOUR_ACCESS_KEY";

public static final String URL = "https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub";

public static void main(String[] args) throws Exception {

MutableCapabilities capabilities = new MutableCapabilities();

capabilities.setCapability("browserName", "Chrome");

capabilities.setCapability("browserVersion", "103.0");

HashMap<String, Object> browserstackOptions = new HashMap<String, Object>();

browserstackOptions.put("os", "Windows");

browserstackOptions.put("osVersion", "10");

browserstackOptions.put("projectName", "BStack Accessibility Test");

browserstackOptions.put("buildName", "BStack Accessibility Sample Build");

capabilities.setCapability("bstack:options", browserstackOptions);





 WebDriver driver = new RemoteWebDriver(new URL(URL), caps);

 driver.get("https://www.google.com");




 JavascriptExecutor jse = (JavascriptExecutor)driver;

 Path path = Paths.get("path/to/axe.min.js");




 String content = new String(Files.readAllBytes(path));

 jse.executeScript(content);




 File output = new File("path/to/report.json");

 FileWriter writer = new FileWriter(output);

 String result = String.valueOf(jse.executeAsyncScript("var callback = arguments[arguments.length - 1]; axe.run().then(results => callback(results));"));

         Gson g = new Gson();

         String p = g.toJson(result);

         writer.write(p);

 writer.flush();

 writer.close();

 driver.quit();

}

}

Step 3: An automated report is generated with the results in a format as shown below. 

BrowserStack Automate Accessibility Test Report

In this nested JSON report:

• Violations indicate the elements that have failed to meet the accessibility rules.
• Passes indicate the elements that have successfully met the accessibility rules.
• Incomplete lists the elements that could not be tested to completion due to technical limitations or system errors. It is these elements that are prime targets for testing using Screen Readers.
• Inapplicable enlists the elements that do not correspond to the WCAG rules and can be ruled out from the scope of accessibility tests.

The Way Forward

In a rapidly developing web community trying to be inclusive to all users regardless of their disabilities, it is imperative that accessibility testing is included in QA operations from the get-go.

Verifying sites as having met the Accessibility Guidelines allows organizations to promote it to people with special needs and also avoid heavy penalties if they fall afoul of the government legislations in place for ensuring an inclusive World Wide Web.

Accessibility tests should take into account the exact real-time user conditions to identify any potential bottlenecks and loopholes and ensure that the user experience has been optimized even for people with special needs. 

Thus, using emulators and simulators for accessibility tests is often insufficient, and real device testing has to be performed for optimal results. 

Since obtaining physical devices to test such features is both expensive and maintenance-heavy, BrowserStack allows the use of a real-device cloud with more than 3000+ browser device combinations with compatibility to Screen Readers and solutions like BrowserStack Automate to ensure frictionless accessibility tests across the board.

Try BrowserStack for Free