Accessibility Test Automation in DevOps Environment

Source – capgemini.com

(1) Background: Accessibility (AX)

Accessibility or AX testing is a subset of usability testing. In accessibility testing, users under consideration include persons with disabilities as well to test both usability and accessibility. Considering the range of difficulties people may be dealing with, it is important to build applications that everyone can use. There standards  have been established to address accessibility need in section 508 of Information and Communication Technology (ICT) and in web contact accessibility guidelines (WCAG).

(2) AX and DevOps

In order to adapt to a continuous delivery in DevOps, AX testing needs to be considered within the early phases of the project. The accessibility testing scope should be finalized at the design phase, followed by test strategy. The tools that can support this should also be considered early in the project lifecycle.

It is generally not possible to automate all the accessibility test cases. The best automated accessibility testing can only verify 25% of WCAG checkpoints and even then, the results are assessed manually. In order to build automated accessibility tests in a continuous integration test suite, only the tests that are 100% objective should be included. Covering 10% of WCAG checkpoints is considered good while automating AX testing.

(3) AX Test Automation in DevOps

It has become essential to test early in the development process to find and correct accessibility conformance issues. This offers the potential to save valuable time and money, especially because it can provide feedback to the development team who can make sure that those issues are taken into consideration as quickly and as early as possible. With automated AX tests, accessibility concerns can be identified and remediated prior to releasing the code to the test team.

In addition, automated testing tools give developers a complete view of accessibility issues like alternative text and color contrast. AX tools recommend the necessary corrections to improve the user experience while adhering to accessibility standards and government regulations (Web Content Accessibility Guidelines, US Section 508, Americans with Disabilities Act).

There are many tools available to help ensure accessibility of applications. Some of these include:

• Web developer toolbars in browsers to disable CSS and check for logical reading orders (the non-CSS view is what will be read with a screen reader)
• Browser add-ons that reveal the underlying accessibility information (like WAVE for Firefox or Chrome)
• Desktop tools like Color Oracle to help simulate colorblindness
• Screen readers that read the screen (people are mostly familiar with JAWS or VoiceOver for a desktop screen reader).  The accessibility options on most smartphones

Many of the tools are simple to use and don’t add a lot of overhead if some checks are done early in the development process.

(4) Standard AX Tools

Some of the industry standard tools in AX space include the following:

The ATF-WAVE web accessibility solution, which –

• Captures the HTML of each unique page of application navigated via WebDriver, then starts the Wave accessibility tool. The Document Object Model (DOM) is then checked to ensure there are no WAVE errors, takes a screenshot of it, and fails the test in the case that there are inconsistencies. This is all done during a dedicated accessibility stage of pipeline construction.

IBM Digital Content Checker –

• Allows anyone to easily upload and verify the accessibility of HTML content or EPUB documents so that clients, employees, and constituents encounter no issues when accessing the information in both web and mobile applications.
• Quickly examines the content, provides a detailed report of all accessibility violations, and then recommends how to fix the issues so that content conforms to standards and regulations before being published.
• Can be customized for an organization’s internal content development workflow so accessibility is part of the review and approval process.

IBM’s Automated Accessibility Tester –

• Incorporates automated accessibility reporting and auditing capabilities directly within the Selenium testing framework. This improves the quality of the testing environment by adding accessibility checkpoints that run during agile development; this allows violations to be corrected in DevOps before deployment.

(5) Conclusion

Accessibility has become a critical focus area for organizations worldwide in order to improve the user experience on any device, optimize communications, satisfy compliance requirements, and to create an inclusive workplace environment. Organizations are finding better ways to integrate accessibility across the entire enterprise – from the design, development, and testing of applications, to any content that is published on websites. Automating accessibility tests ensures earlier test completion faster release to market.