Evaluating accessibility testing in automated software build processes

Abstract

Today, most software projects utilize an automated build process with unit tests, code quality checks, end-to-end integration tests, and more. To make software usable by as many people as possible, regardless of capabilities, accessibility testing must be integrated into the build process. The goal is highly accessible software where issues are found early in the development lifecycle. In this work, we have investigated the most common accessibility testing tools suitable for integration in an automated build process and split the tools into two categories based on their rulesets. The rulesets are evaluated against a well known demonstration site for calculation of the rulesets' precision, recall, and $F_1$ scores. Finally, we discuss the implications of their scores and how accessibility testing tools can be integrated into an automated build process.