ARIA AT App
Enabling Interoperability for Assistive Technology Users.
The Challenge
Conflicting Screen Reader Experiences Make Web Accessibility Unreliable
Today, different screen readers provide conflicting experiences when presenting a web page, disadvantaging or even excluding some users. These disparities arise from ARIA, the accessibility framework, being interpreted differently across sites, screen readers, and browsers. While the release of ARIA 1.2 in 2019 introduced more stable standards, there was still no reliable way to monitor how consistently ATs and browsers implemented these updates.
The Solution
Empower equal inclusion by realizing interoperability for AT users.
To solve the problem, I helped design the ARIA-AT app, a tool that helps us understand the current state of support for ARIA by Assistive Technologies and build Consensus about Assistive Technologies' behavior. The app provides a user-friendly interface for running and manage tests, viewing results, and understand the current state of support for ARIA, enabling teams to reach Assistive Technology Interoperability.
Efficient Management of Test Plans
The Test Queue allows admins and testers to manage and execute test plans efficiently. Testers can assign themselves a test plan, or start working on one that has already been assigned. It provides an overview of progress and includes options to raise issues directly from each test page during the review and validation process. Once draft test plans are ready, they are added to the queue for community testing.
Seamless Execution of Tests
The Test Run page allows users to execute assigned test plans step by step. During the test process, users can raise issues directly from the page, record results, and track the completion of each test case. This page is essential for validating assistive technology behavior against the established test plans.
Status Summary of Common Web Components Test Plans
The Data Management page provides an overview of the latest test plan versions for common web components and their current status in the ARIA-AT Community Group’s review process. Admins can use this page to manage test plans in the test queue and monitor their progress through various phases.
Detailed Interoperability Reports
The Data Management page provides an overview of the latest test plan versions for common web components and their current status in the ARIA-AT Community Group’s review process. Admins can use this page to manage test plans in the test queue and monitor their progress through various phases.
I paid close attention to every detail in the UI and UX, ensuring a seamless and accessible experience for all users. From intuitive navigation to clear labeling, I focused on creating a design that was not only functional but also inclusive. Every interaction was carefully considered to ensure that users, could easily navigate the app and access the tools they needed.
Approach
Given that ARIA AT was my first project in accessibility, I needed to establish a strong foundation of knowledge. This involved understanding both the app's goals and specific technical concepts that were new to me. To address this learning gap, I opted for a collaborative approach in my design process, fostering an environment of open communication and transparency throughout the project.
Ensuring a shared understanding
I began our research by conducting a series of interviews with the main stakeholders from the W3C community group to gain a better understanding of the vision of the project and learn who and how would be using the ARIA AT app.
I also worked with my team at Bocoup and our stakeholders to identify and coordinate usability studies with testers, screen reader developers and vendors, and accessibility experts. I tested the prototype developed in 2019 with 8 participants.
During these sessions, I was able to collect information that helped me understand pain points in the prototype, overlapping issues, user needs, and areas for improvement.
Building trust through transparency
Before jumping into the ideation phase, I created User Personas, a series of Use Cases and User Journeys to help us map the different scenarios, user goals, and their tasks in order to better identify the pain points and opportunities to improve the existing experience.
I leveraged the team's existing familiarity with Github by using it as a platform to share my work. Every design deliverable was first shared as an issue for feedback. Once incorporated, these deliverables were moved to the project's Wiki, a central repository where everyone could easily reference them.
Ideation
After having our initial research in place, understanding user needs and the requirements for the app, I moved on to create the first set of wireframes.
Addressing Accessibility in the Ideation Phase
During the ideation phase, I encountered a key challenge: ensuring the design process itself was accessible to users with screen readers. Traditionally, some design artifacts, like wireframes, present accessibility barriers due their visual nature. Because this project included stakeholders who are blind, obtaining their valuable feedback was crucial.
Creating Inclusive Wireframes
To address this challenge, I explored different alternatives to make wireframes accessible. Initially, I described them verbally during meetings. However, this approach presented limitations: time constraints and stakeholder availability. Seeking a more asynchronous solution and inspired by how screen readers navigate the web, I developed a framework for describing wireframes using text. A framework I called: “Text-Based Wireframes”, transforming a historically non-inclusive design artifact into an inclusive and accessible one.
I wrote more in detail about this framework in my article “Improving Wireframe Accessibility: A more inclusive Design process”.
Validation
In May of 2020, we ran a pilot test of the first version of the ARIA AT app. The goal of the test was to evaluate the effectiveness of the app in facilitating manual testing and test results collection.
This was a chance to test multiple aspects of the ARIA-AT project at the same time:
- The content of the tests themselves
- The process of enlisting and training testers
- The UX of the app
- The process of resolving conflicts in results between different testers
The Pilot Test successfully collected results across the different AT and browser combinations. In addition, it generated usage data and UX issues that helped prioritize the next phase of work on ARIA-AT and improve future test runs.
In short, the Pilot Test validated the ARIA AT project and paved the way for future improvements.
Beyond Launch
Since its official launch in 2020, the ARIA AT app has undergone significant enhancements across several key areas. User experience has been a top priority, with performance improvements making the app smoother and more efficient to use. We've also expanded the breadth of testing by increasing the number of available tests and supporting a wider range of browser and Assistive Technology combinations for more robust data collection. Finally, in early 2024 I started the process of addressing the long-overdue need for a visual design and branding refresh, creating a more cohesive and user-friendly experience overall.
Impact
The ARIA-AT App has enabled the W3C ARIA-AT community group to increase their testing capacity significantly. This is the first time that the community group has been able to gather interoperability data, which they can use to advocate for the highest-priority accessibility fixes with assistive technology vendors and browser implementers.
Learnings
Working with Meta and the W3C ARIA-AT Community group has been one of the most valuable learning experiences in my career, particularly in the realm of accessibility and inclusion when it comes to design processes.
I’m humbled to have been part of this project and excited for the future of ARIA AT and its positive impact on how assistive technology users experience the web.
