ARIA AT
An app that helps evaluate how consistent the experience for screen reader users is between different Assistive Technologies.
Role
Lead Designer | Research, Ideation, Visual Design, Usability Testing
Client
W3C, Meta
Timeline
2019 – 2024
Overview
People who are visually impaired experience the web differently depending on the assistive technology and browser they are using. This happens because ARIA (a set of attributes that can be added to standard HTML code to make web content more accessible for people with disabilities who use assistive technologies) has not been consistently interpreted across websites, screen readers, and browsers due to a lack of stable standards.
With the release of ARIA 1.2 in 2019, the standards to author accessible experiences were finally stable enough to deliver a consistent experience for assistive technology users.
Later that year, the W3C's ARIA-AT community group approached Bocoup to design a test suite and test planning workflow prototype. The prototype served as the basis for a new testing system that we started developing in 2020 with the goal of gathering data to get buy-in from assistive technology vendors and browser implementers about the most critical accessibility fixes.
My Role
I worked on the project as a lead designer from 2020 to 2024. I was responsible for the research, experience strategy, producing all major deliverables and presenting these to stakeholders.
In early 2024 I began the process of polishing the visual design and defining the branding for the app.
Challenge
We knew stakeholders needed a way to not only collect interoperability data between different browsers and assistive technologies, but also coordinate test authoring and test running with W3C community group members, assistive technology vendors, browser implementers, and spec writers, and a way to facilitate gathering feedback at the different stages of the testing process.
The Goals
Design and develop a test management application that allows:
- Admins to manage tests
- Tests to be run with multiple testers simultaneously
- Results to be reviewed efficiently
- Reports to be published
Approach
Given that ARIA AT was my first project in accessibility, I needed to establish a strong foundation of knowledge. This involved understanding both the app's goals and specific technical concepts that were new to me. To address this learning gap, I opted for a collaborative approach in my design process, fostering an environment of open communication and transparency throughout the project.
Ensuring a shared understanding
I began our research by conducting a series of interviews with the main stakeholders from the W3C community group to gain a better understanding of the vision of the project and learn who and how would be using the ARIA AT app.
I also worked with my team at Bocoup and our stakeholders to identify and coordinate usability studies with testers, screen reader developers and vendors, and accessibility experts. I tested the prototype developed in 2019 with 8 participants.
During these sessions, I was able to collect information that helped me understand pain points in the prototype, overlapping issues, user needs, and areas for improvement.
Building trust through transparency
Before jumping into the ideation phase, I created User Personas, a series of Use Cases and User Journeys to help us map the different scenarios, user goals, and their tasks in order to better identify the pain points and opportunities to improve the existing experience.
I leveraged the team's existing familiarity with Github by using it as a platform to share my work. Every design deliverable was first shared as an issue for feedback. Once incorporated, these deliverables were moved to the project's Wiki, a central repository where everyone could easily reference them.
Ideation
After having our initial research in place, understanding user needs and the requirements for the app, I moved on to create the first set of wireframes.
While testing the prototype, some of the admins expressed having issues with the steps required to start contributing with tests and several testers found executing test plans confusing, so I focused my first efforts on improving the experience for admins and testers (ARIA-AT's primary users).
Addressing Accessibility in the Ideation Phase
During the ideation phase, I encountered a key challenge: ensuring the design process itself was accessible to users with screen readers. Traditionally, some design artifacts, like wireframes, present accessibility barriers due their visual nature. Because this project included stakeholders who are blind, obtaining their valuable feedback was crucial.
Creating Inclusive Wireframes
To address this challenge, I explored different alternatives to make wireframes accessible. Initially, I described them verbally during meetings. However, this approach presented limitations: time constraints and stakeholder availability. Seeking a more asynchronous solution and inspired by how screen readers navigate the web, I developed a framework for describing wireframes using text. A framework I called: “Text-Based Wireframes”, transforming a historically non-inclusive design artifact into an inclusive and accessible one.
I wrote more in detail about this framework in my article “Improving Wireframe Accessibility: A more inclusive Design process”.
Validation
In May of 2020, we ran a pilot test of the first version of the ARIA AT app. The goal of the test was to evaluate the effectiveness of the app in facilitating manual testing and test results collection.
This was a chance to test multiple aspects of the ARIA-AT project at the same time:
- The content of the tests themselves
- The process of enlisting and training testers
- The UX of the app
- The process of resolving conflicts in results between different testers
The Pilot Test successfully collected results across the different AT and browser combinations. In addition, it generated usage data and UX issues that helped prioritize the next phase of work on ARIA-AT and improve future test runs.
In short, the Pilot Test validated the ARIA AT project and paved the way for future improvements.
Beyond Launch
Since its official launch in 2020, the ARIA AT app has undergone significant enhancements across several key areas. User experience has been a top priority, with performance improvements making the app smoother and more efficient to use. We've also expanded the breadth of testing by increasing the number of available tests and supporting a wider range of browser and Assistive Technology combinations for more robust data collection. Finally, in early 2024 I started the process of addressing the long-overdue need for a visual design and branding refresh, creating a more cohesive and user-friendly experience overall.
Impact
The ARIA-AT App has enabled the W3C ARIA-AT community group to increase their testing capacity significantly. This is the first time that the community group has been able to gather interoperability data, which they can use to advocate for the highest-priority accessibility fixes with assistive technology vendors and browser implementers.
Learnings
Working with Meta and the W3C ARIA-AT Community group has been one of the most valuable learning experiences in my career, particularly in the realm of accessibility and inclusion when it comes to design processes.
I’m humbled to have been part of this project and excited for the future of ARIA AT and its positive impact on how assistive technology users experience the web.