Manual Accessibility Testing Methodology (Web & Mobile Apps)

Introduction

Welcome to the Access Squad auditor training curriculum. As an accessibility expert, and particularly as a blind auditor, your role within the software development lifecycle goes far beyond merely running automated tools and generating generic compliance reports. Your true, indispensable value lies in the lived experience you bring directly into the testing process. While automated tools can identify structural HTML flaws in the codebase, they simply cannot tell you what it actually feels like to operate a dynamic website or a complex mobile application using a screen reader on a daily basis.

This comprehensive guide outlines the structured methodology for conducting robust manual accessibility testing across both web platforms and native mobile applications. This approach is specifically adapted from the global W3C standard, WCAG-EM (Website Accessibility Conformance Evaluation Methodology), and fundamentally tailored to leverage the unique perspective, deep assistive technology expertise, and practical workflows of a blind auditor.

Understanding the Methodology: Web vs. Mobile

The Website Accessibility Conformance Evaluation Methodology (WCAG-EM) is a standardized, internationally recognized approach provided by the World Wide Web Consortium (W3C) for evaluating how well a digital product conforms to the Web Content Accessibility Guidelines (WCAG). For the Access Squad, we purposefully adapt this methodology to make it highly practical, highly efficient, and impactful for manual auditors executing real-world audits. This methodology provides a dependable, structured path spanning from the initial scoping of an audit all the way through to final reporting.

Furthermore, while the WCAG framework was originally conceived and written exclusively for the web platform, its four foundational principles—Perceivable, Operable, Understandable, and Robust (often abbreviated as POUR)—apply seamlessly and equally to native mobile applications. However, physically testing native iOS and Android applications requires a slightly different technical approach, shifting focus onto native mobile screen readers (VoiceOver for the Apple ecosystem and TalkBack for Android), complex multi-finger touch gestures, and the proprietary accessibility APIs built into each respective mobile platform.

Step 1: Defining the Evaluation Scope

Before you begin interacting with the target website or mobile app, you must establish and clearly understand the exact boundaries of your audit. What exactly are you evaluating today?

Audit Boundary: Is this an assessment of a complete web portal (e.g., an entire banking site), or is it tightly scoped to just a specific, critical user flow (e.g., the shopping cart and checkout process)?
Platform Scope: Are you testing a native iOS application, an Android application, a Responsive Web platform, or a cross-platform combination of all three?
Conformance Target: What is the specific target level of conformance required by the client or internal policy? (The industry standard and legal benchmark is generally WCAG 2.1 or WCAG 2.2 Level AA).
Assistive Technology Baseline: Which specific combinations of assistive technologies and web browsers will be utilized for the evaluation? (e.g., NVDA paired with Mozilla Firefox on Windows, JAWS with Google Chrome, VoiceOver with Safari on macOS/iOS, or TalkBack on Android).

Taking the time to rigidly define the scope upfront prevents you from getting helplessly lost in massive, enterprise-level applications. This ensures that your audit remains tightly contained within its budget and timeline while consistently delivering highly focused, actionable remediation results for the software development teams.

Step 2: Exploring the Target and Selecting Samples

It is practically impossible and highly inefficient to test every single generated page of a large-scale website or every conceivable edge-case screen of a complex native mobile application. Strategic sampling is therefore an essential pillar of accessibility auditing. According to WCAG-EM principles, you must curate a highly representative sample that adequately covers all the different types of informational content, interactive functionality, and varying UI technologies utilized across the platform. When you first open a target website or launch a mobile app, intentionally resist the urge to immediately hunt for accessibility bugs. Instead, spend the first 15 to 30 minutes simply exploring the digital environment as a regular, goal-oriented user. Try to comprehend the core purpose and business logic of the platform. Navigate sequentially through the primary navigation menus, read through the top-level heading structures to grasp the content hierarchy, and build a mental model of the general interface layout. This preliminary reconnaissance provides invaluable context that will immensely improve the quality of your bug reporting later on.

Selecting the Audit Sample

The specific pages or app screens you formally select for the deep-dive manual audit must deliberately include:

Core Pages and Screens: The platform's Homepage, the login or user registration screen, account dashboards, global settings pages, contact forms, and critical help/support documentation.
Key User Flows: End-to-end interactive processes are the most critical elements of any platform. If auditing an e-commerce site, the complete flow—starting from utilizing the search bar to find a specific product, adding the item to the cart, and successfully completing the final payment checkout—is absolutely vital. You must select and evaluate every single step in that sequential flow.
Diverse UI Components: Ensure your sample purposefully includes pages that demonstrate a wide variety of UI patterns and complexities. This includes pages housing complex data tables, advanced multimedia players (video/audio with captions), modal dialogs (pop-ups), complex date pickers, and dynamic interactive widgets (like tabbed interfaces, carousels, or accordions).
Random Sampling: Deliberately select a few entirely random pages (making up roughly 10% of your total page sample) from obscured corners of the application. Random sampling is a proven technique for catching template-level inconsistencies and fragmented code that developers might have missed outside of their core development flows.

Step 3: Combining Automated and Manual Testing

It is a dangerous, yet common misconception within the tech industry that digital accessibility testing can be fully automated. The reality is that even the most advanced automated testing tools (such as Deque's Axe Core, WebAIM's WAVE, or Google Lighthouse) can only algorithmically detect about 30% to a maximum of 40% of all existing accessibility issues. For instance, an automated scanner can easily flag whether an `` tag is missing its `alt` attribute, but the tool possesses zero semantic intelligence to determine if the `alt` text provided by the developer actually makes contextual sense or provides equivalent meaning to a blind user.

As a highly trained Access Squad auditor, you must view automated tools strictly as a baseline foundation, never as a replacement for human evaluation.

The First Pass: If you have access to accessible, screen-reader-friendly automated testing reports, review them thoroughly before starting your manual testing. This allows you to quickly identify global, systemic issues across the codebase (such as a missing or invalid <html lang="en"> attribute, or widespread, systemic color contrast failures).
Mandatory Manual Verification: You must manually verify the automated findings. Automated tools are known to occasionally produce false positives that require a human to dismiss.
Your Core Value Proposition: The remaining 60-70% of accessibility barriers—such as invisible keyboard traps, illogical and chaotic focus ordering, the complete absence of status context for screen reader users, and highly confusing dynamic DOM changes—can exclusively be discovered through rigorous manual testing conducted by a native expert navigating via assistive technology.

Step 4: Structured Manual Testing (The Core Checklist)

When you transition into the manual audit phase, maintaining a rigorously structured approach is paramount to ensuring consistency and quality across the entire Access Squad. Utilize the following core pillars to dictate and guide your testing methodology on every single sampled page or mobile screen. For a blind auditor evaluating a desktop web platform, keyboard navigation represents the absolute primary method of interaction and traversal. - Complete Reachability: Can you successfully reach every single interactive element (text links, form submit buttons, form input fields, complex custom dropdowns) using strictly the `Tab` key (or specific directional arrow keys for composite widgets like radio button groups or tablists)? - Reliable Operability: Once the keyboard focus is successfully resting on the element, can you predictably activate or toggle the element using the correct standard keystrokes (typically the `Enter` key to trigger links, and either the `Spacebar` or `Enter` keys to activate buttons and checkboxes)? - Focus Traps (WCAG 2.1.2): This is a critical barrier. Does your keyboard focus ever become permanently trapped inside a specific component—most frequently within third-party modal dialogs, promotional overlays, or complex embedded widgets—making it impossible to move focus forward or backward, ultimately forcing the user to forcefully restart their web browser session? - Visible Focus Indicators: While totally blind auditors rely entirely on audio feedback from their screen reader to track the current focus position, WCAG strictly mandates a visible visual focus indicator. You must collaborate closely with sighted or low-vision team members (or rely on automated helper scripts) to definitively confirm that whenever an element receives keyboard focus internally, it also displays a clear, highly visible outline (focus ring) for sighted users relying on keyboard navigation due to motor impairments. - Logical Focus Order: Does the programmatic focus order seamlessly follow the visual, logical reading order of the page? (Generally, this should flow left-to-right, then top-to-bottom for English language platforms). Proficient screen reader users very rarely consume a long web page linearly from top to bottom. Instead, they rely heavily on semantic HTML structural elements to rapidly "skim" and navigate the content. - Headings: By tapping the 'H' key (in NVDA or JAWS), critically evaluate whether the heading structure is built logically and sequentially (starting at H1, descending to H2, then H3, etc.) without skipping structural levels haphazardly. Does every heading accurately and concisely describe the specific content nested beneath it? Ideally, there should be exactly one descriptive `H1` representing the primary subject matter of that specific page. - Landmarks/Regions: Utilize landmark navigation shortcuts. Can you instantly and reliably jump to the main content area ('main'), the primary navigation menus ('navigation'), the site search functionality ('search'), or the global header region ('banner')? Are these ARIA landmarks or HTML5 sectioning elements applied correctly without redundant overuse? - Lists: Check if visually grouped, related items (such as site navigation menus, footer links, or product feature sets) are properly marked up using native HTML list tags (`

`). Semantic lists allow the screen reader to explicitly announce context like: "List with 5 items," providing the blind user with a clear, immediate mental map of the content's size and structure.

C. Forms and Complex Error Handling

Transactional forms, user registrations, and checkout flows frequently serve as the most critical—and most inaccessible—barriers within an application. - Programmatic Labels: Does every single form input control (text input boxes, radio buttons, select dropdowns, checkboxes) feature a programmatic label (`` or `aria-label`) tightly associated with it? When the element receives focus, the screen reader must seamlessly announce both the name of the input and its current role/state. - Clear Instructions: Are mandatory "required" fields communicated verbally by the screen reader, rather than relying solely on a purely visual red asterisk? - Error Identification and Management: If you intentionally submit a complex form with missing data or an improperly formatted email address, how does the system handle and communicate that failure? The absolute best practice dictates that focus should instantly shift to an error summary at the top of the form, or directly to the first offending input field. Furthermore, specific error messages must be programmatically tied directly to their respective fields (often via `aria-describedby`), allowing the screen reader to immediately read out the exact reason for the failure. Simply turning the border of the input field red is entirely inaccessible to a blind user.

D. Images and Multimedia Content

- Meaningful / Informative Images: Do images that convey essential context, data charts, or instructions possess concise, highly accurate, and descriptive `alt` text that provides a fully equivalent informational experience? - Decorative Images: Are purely decorative visual elements, visual spacers, or generic background stock photos deliberately hidden from the screen reader? Developers should achieve this by using empty alt attributes (`alt=""`) or ARIA roles (`aria-hidden="true"`) to significantly reduce unnecessary audio clutter and noise for the user.

E. Custom Widgets and ARIA (Accessible Rich Internet Applications)

In modern web development, engineers rarely utilize raw, native HTML elements for complex UI. They frequently build custom dropdown menus, bespoke modal dialogs, or intricate tab structures using generic `

` and `` tags. This practice inherently strips away all native browser accessibility features. - Name, Role, Value (WCAG 4.1.2): Rigorously validate whether these custom-built UI components have utilized proper ARIA attributes to artificially restore their meaning. Does a custom tab interface actually possess the correct role (`role="tab"`), a descriptive accessible name, and its current selection state (e.g., `aria-selected="true"`)? - Dynamic DOM Updates: If the application dynamically injects new content onto the screen without triggering a full page reload (for example: a shopping cart notification stating "Item successfully added," or a live-updating search results counter), is that purely visual notification properly piped to the screen reader using ARIA Live Regions (such as `aria-live="polite"` or `role="alert"`)?

F. Color Contrast and Visual Presentation

To execute a truly holistic and comprehensive audit at Access Squad, you must account for the needs of users with low vision (partial sight) and color blindness. - Contrast Collaboration: Work synergistically with an auditor who possesses residual low vision or a sighted ally. They will execute specialized tools like the Colour Contrast Analyser to definitively verify that all text meets the strict minimum contrast ratios (4.5:1 for standard body text, and 3:1 for large text or essential UI components). - Text Zooming and Reflow: Verify that when the web browser is zoomed to 200%, the text does not become permanently clipped off-screen, and content blocks do not violently overlap one another, rendering the page entirely unreadable.

Specific Approaches for Native Mobile Apps

Auditing native applications deployed on mobile operating systems demands a significant adaptation of standard web methodologies. While the underlying POUR principles remain constant, the physical mechanics of navigation and interaction shift dramatically on touchscreens. On touch-based interfaces, screen reader users navigate leveraging two primary modalities. As an auditor, you are required to rigorously test both: - Sequential Swipe Navigation: By swiping left and right horizontally across the glass, you force the screen reader to jump linearly from one interactive element to the next. Auditor task: Does the focused sequence of elements follow a logical, predictable interaction flow? Are there critical functional elements (like a 'close' button) that the screen reader bypasses entirely during the swipe sequence? - Touch to Explore: Dragging a single finger physically across the layout of the screen. Auditor task: Do the elements you physically touch announce their names and roles correctly and rapidly? If buttons are visually clustered too tightly or overlapping, does this create a confusing or impossible touch exploration experience?

2. Custom Actions and Gestures

Both VoiceOver (iOS) and TalkBack (Android) rely heavily on specialized "Custom Actions" accessed via vertical swipe gestures (swiping up or down on a single focused element). - Evaluate whether long, repetitive lists (such as an email inbox view) support the use of custom action menus. Instead of rendering three separate, cluttered buttons (Delete, Mark Unread, Archive) next to every single email—which makes sequential swiping exhaustingly long—developers should implement Custom Actions to save the user immense amounts of time and effort. - Ensure that any interaction requiring complex physical gestures (like a three-finger swipe, or holding and dragging a slider precisely) always provides a simplified, single-pointer alternative (like a simple button tap) to accommodate users with severe motor impairments.

3. Element State Announcements on Mobile

Mobile applications are heavily populated with Toggle Switches and Bottom Navigation Tabs. - Verify that when a bottom navigation tab is actively selected, the screen reader does not merely read the tab's name, but explicitly announces the state as "Selected" or "Selected, [Tab Name]". - For Toggle switch components, the screen reader must explicitly announce whether the switch is currently "On" or "Off". If a developer utilizes a custom `

` without applying the proper native mobile accessibility Traits/Roles, the screen reader may only read the label text without ever indicating that the element is interactable or what its current state is. When a pop-up window, alert dialog, or bottom sheet slides onto the screen on a mobile device: - Does the focus of VoiceOver/TalkBack instantly and automatically shift onto the very first element inside that newly opened pop-up? - Ensure that the user cannot accidentally "slip" their focus out of the pop-up by swiping their finger. The background UI (the main screen sitting behind the modal) must be entirely hidden from the screen reader using platform-specific APIs (such as setting `importantForAccessibility="noHideDescendants"` on Android, or ensuring `accessibilityElementsHidden` is applied correctly on iOS).

Step 5: Documenting and Reporting Findings

The technical ability to discover obscure accessibility bugs is only half of your required skill set; reporting them professionally, clearly, and persuasively so that developers and project managers understand them is the other critical half. A bug report authored directly by a blind auditor carries immense weight, authority, and urgency specifically because it highlights real-world User Impact.

For every single accessibility barrier you uncover, your formal report must comprehensively detail:

Specific Location: The exact web URL, or the specific Screen Name / UI Component within the mobile app.
The Core Issue: Explain precisely what technical functionality is failing or defying accessibility expectations.
WCAG Criterion: Map the failure directly back to a specific, measurable W3C standard (e.g., WCAG 4.1.2 - Name, Role, Value).
Steps to Reproduce: Provide crystal-clear, step-by-step instructions enabling a sighted developer to turn on their own screen reader and accurately replicate the exact failure you experienced.
User Impact (The Superpower of Lived Experience): This is where you differentiate yourself from an automated scanner. Do not simply write: "Button is missing an aria-label." Instead, write: "While attempting to finalize payment in the shopping cart, the final confirmation button is announced exclusively as 'unlabelled button'. As a totally blind screen reader user, I lose all context for this critical action, making me terrified of accidentally losing money, ultimately forcing me to abandon the purchase entirely."
Suggested Remediation: Provide concise, accurate technical advice on how to fix the broken code. (For example: "Consider adding visible text inside the <button>Pay Now</button> element, or implement aria-label='Pay Now' if the button must remain a graphical wallet icon.")

Conclusion

Conducting a comprehensive manual accessibility audit as a blind auditor is a highly skilled, deeply investigative process that demands both technical rigor and methodological depth. By merging the formal, standardized structure of the W3C's WCAG-EM guidelines with the unparalleled, nuanced insights of your daily lived expertise using assistive technologies, you empower software engineering teams with a critical perspective that is entirely impossible to generate via automated robotic software tools.

Always remember the core mission of the Access Squad: You are not merely checking off arbitrary boxes on a checklist to achieve legal compliance. The reports and findings you present serve as a fundamental voice of advocacy for digital equity. You are ensuring that modern technology, digital services, and software innovations can be independently enjoyed, accessed, and operated by every single individual, regardless of their physical abilities. Remain meticulous in your sampling methodology, disciplined in executing your core checklist, and deliver your reporting with a powerful blend of user empathy and technical authority. Your dedicated work is proactively shaping the foundational fabric of a vastly more inclusive digital world every single day.

What do you think?

Give your reaction to this article

Manual Accessibility Testing Methodology (Web & Mobile Apps)