betaFeedback

WorksComputer Science

InterA11y: Interactive Web Accessibility Error Detection with LLM Agents

Abstract

Large Language Models (LLMs) have shown potential to expand the coverage of web accessibility evaluation. However, current LLM-based checkers neglect much of the interactive and dynamic web contexts, resulting in limited coverage, high false discovery rates, and inconsistent performance across models. We present InterA11y, an agentic accessibility evaluation system that combines rubrics and interactive tools to support LLM agents in evaluating accessibility effectively. The rubrics decompose WCAG success criteria (SC) into testable aspects and guide agents in collecting and interpreting evidence. The interactive tools inspect or operate a webpage deterministically to collect evidence, such as focus appearances and error messages. We developed these rubrics and tools on 682 curated test cases grounded in established accessibility guidelines that target 10 commonly violated SC. We evaluated InterA11y on 197 expert-annotated cases sampled from 54 real webpages. Results show InterA11y achieves higher recall than the strongest evaluated LLM-driven baseline (79.0% vs. 44.4%), while maintaining comparable precision to best rule-based checkers (93.1% vs. 90.3-97.6%), with little variability between models. Further, when equipped with our interactive tools, general-purpose web agents detect more accessibility errors with fewer false positives and less token usage than without these tools.

CCS Concepts

• Information systems → Web interfaces; • Human-centered computing → Accessibility systems and tools; • Computing methodologies → Intelligent agents.

Introduction

Identifying accessibility barriers in websites and web applications is essential to ensuring equitable web access. The growing use of LLMs in web development makes the need for reliable accessibility evaluation more urgent [23, 37]. Currently, web accessibility evaluation requires substantial expertise [44] and manual efforts [19, 53], because automated rule-based checkers provide limited coverage of guidelines [14, 58] and relying on them alone can be risky [4, 54]. LLM-based checkers have been proposed to address this coverage gap, especially for assessing errors that require semantic understanding of webpage structure and content [17, 25, 34, 41].

Despite these potential benefits, current LLM-based checkers are limited in two respects: (1) lack of dynamic or interactive probing and (2) high variability across models. First, current checkers examine static web contexts, such as HTML, JavaScript, and screenshots. Accessibility evaluation under the four Web Content Accessibility Guidelines (WCAG) principles—perceivable, operable, understandable, and robust—involves examining how webpages behave during user interaction, particularly for criteria related to keyboard navigation, focus management, and feedback [5]. In particular, “operable” targets the accessible interaction of web components but is largely left out by current LLM-based checkers, which have mostly targeted static content such as text alternatives and labels [1]. Second, as we will show through our evaluation of current checkers, they are susceptible to different foundation models when tested on the same detection tasks, with false discovery rates (FDR) varying up to 26.5 percentage points across models. Such variability raises concerns about the reliability of LLM-based checkers in practical web development and testing workflows. Similarly, prior work has shown that general-purpose AI agents also face reliability challenges on several WCAG success criteria [43].

To improve LLMs’ performance in web accessibility evaluation, motivated by prior work where LLM agents are used to identify accessibility errors through interaction [65], we introduce InterA11y, an agentic accessibility evaluation system supported by interactive tools and rubrics. The tools render and interact with a page programmatically, collecting evidence of markups and metadata, rendered appearance, and how it responds to keyboard, focus, and screen reader usage. The rubrics support an LLM agent in interpreting this evidence and running additional tools in multiple turns to determine whether an aspect of a WCAG success criterion (SC) is met. Our rubrics are informed by existing checkers (GenA11y [25]), guidelines [5, 56], and regulations [15, 39]. In this paper, we focus on 10 success criteria that are particularly relevant to interactive web contexts, covering all four WCAG principles, e.g., Non-text Content (1.1.1), Focus Order (2.4.3), Error Identification (3.3.1), and Status Messages (4.1.3).

To develop our system, we curated 682 test cases covering these 10 success criteria. The test set includes 223 cases from the W3C’s Accessibility Conformance Testing (ACT) test cases [56] and 459 AI-generated cases independently validated by the authors. We separately analyze the contributions of rubrics and agentic interactive tools using this test set. Results show that adding InterA11y rubrics over GenA11y improved recall from 31.0% to 64.2%, while reducing false discovery rate (FDR) from 39.2% to 31.1%. Adding interactive tool use further increased recall to 76.4% and decreased FDR to 9.4%—lower than many rule-based checkers.

We evaluated InterA11y with 197 expert-labeled cases sampled from 54 real webpages. We compared against three types of baselines: (1) rule-based checkers, (2) existing LLM-based checkers, and (3) general-purpose web agents. Results show that InterA11y achieves higher recall (79.0%) than the strongest evaluated general-purpose web agent (36.0%) and LLM-based checker (44.4%). InterA11y maintains comparable precision to rule-based checkers (93.1% vs. 95.7%), while general-purpose web agents had much lower precision (∼ 77%). Further, adding InterA11y’s interactive tools to the general-purpose web agents improved both their precision and recall while lowering token usage and execution time.

In summary, we make the following contributions in this paper:

  1. An empirical study of LLM-based web accessibility checkers and a characterization of their failure modes and their variability in detection performance.

  1. The InterA11y system, which combines rubrics and interactive tools to support reliable agentic evaluation of 10 WCAG success criteria across four accessibility principles.

  1. An evaluation of InterA11y, including controlled ablations on 682 curated test cases and comparisons with baseline techniques on 197 expert-labeled cases.

Related Work

Our work is grounded in web accessibility standards and informed by prior work in accessibility evaluation practices, rule-based, LLM-based, and agentic accessibility checkers.

Web Accessibility Standards and Practices

The Web Content Accessibility Guidelines (WCAG), developed and maintained by the World Wide Web Consortium (W3C), is the primary international standard for web accessibility [5]. WCAG is organized around four core principles, known as the acronym “POUR”: Perceivable, Operable, Understandable, and Robust. These principles are operationalized through success criteria, which are commonly mandated by governmental regulations [15, 51, 52].

To guide accessibility evaluation in practice, the W3C developed the WCAG Evaluation Methodology (WCAG-EM), which provides a structured process for assessing accessibility across entire websites [53]. However, manual evaluation remains resource-intensive, requiring trained evaluators and substantial time to assess compliance with WCAG success criteria [21]. Many criteria also require contextual judgment [27] or examination of how users interact with web content [30], making comprehensive evaluation difficult to scale. The need for expert judgment, contextual interpretation, and time-intensive manual evaluation motivates the development of automated approaches that can reduce evaluation effort while supporting a broader range of accessibility requirements.

Rule-Based Checkers and Their Limitations

Automated accessibility checkers reduce the effort required for manual evaluation by identifying potential WCAG violations. Existing approaches can be broadly divided into static checkers, which primarily inspect webpage structure and properties, and dynamic checkers, which additionally execute interactions to examine runtime behavior. Both approaches have improved the scalability of accessibility evaluation, but their coverage remains limited.

Static Checkers

Static checkers such as axe [14], WAVE [58], and HTML validators [40, 46] primarily inspect webpage markup, DOM structures, and element properties to identify accessibility violations [48]. They are fast, easy to integrate into development workflows, and capable of evaluating large numbers of webpages automatically [54]. Some checkers are specifically engineered to limit false positives [14].

Static checkers are constrained by predefined heuristics and static contexts, limiting their coverage in accessibility evaluation [25, 64]. Fischer et al. systematically analyzed widely-used automated accessibility checkers and found that they collectively covered only one sixth of WCAG success criteria [18]. Similarly, He et al. reported that WAVE covers only 13 of 86 WCAG success criteria [25]. Prior research has also documented the tradeoff between their false positives and coverage [54].

Dynamic Checkers

To extend automated evaluation into interactive aspects, dynamic checkers interact with webpages or alter properties to inspect dynamic behavior. By observing how interfaces respond to programmatic actions, these tools can identify accessibility issues that may only become apparent during interaction, such as keyboard navigation failures [3, 7, 8, 9], focus indicator visibility [3], and missing feedback [31]. Another line of work modifies viewport widths to inspect layout [57] and reflow [10] failures.

However, interaction alone does not overcome the limitations of predefined rules. Rule-based dynamic checkers can observe interface behavior, but their assessments are constrained by the conditions encoded in their detection logic and lack of language understanding. Similar limitations have been observed in mobile accessibility evaluation, where rule-based checkers miss errors involving screen reader feedback [64] and interactive functionalities [65].

Static and dynamic rule-based checkers have improved the scalability of automated accessibility evaluation, but both face limitations in coverage and contextual interpretation. Addressing these limitations motivates approaches that combine semantic reasoning with evidence collected through webpage interactions.

LLM-Based and Agentic Checkers

Recent research has explored the use of Large Language Models (LLMs) to extend automated accessibility evaluation beyond predefined rules [1]. For example, GenA11y [25], AccessGuru [17], and López-Gil and Pereira [34, 35] use LLMs to identify web accessibility issues that require semantic understanding, while Gu et al. [21] investigate the use of multimodal LLMs as copilots for web accessibility auditing. Prior research demonstrates how LLM-based techniques can extend accessibility evaluation coverage.

Current LLM-based checkers face similar limitations as static rule-based checkers, in that they mostly ingest static code and screenshots without consideration of dynamic contexts. Recent research has started to explore agentic accessibility evaluation. Mohanbabu et al. [22] observed significantly degraded performance when computer-use agents performed everyday computer tasks using assistive technologies. Santana et al. [43] further demonstrated that successful web task execution and accurate accessibility evaluation are distinct capabilities, and general-purpose web agents remain unreliable at the latter. In mobile apps, TaskAudit built a custom agent to operate screen readers and detect accessibility errors, but are still susceptible to false positives and unreliable task execution [65].

However, dynamic and agentic testing of webpages are less studied and understood. In this paper, we first conduct experiments to understand current challenges, then propose techniques to support more reliable agentic accessibility assessment.

Evaluation of Current Checkers

We begin by examining current rule-based and LLM-based accessibility checkers’ performance on test cases specifically designed to cover various dimensions of WCAG success criteria. Specifically, we selected 10 representative success criteria based on whether they require manual assessments in practice [13] and whether prior work has identified a need for dynamic testing. Table 1 provides a general description of the selected criteria. For these criteria, we curated a set of 682 test cases that are designed to assess checkers’ conformance to WCAG.

SCTitlePrincipleRankReference
1.1.1Non-text ContentPerceivable#6[66]
1.4.1Use of ColorPerceivable#11[11, 24]
1.4.13Content on Hover or FocusPerceivable#21[5]
2.1.1KeyboardOperable#5[3, 7, 8]
2.1.2No Keyboard TrapOperable#29[9, 25]
2.4.3Focus OrderOperable#4[7, 20, 25]
2.4.4Link Purpose (In Context)Operable#16[34] [32]
2.4.7Focus VisibleOperable#7[3, 7]
3.3.1Error IdentificationUnderstandable#22[5]
4.1.3Status MessagesRobust#17[5]

Table 1. Selected WCAG success criteria, with ranks by the number of cases requiring manual testing [13], and relevant references motivating the need for dynamic checking.

Curating a Conformance Test Set

We curate a test set for evaluating accessibility checkers’ conformance to WCAG success criteria by combining relevant ACT test cases and LLM-generated additional test cases. Our conformance test set includes 682 cases in total (223 ACT and 459 LLM-generated), covering 72 rules in 10 success criteria. Each criterion has between 25 and 174 test cases (M=68.2M = 68.2, SD=50.6SD = 50.6). 316 of the 682 cases are positive (46.3%) and 366 are negative.

ACT Test Cases

We first included ACT test cases relevant to the selected criteria. The ACT rules are developed by the W3C to “test conformance with Web Content Accessibility Guidelines (WCAG) 2.2, ... and other accessibility practices” [56]. Although passing these rules does not guarantee WCAG compliance, the rules represent an industry consensus on how WCAG should be implemented [55]. In total, there are 223 ACT test cases covering 14 ACT rules on 6 of the 10 selected criteria.

LLM-generated Test Cases

The ACT only provides a partial coverage of the select criteria, with especially limited coverage on the interactive aspects. We constructed an expansion set to the ACT test cases, following the same design principles: each case should target one aspect of a criterion, be a minimal representation of the requirement, and be easily consumable by testing tools. Two of the authors examined each of the selected criteria, including its official documentation [5], ACT rules [56], assessment standards from governmental agencies [15, 52], and industry best practices [19]. From these standards and guidelines, the two authors individually identified a list of additional rules needed for each criterion, then discussed to reach a final consensus list. In total, the authors identified 58 additional rules across 9 success criteria. While these are unlikely to cover all situations where a person with disability might encounter a barrier, we believe they offer a breadth of coverage that reflects the intent of current standards.

We then used Claude Code [2] with Opus 4.6 to generate passing and failing test cases for these rules. For each rule, the coding agent is supplied with success criteria documentation and relevant excerpts from standards and guidelines. It is prompted to generate a variety of instances where these rules are applicable (e.g., by varying content type, UI component, language, technical implementation).

Four authors independently annotated the generated cases, with each case receiving two annotations. Each annotator was asked to assess, using necessary assistive tools or by examining source code, whether each test case’s label was correct and whether any labels leaked into the test case itself. Annotators received guidelines relevant to each test case’s success criterion. Each annotator first practiced annotations on 20 ACT test cases until they were familiar with the interface and instructions. We excluded any test case where at least one annotator rejected and removed leaked labels from any flagged test cases.

Claude Code initially generated 511 test cases for the 58 rules. The authors each reviewed between 250 and 260 test cases, resulting in 1022 annotations. Annotators agreed on 89.8% of the cases, with Cohen’s κ=0.814\kappa= 0.814 indicating strong agreement. We excluded 52 of the 511 cases where at least one author rejected, resulting in 459 final expansion cases.

Experiment Setup

In this experiment, we ran rule-based checkers and LLM-based checkers from publicly available implementations. Experiments ran in isolated Google Chrome browsers on a Rocky Linux 10.2 server, using 1280 × 900 px viewports, except when resized by tools.

Rule-Based Checkers

We include 11 static and dynamic rule-based checkers: Alfa [45], ASLint [6], axe [14], BAGEL1, [7], Editoria11y [50], HTML_CodeSniffer [46], IBM Equal Access [28], LYNX [60], Nu Html Checker [62], QualWeb [33], and Testaro [29]. Following Pool’s finding that ensemble testing with rules from different checkers improves testing performance [42], we also fit and compute the best rule ensemble2 on our conformance test set, approximating a best possible rule-based checker.

LLM-Based Checkers

We include three LLM-based checkers from prior work: GenA11y [25], which extracts page contents per success criterion and judges accessibility with an LLM; AccessGuru [17], which combines axe and an LLM-based semantic detector; and López-Gil and Pereira’s technique [35], which prompts an LLM with criterion description and explanations. We include these three because of their broad coverage, public availability, and complementary approaches. While the public implementations of all three checkers use older models (e.g., GPT-4o, Gemini 2.0 Flash), we test them using recent models for fairness and consistency. To support the evaluation of variability across foundation models, we test with four models: Claude Haiku 5.5, Gemini 3.7 Flash, GLM 5.3 Flash, and GPT-6 Luna, all with reasoning set to “high.”

Results

Recall and False Discovery Rate

Figure 1a and Figure 1b compare the recall and false discovery rate (FDR=1−precisionFDR = 1 - precision) for all checkers evaluated. The fitted rule ensemble [42] includes 65 unique rules drawn from all eleven rule-based checkers.

Figure 1. Mean recall and FDR of accessibility checkers on the conformance test set. Error bars show ranges across models.

Rule-based checkers consistently scored lower recall on this set that requires dynamic contexts. Surprisingly, many rule-based checkers had high false detection rates, with a mean FDR of 36.4% and only axe achieving a 0% FDR, consistent with earlier benchmarks that found higher completeness to come with lower correctness [54]. The rule ensemble achieved best recall (33.5%) among rule-based checkers, but still had high FDR (30.3%). One possible explanation is that our test set oversamples edge cases and may not reflect real-world accessibility error distributions, but serves as a stress test.

All three LLM-based checkers achieved higher recall than any single rule-based checker, showing LLMs’ strong potential in web accessibility assessment. The rule ensemble approaches or surpasses several models tested. LLM-based checkers still have limited coverage in these criteria, with recall around 30–50%. We also see large performance variability across foundation models. For example, the largest range of recall is 14.6 percentage points (López-Gil and Pereira) and the largest range of FDR is 26.5 percentage points (AccessGuru). A model can perform best in one system, but worst in another. Such sensitivity to models can make it difficult to apply LLM-based checkers in practice.

LLM-Based Checker Error Analysis

For the model with best F1 in each LLM-based checker, we analyzed reasons for its false positives and false negatives. Figure 2 breaks down the causes, which can be broadly categorized into three types: rules and rubrics (#1, #5), evidence (#2, #3, #4), and model output (#6, #7). Across all three checkers, most false negatives were due to missing rubrics in prompts and missing or incomplete evidence. Most false positives were due to incomplete evidence and misinterpreting evidence or rules. Model errors represented a small portion of all errors.

Figure 2. Error reasons of accessibility checkers using their best evaluated models. Each error can map to multiple reasons.

The InterA11y System

We design the InterA11y system to address the two main error sources identified in § 3.3: rubrics and evidence. To quantify and compare the improvements brought by new rubrics and evidence collection techniques, we develop InterA11y based on the closest baseline, GenA11y [25], which analyzes accessibility by success criterion and already collects basic evidence, e.g., HTML contents, CSS properties, and screenshots.

InterA11y is designed to be an agentic system: the agent invokes interactive tools in multiple turns as needed, following the guidance of rubrics. The InterA11y harness supports this process, where it manages the collected evidence, LLM contexts, and browser resources. We re-use GenA11y’s rubrics where applicable but develop our own harness, tools, and rubrics.

InterA11y System Design

Figure 3 illustrates how webpages are assessed through the InterA11y system pipeline. For each success criterion, the harness applies relevant scan tools to collect initial evidence and conducts an LLM scan3 to identify candidate elements. It combines candidates identified through heuristics and the LLM scan, then applies deterministic checks for cases with conclusive measurements (i.e., similar to rule-based checkers). Then, an LLM guided by rubrics evaluates the evidence and collects new evidence using agentic tools as needed. Once sufficient evidence is collected, the model generates a final verdict for each candidate.

Figure 3. Illustration of the InterA11y data pipeline from a Gemini 3.7 Flash run on the 682-item conformance test set. Numbers under each stage indicate the test cases reaching that stage. (1)–(4) show examples of four possible outcomes from the pipeline.

Interactive Tools

Table 2 provides an overview of InterA11y’s interactive tools. The tools can be broadly divided into five categories: (1) inspection of programmatic page information, (2) capture of rendered visual information, (3) measurement of rendered properties, such as contrast, (4) interaction of inputs and controls, and (5) monitoring of page responses to interaction. Although GenA11y [25] included several scan tools, we found them to be insufficient in evidence collection based on our previous analysis and therefore did not adopt them in our system. In our implementation, we pay close attention to reproducing assistive technology output realistically (e.g., screen reader). To this end, we integrate an open-source virtual screen reader built by Guidepup,4 which could be replaced by real assistive technology streams in a production system. Tools are also engineered to accommodate loading times and support robust re-tries. In total, we built 46 scan and agentic tools, including 24 in interaction and monitoring. A list of all tools is available in Appendix § 10.

CategoryScan toolsAgentic tools
Inspection42
Capture33
Measurement64
Interaction182
Monitoring22

Table 2. InterA11y’s scan tools and agentic tools by category.

Rubrics

InterA11y follows rubrics in its analysis of each candidate to reason about available evidence, call necessary tools, and make a decision. Table 3 provides a summary of InterA11y’s rubrics. In authoring the rubrics, we considered WCAG Understanding documents,5 Techniques,6 ACT rules [56], and U.S. [39] and European [15] testing requirements. GenA11y [25] provides

CategoryWCAG SCGenA11yInterA11yTotal
Content alternatives1.1.1549
Link purpose2.4.4314
Keyboard and focus2.1.1, 2.1.2, 2.4.3, 2.4.701111
Cues and feedback1.4.1, 1.4.13, 3.3.1, 4.1.33811

Table 3. InterA11y’s rubrics by category and by source.

a validated starting point for our rubrics development. Of the 35 rubric items included in InterA11y, 11 were originally developed in GenA11y, covering 4 success criteria. However, these rubrics do not include aspects related to interaction, feedback, and tool use. Therefore, we expand them to cover all 10 selected criteria by developing an additional 24 rubric items. Each rubric item decides one aspect of a criterion, specifying evidence required and decision method based on WCAG’s documentations.

Illustrative Examples

Figure 3 shows four exemplar webpage inputs processed by the InterA11y pipeline. Example (1) shows an element excluded by the harness, because it is hidden. Example (2) shows an image without an alt text and is deterministically decided to fail. Example (3) shows an image with proper alt text and is decided by the LLM based on its screenshot without using agentic tools. Example (4) shows UI elements that use color to represent meaning. The rubric for SC 1.4.1 includes the language: “A graphic fails when categories or states are distinguished only by colour...” with available tools listed. The agent thus uses the tool to view a

grayscale version of the elements, then reaches the “fail” verdict. Additional illustrative examples are available in Appendix § 10.

Ablation Study

To understand the contribution of rubrics and interactive tools on InterA11y’s performance, we conducted an experiment with five ablations: (1) original GenA11y [25], (2) GenA11y + deterministic checks, replacing GenA11y’s heuristic tools, (3) GenA11y + InterA11y’s rubrics, (4) GenA11y + deterministic checks + InterA11y’s rubrics, and (5) the full InterA11y system with deterministic checks, rubrics, and agentic tool execution. The experiment was run on the conformance test set described in § 3.1, using the same LLMs and configurations. Statistical analyses were conducted on the case level with two-sided paired permutation tests [59] with Holm–Bonferroni correction [26].

Recall and False Discovery Rate

Figure 1c shows the performance of the InterA11y ablations, while GenA11y’s performance is shown in Figure 1b. While adding deterministic checks improves average recall and FDR by around 10%, the main improvements are from rubrics and interactive tool use. Adding InterA11y’s rubrics improves mean recall from 31.0% to 64.2% (p<.001p < .001) and reduces mean false discovery rate (FDR) from 39.2% to 31.1% (p=.057p = .057). Adding agentic tool use increased recall from 69.6% to 76.4% (p=.18p = .18) and decreased FDR from 26.7% to 9.4% (p<.001p < .001). Adding evidence collected by tools also made the system robust to model differences, reducing the range of recall across four foundation models from 24.4 in rubrics only to 8.2 percentage points (p=.001p = .001) in the full system and the range of FDR from 18.6 to 5.6 points (p=.001p = .001), best among all LLM-based checkers evaluated.

Figure 4 shows ablation performance by rubrics category according to Table 3. Both InterA11y’s rubrics and interactive tools are especially useful for improving recall of “keyboard and focus,” while rubrics alone are useful in improving recall of “content alternatives” and “cues and feedback.” Across all categories, adding interactive tools reduces FDR over rubrics and GenA11y.

Mean recall and FDR across accessibility error categories

Figure 4. Mean recall and FDR across accessibility error categories in the ablation. Error bars show ranges across models.

Error Analysis

Following the same approach as in § 3.3, we analyzed the error reasons of InterA11y’s ablations. Figure 2 compares InterA11y’s error reasons with existing LLM-based checkers. Results show that our rubrics fill a significant gap left by current checkers (#1). Agentic tool use further reduces the errors caused by missing, incomplete, or misinterpreted evidence (#2, #3, #4) from 121 to 35, a 71% reduction. Interestingly, adding tool use removed 30 false positives caused by misinterpreting rules (#5), but introduced 19 false negatives. Overall, the full InterA11y system provides significant improvements to existing LLM-based checkers thanks to improved rubrics coverage and agentic tool use.

Expert Evaluation

Because our initial development and analyses focused on minimal test cases that are less representative of real-world webpages, we conducted an evaluation on 197 expert-annotated cases from 54 real webpages. Informed by Samarasinghe et al.’s recent work on AI agents’ ability to evaluate accessibility [43], we compared InterA11y with rule-based checkers, LLM-based checkers, and two general-purpose agents: Browser Use [38] and AgentLab [12]. These web agents are equipped with generic tools designed for browser-related tasks, such as input, scroll, and form controls. We prompted each agent using the corresponding InterA11y rubrics.

As a follow-up, we introduced InterA11y’s tools to the general-purpose agents and evaluated their performance differences. We also compared token usage and execution time across all checkers.

Dataset Construction

To construct a diverse dataset covering major uses of the web, we sampled 54 webpages from 48 websites in 14 areas, e.g., retail, education, finance, social networking, and public record services. A list of all websites is available in Appendix A. For reproducibility, we saved the webpages’ code and resources. We randomly sampled 300 elements from these saved webpages. An author manually went through the elements and filtered out obviously out-of-scope ones, including videos, elements that are neither visible nor focusable, and elements that require dynamic server resources. In total, we included 211 elements for annotation.

Two experts in accessibility, each with over five years of experience developing or researching assistive technologies, were recruited for this annotation. For each element, a brief description of a success criterion was shown, and the target element was highlighted on the webpage. Each annotator was asked to inspect it regarding the provided criterion. The experts each annotated 103 and 108 elements, respectively. Two authors with prior accessibility research experience separately annotated these 211 elements. In total, each element received annotations from one expert and one author. We compared these annotations and found 14 disagreements. These cases were sent to the other expert for review, and we recorded the majority label. In addition, 14 other cases were found to be unreliable (e.g., webpage contents changed) and were excluded. The final expert-annotated dataset contained 197 unique elements, with 107 positives (54.3%) and 90 negatives.

Experiment Setup

We used the same configurations for rule-based and LLM-based checkers tested in § 3.3, but excluded the checker by López-Gil and Pereira [35], because (1) it was designed for small test cases and did not generate element-level findings; and (2) it sent full HTML in its prompts, exceeding token limits in at least 17 cases.

For Browser Use [38] and AgentLab [12], we used their default configuration. We provided relevant InterA11y rubrics as their initial user message (task). For the conditions with InterA11y tools, we registered our tools using APIs recommended in their documentations. Otherwise, we kept their configurations the same. Statistical analyses were conducted on the case level with two-sided paired permutation tests [59] with Holm–Bonferroni correction [26].

Results

Recall and False Discovery Rate

Figure 5 shows the performance comparison between systems evaluated. For brevity, we only show the best rule-based checker by F1 (axe) and rule ensemble. On these real-world websites, we observe comparable recall between axe and LLM-based checkers, with LLM-checkers averaging 3–4 times higher FDR. Similar to § 3.3, we still see variation

Figure 5. Mean recall and FDR of accessibility checkers on expert-labeled elements. Error bars show ranges across foundation models.

between models, with GenA11y’s recall ranging 23.4 percentage points (pp) and FDR ranging 14.2 pp. Surprisingly, general-purpose web agents, with their many tools targeted at interaction, do not achieve better results on average. Their FDRs are especially high, more than twice that of LLM-based checkers on average. Prior work has shown the unreliability of tool-use preferences in LLMs [16] and having too many tools may have exacerbated this problem.

InterA11y consistently achieves 76.6–84.1% recall across models, with only a 2.0 percentage point FDR range between 5.7% and 7.8%. Its recall is higher than all other checkers tested (all p<.05p < .05) and FDR is similar to axe (2.4%, p=.89p = .89) and rule ensemble (9.7%, p=.92p = .92). Breaking down performance by category, InterA11y achieves almost perfect recall and high reliability for “content alternatives” and “link purpose,” while identifying 50% “cues and feedback” issues when current checkers could detect almost none. In terms of “keyboard and focus,” LYNX [60] and BAGEL [7] rules contribute to recall of 42.9%, but also a relatively high FDR of 25%. InterA11y detects 52.7% of errors at an FDR of only 8.8%. Across all categories, InterA11y maintains an average FDR of less than 10%.

General-Purpose Web Agents

To further understand the effectiveness of InterA11y’s tools, we supply them to general-purpose agents. On average, recall increases from 22.2% to 44.9% with AgentLab and from 36.0% to 42.3% with Browser Use. FDR decreases from 22.7% to 11.7% with AgentLab and from 24.4% to 16.6% with Browser Use. Figure 6 shows a breakdown by accessibility error category. In most categories, introducing InterA11y tools improves or maintains recall while reducing FDR. This is especially prominent for content alternatives and link purpose in AgentLab.

Figure 6. Mean recall and FDR of general-purpose web agents without and with InterA11y interactive tools, by accessibility error categories. Error bars show ranges across models. AL = AgentLab [12]. BU = Browser Use [38].

Token Usage, Cost, Execution Time

We analyze the cost implications of InterA11y in the context of LLM-based and general-purpose agents. Averaged across models, InterA11y spends 31–81% more tokens per case than LLM-based checkers, while improving recall by over 30 pp and reducing FDR. InterA11y spends 14–38% fewer tokens than general-purpose agents. On average, using the cheapest model with InterA11y (GPT-6 Luna) costs $0.0094 per case. As a comparison, the cheapest model with an LLM-based checker (Claude Haiku 5.5 with AccessGuru) costs $0.0030 per case. InterA11y is the fastest agentic checker tested, taking 109 seconds per case on average.

When both general-purpose agents are equipped with InterA11y tools, their token usage and completion time decrease on average.

Discussion

InterA11y presents a novel approach of integrating deterministic tool use and LLM-based context understanding to identify challenging dynamic accessibility problems on the web. Our tool design exposes browser interactions that are relevant to accessibility evaluation, echoing prior findings that web agent interfaces affect effectiveness and efficiency [47, 63]. InterA11y also parallels

Figure 7. Token usage and execution time per case, averaged across all models. Error bars show standard deviation.

recent work in agentic web testing [49], translating high-level requirements into evaluation procedures and grounding judgment in observed content and behavior.

By combining guideline-grounded rubrics and interactive tools, InterA11y provides an agentic solution to reducing false positives, enhancing reliability, and expanding coverage. Our experiments further show that these interactive tools provide reusable evaluation capabilities across different web agents.

Automating these tasks frees up repetitive human efforts that currently require extensive training [39] and time [44], saving human expertise for more impactful tasks. The improvements in false discovery rate and model variability can open up practical applications, such as integrating into CI workflows. Prior work has also shown benefits of sharing agent-collected mobile accessibility evidence to testers [48]. Similarly, developer toolkits can present interactive evidence collected by InterA11y to support bug reporting, triaging, and reproduction of web accessibility issues.

Concurrent work by Mishra et al. [36] similarly uses agents and browser tools, including interaction probes. InterA11y instead organizes tool use around finer-grained rubrics. While their evaluation focuses on page-level agreement, InterA11y systematically evaluates agentic tool use’s contributions to element-level reliability, model variability, and their benefits to general-purpose web agents.

Limitations and Future Work

Due to limited availability of high-quality validated web accessibility datasets, we constructed our own conformance test set, which supported both our development and ablation study. While the ablation still provided useful insights into each system component, future work should explore creating high-quality dynamic web accessibility testing datasets.

Our expert-annotated webpages were collected from real websites, but they are still simplified as offline replications. Future work should explore and evaluate dynamic accessibility testing that covers server-side and authenticated functionalities.

Conclusion

In this paper, we constructed a conformance test set designed for dynamic accessibility errors to characterize the main challenges to current LLM-based checkers, particularly in rubrics and evidence. We then proposed InterA11y, an agentic accessibility checker that combines rubrics and interactive tools to support reliable and accurate detection. InterA11y contributes a viable pathway for detecting dynamic web accessibility errors and ensuring equitable access. Our results motivate the development of reusable tools grounded in guidelines to expand reliable web accessibility testing.

Ethics and Privacy Statement

The expert study was reviewed by our University’s Institutional Review Board (IRB) and received an exempt determination. The implementation of InterA11y, including its prompts, tools, and harness, was supported by AI with extensive guidance from the authors. The experiments and data analysis were supported by AI, with manual verification of data and results by the authors.

References

Details and Examples

We provide additional details of InterA11y and expert study, as well as examples of InterA11y’s execution. Table 4 shows a list of all scan tools and agentic tools and their corresponding success criteria. Table 5 shows a list of all websites examined by experts. Figure 8 shows examples of InterA11y’s correct and incorrect judgments with tool call traces and error reasons.

CategoryScan toolsAgentic toolsSC
Inspectioninventory (links); inventory (images/media); inventory (lookalike glyphs); run (at-rest errors)query_ax_node; resolve_destination1.1.1, 1.4.1; 2.4.4, 3.3.1; 4.1.3
CapturecaptureIndicator; sequenceMap; cropOf (graphics)request_hi_res_crop; set_state_and_capture; capture_full_page1.1.1, 1.4.1; 2.4.3, 2.4.7
Measurementcolours (text contrast); linkCues; styleDiff (state peers); styleDiff (form labels); styleDiff (colour-coded siblings); overlapsOtherContentmeasure_geometry_live; compute_contrast_ratio; resolve_part_color; compare_named_regions1.1.1, 1.4.1; 1.4.13, 2.4.7
Interactionwalk (Tab); walk (Shift+Tab); detectKeyboardTraps; detectFocusRetentionTraps; detectFixedSetConfinementTraps; detectFocusRejection; compositeWalk; activateOnce (keyboard); activateOnce (pointer)keyboardInRevealed; typeInto; hoverReveal; testTrigger (pointer travel); testTrigger (Escape); testTrigger (focus parity); scenario (empty input); scenario (invalid input); scenario (valid input)interact_and_observe; render_with_overrides1.4.1, 1.4.13; 2.1.1, 2.1.2; 2.4.3, 2.4.4; 2.4.7, 3.3.1; 4.1.3
MonitoringidleWatchtestTrigger (persistence)observe_state_after_activation; probe_screen_reader_after_action1.4.13, 2.1.1; 2.1.2, 2.4.3; 3.3.1, 4.1.3
Total3313

Table 4. InterA11y tools and corresponding success criteria.

AreaBase URLs
General Retail and Department Storesamazon.com (3), temu.com
Fashion and Apparelboohooman.com, gymshark.com, www2.hm.com, macys.com, nordstrom.com, reebok.com
Home and Lifestyle Retailwayfair.com
Restaurantsdominos.com, panerabread.com, sweetgreen.com
Travel and Transportiflysouthern.com, hotels.spirit.com, vueling.com
Technology and Electronics Retailnewegg.com
Media and Entertainmentbbc.com, buzzfeed.com, espn.com, nymag.com, rottentomatoes.com
Education and Learningdoomersion.com, kahoot.com, quizlet.com
Finance and Investmentetfdb.com, wallstreetoasis.com, finance.yahoo.com (2)
Real Estateopendoor.com, zillow.com
Social Networks and Communitieslinkedin.com, quora.com, reddit.com (3)
Government and Public Servicesvitalchek.com
Industrial and Agricultural Servicesarterа.com, grazemate.com, morsecorp.com
Business Software and Servicescalendly.com, cloudflare.com, fedl0.ai, drive.google.com, jobs.ashbyhq.com (2), ix.dev, klaviyo.com, notion.com, o11.ai, openroll.com, reacherapр.com, therubric.ai

Table 5. Expert evaluated websites’ URLs. Parentheses show the number of pages included from the domain if greater than 1.

Illustrative examples of InterA11y’s correct and incorrect judgments, showing tool call traces and error reasons.

Figure 8. Illustrative examples of InterA11y’s correct and incorrect judgments, showing tool call traces and error reasons.

References

  1. [1]Wajdi Aljedaani and Rubel Hassan Mollik. 2026. Large Language Models for Web Accessibility: A Systematic Literature Review. In Proceedings of the 23rd International Web for All Conference (Dubai, United Arab Emirates) (W4A ’26). Association for Computing Machinery, New York, NY, USA, 160–171. doi:10.1145/3800424.3800452DOI
  2. [2]Anthropic. 2026. Claude Code. https://claude.com/product/claude-code
  3. [3]Saidarshan Bhagat, Shilpa Oswal, Padmajoshi, and Shubham Gupta. 2026. SATYA: An Automated Framework for Keyboard Navigation and Focus-Indicator Detection for Accessibility Testing Using Computer Vision. In Proceedings of the 23rd International Web for All Conference (Dubai, United Arab Emirates) (W4A ’26). Association for Computing Machinery, New York, NY, USA, 126–130. doi:10.1145/3800424.3800426DOI
  4. [4]Tommaso Calò, Alexandra-Elena Gurita, and Luigi De Russis. 2026. Measuring the Semantic Accessibility Gap in LLM-Generated Web UIs. In Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (Barcelona, Spain) (CHI EA ’26). Association for Computing Machinery, New York, NY, USA, Article 461, 5 pages. doi:10.1145/3772633.379964DOI
  5. [5]Alastair Campbell, Chuck Adams, Rachael Bradley Montgomer y, Michael Cooper, and Andrew Kirkpatrick. 2024. Web Content Accessibility Guidelines (WCAG) 2.2. W3C (2024). https://www.w3.org/TR/WCAG22/
  6. [6]Cezary Tomczyk. 2021. ASLint. Accessed: Oct 1, 2026.
  7. [7]Paul T. Chiou, Ali S. Alotaibi, and William G. J. Halfond. 2023. BAGEL: An Approach to Automatically Detect Navigation-Based Web Accessibility Barriers for Keyboard Users. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 45, 17 pages. doi:10.1145/3544548.3580749
  8. [8]Paul T. Chiou, Ali S. Alotaibi, and William G. J. Halfond. 2021. Detecting and localizing keyboard accessibility failures in web applications. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Athens, Greece) (ESEC/FSE 2021). Association for Computing Machinery, New York, NY, USA, 855–867. doi:10.1145/3468264.3468581
  9. [9]Paul T. Chiou, Ali S. Alotaibi, and William G. J. Halfond. 2023. Detecting Dialog-Related Keyboard Navigation Failures in Web Applications. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). 1368–1380. doi:10.1109/ICSE48619.2023.000120DOI
  10. [10]Paul T. Chiou, Robert Winn, Ali S. Alotaibi, and William G. J. Halfond. 2024. Automatically Detecting Reflow Accessibility Issues in Responsive Web Pages. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (Lisbon, Portugal) (ICSE ’24). Association for Computing Machinery, New York, NY, USA, Article 147, 13 pages. doi:10.1145/3597503.3639229
  11. [11]Ming-Chang Chiu, Shicheng Wen, Pin-Yu Chen, and Xueze Ma. 2026. MegaCoin: Enhancing Medium-Grained Color Perception for Vision-Language Models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 37035–37313. doi:10.1609/aaai.v40i44.41062
  12. [12]Theodore Le Sellier de Chezelles, Maxime Gasse, Alexandre Lacoste, Massimo Caccia, Alexandre Drouin, Léo Boisvert, Megh Thakkar, Tom Marty, Rim Assouel, Sahar Omid Shafiey, Laurenne Keunho Jang, Xing Han Lu, Ori Yoran, Dehan Kong, Frank F. Xu, Siva Reddy, Graham Neubig, Quentin Cappart, Russ Salakhutdinov, and Nicolas Chapados. 2025. The BrowserGym Ecosystem for Web Agent Research. Transactions on Machine Learning Research (2025). Expert Certification. https://openreview.net/forum?id=5298KGmv3
  13. [13]Deque. 2024. The Automated Accessibility Coverage Report. https://accessibility.deque.com/hubfs/Accessibility-Coverage-Report.pdf
  14. [14]Deque. 2026. Ax: Accessibility Testing Tools and Software. https://www.deque.com/axe/
  15. [15]European Telecommunications Standards Institute (ETSI). 2021. EN 301 549 V3.2.1: Accessibility requirements for ICT products and services. ETSI. https://www.etsi.org/deliver/etsi_en/301500_301599/301549/03.02.01_60/en_301549v030201p.pdf
  16. [16]Kazem Fanghian, Weixiao Wang, Yize Cheng, Siddhant Bharti, Gaurang Sriraman, Sriram Balasubramanian, Parsa Hosseini, and Soheil Feizi. 2025. Tool Preferences in Agentic LLMs are Unreliable. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christos Christophidoulopoulos, Tanmay Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 20954–20969. doi:10.18653/v1/2025.emnlp-main.1060
  17. [17]Nadeen Fathallah, Daniel Hernández, and Steffen Staab. 2025. AccessGuru: Leveraging LLMs to Detect and Correct Web Accessibility Violations in HTML Code. In Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’25). Association for Computing Machinery, New York, NY, USA, Article 88, 22 pages. doi:10.1145/3663547.3746360
  18. [18]Thomas Fischer, Björn Lundell, and Jonas Gamielsson. 2025. Coverage of web accessibility guidelines provided by automated checking tools. Universal Access in the Information Society 24, 4 (2025), 3615–3637.DOI
  19. [19]Google Developers. 2025. Test Manual. https://web.dev/learn/accessibility/test-manual. Accessed: April 3, 2025.
  20. [20]Ming Gu, Lei Pei, Sheng Zhou, Ming Shen, Yuxuan Wu, Zirui Gao, Ziwei Wang, Shuo Shan, Wei Jiang, Yong Li, and Jiajun Bu. 2025. Towards an Inclusive Mobile Web: A Dataset and Framework for Focusability in UI Accessibility. In Proceedings of the ACM on Web Conference 2025 (Sydney NSW, Australia) (WWW ’25). Association for Computing Machinery, New York, NY, USA, 5096–5107. doi:10.1145/3696410.3714523
  21. [21]Ming Gu, Ziwei Wang, Sichen Lai, Zirui Gao, Sheng Zhou, and Jiajun Bu. 2026. Towards scalable web accessibility audit with MLLMs as copilots. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 38515–38523.arxiv.org/abs/2511.03471
  22. [22]Ananya Gubbi Mohanbabu, Rosiana Natalie, Brandon Kim, Anhong Guo, and Amy Payal. 2026. A11y-CU: Automated Accessibility Gap in Computer Use Agents. , Article 225 (2026), 26 pages. doi:10.1145/3772318.3791896
  23. [23]Alexandra-Elena Gurita and Radu-Daniel Vatavu. 2025. When LLM-Generated Code Perpetuates User Interface Accessibility Barriers, How Can We Break the Cycle?. In Proceedings of the 22nd International Web for All Conference (Sydney, Australia) (W4A ’25). Association for Computing Machinery, New York, NY, USA, 124–134. doi:10.1145/3744257.3744266DOI
  24. [24]Kazuki Hayashi, Shintaro Ozaki, Yusuke Sakai, Hidetaka Kamigaito, and Taro Watanabe. 2026. Diagnosing Vision Language Models’ Perception by Leveraging Human Methods for Color Vision Deficiencies. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Vera Demberg, Kentaro Inui, and Lluís Marquez (Eds.). Association for Computational Linguistics, Rabat, Morocco, 7582–7605. doi:10.18653/v1/2026.eacl-long.356DOI
  25. [25]Ziyao He, Syed Fatiul Huq, and Sam Malek. 2025. Enhancing Web Accessibility: Automated Detection of Issues with Generative AI. Proc. ACM Softw. Eng. 2, FSE, Article FSE101 (June 2025), 24 pages. doi:10.1145/3729371DOI
  26. [26]Sture Holm. 1979. A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics 6, 2 (1979), 65–70.DOI
  27. [27]Syed Fatiul Huq, Abdulaziz Alshayban, Ziyao He, and Sam Malek. 2023. #A11yDev: Understanding Contemporary Software Accessibility Practices from Twitter Conversations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 217, 18 pages. doi:10.1145/3544548.3581455DOI
  28. [28]IBM Equal Access. 2025. W3C ACT Implementation. Accessed: 2026-07-03.
  29. [29]Jonathan Pool. 2026. Testaro. Accessed: Oct 1, 2026. https://github.com/cvs-health/testaro
  30. [30]J. Bern Jordan, Victoria Van Hyning, Mason A. Jones, Rachael Bradley Montgomery, Elizabeth Bottner, and Evan Tansil. 2024. Information Wayfinding of Screen Reader Users: Five Personas to Expand Conceptualizations of User Experiences. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (St. John’s, NL, Canada) (ASSETS ’24). Association for Computing Machinery, New York, NY, USA, Article 47, 7 pages. doi:10.1145/3663548.3688539DOI
  31. [31]Amy J. Ko and Xing Zhang. 2011. FeedLack Detects Missing Feedback in Web Applications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Vancouver, BC, Canada, 2177–2185. doi:10.1145/1978942.1793337 Anukriti Kumar, Tanushree Padath, and Lucy Lu Wang. 2025. Benchmarking PDF Accessibility Evaluation: A Dataset and Framework for Assessing Automated and LLM-Based Approaches for Accessibility Testing. In Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility (Denver, Colorado, USA) (ASSETS ’25). Association for Computing Machinery, New York, NY, USA, Article 64, 24 pages. doi:10.1145/3663547.3746380
  32. [33]LASIGE, Faculdade de Ciências da Universidade de Lisboa. 2026. QualWeb: an automatic accessibility evaluator. Source code: https://github.com/qualweb/core. Accessed: July 9, 2026. http://qualweb.di.fc.ul.pt/evaluator/
  33. [34]Juan-Miguel López-Gil and Juanan Pereira. 2025. Turning manual web accessibility success criteria into automatic: an LLM-based approach. Universal Access in the Information Society 24 (2025), 837–852. doi:10.1007/s10209-024-01108-zDOI
  34. [35]Juan-Miguel López-Gil and Juanan Pereira. 2026. An empirical investigation of Large Language Models for automated software compliance testing: Evidence from web accessibility. Information and Software Technology 198 (2026), 108215. doi:10.1016/j.infsof.2026.108215
  35. [36]Arjun Mishra, Pranav Karthik, Byungjun Bae, and Dongwook Yoon. 2026. Agentic Web Accessibility Auditing: A Criterion-Specific Framework for Translating WCAG Requirements into Assessments. arXiv:2609.09379 [cs.HC] https://arxiv.org/abs/2609.09379
  36. [37]Peya Mawor, Yi-Hao Peng, Jason Wu, Aaron Steinfeld, and Jeffrey P Bigham. 2025. Codeally: Making AI coding assistants useful for accessible web development. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–15.
  37. [38]Magnus Müller and Gregor Žunič. 2024. Browser Use: Enable AI to control your browser. https://github.com/browser-use/browser-use
  38. [39]Department of Homeland Security (DHS) Accessibility and Usability. 2024. Trusted Tester 5.1. https://github.com/Section508Coordinators/TrustedTester5.1
  39. [40]Pavel Panchaksha, Adam T. Geller, Michael D. Ernst, Zachary Tatlock, and Shoaib Kamil. 2018. Verifying that web pages have accessible layout. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation (Philadelphia, PA, USA) (PLDI 2018). Association for Computing Machinery, New York, NY, USA, 1–14. doi:10.1145/3192366.3192407
  40. [41]Fabio Paternò, Manuela Vinci, Marco Manca, and Nicola Iannuzzi. 2025. How an LLM Can Improve Automatic Web Accessibility Validation?. In Proceedings of the 16th Biannual Conference of the Italian SIGCHI Chapter (CHItaly ’25). Association for Computing Machinery, New York, NY, USA, Article 23, 8 pages. doi:10.1145/3750069.3750310DOI
  41. [42]Jonathan Robert Pool. 2023. Accessibility Metatesting: Comparing Nine Testing Tools. In Proceedings of the 20th International Web for All Conference (Austin, TX, USA) (W4A ’23). Association for Computing Machinery, New York, NY, USA, 1–4. doi:10.1145/3578281.3587282
  42. [43]Vagner Santana, Reinaldo Ferraz, and Jennifer Vieira de Deus. 2026. On the Limits for Web Accessibility Evaluation Using AI Agents. In Proceedings of the 23rd International Web for All Conference (Dubai, United Arab Emirates) (W4A ’26). Association for Computing Machinery, New York, NY, USA, 187–197. doi:10.1145/3800424.3800445DOI
  43. [44]Letícia Seixas Pereira and Carlos Duarte. 2025. Evaluating and monitoring digital accessibility: practitioners’ perspectives on challenges and opportunities. Universal Access in the Information Society (2025). doi:10.1007/s10209-025-01210-wDOI
  44. [45]Siteimprove. 2026. Alfa: a suite of open-source libraries for automated accessibility conformance testing. Source code: https://github.com/Siteimprove/alfa. Accessed: July 9, 2026. https://alfa.siteimprove.com/
  45. [46]Squiz. 2026. HTML_CodeSniffer: a client-side script that checks HTML source code and detects violations of a defined coding standard. Source code: https://github.com/squizlabs/HTML_CodeSniffer. Accessed: July 9, 2026. https://squizlabs.github.io/HTML_CodeSniffer/
  46. [47]Aaron Steiner, Ralph Peeters, and Christian Bizer. 2026. MCP vs RAG vs NLWeb vs HTML: A Comparison of the Effectiveness and Efficiency of Different Agent Interfaces to the Web. In Proceedings of the ACM Web Conference 2026 (Dubai, United Arab Emirates) (WWW ’26). Association for Computing Machinery, New York, NY, USA, 8493–8496. doi:10.1145/3774904.3779283
  47. [48]Maryam Taeb, Amanda Swearngin, Eldon Schoop, Ruijia Cheng, Yue Jiang, and Jeffrey Nichols. 2024. AXNav: Replaying Accessibility Tests from Natural Language. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 962, 16 pages. doi:10.1145/3613904.3642777DOI
  48. [49]Ximen Teoh, Yun Lin, Duc-Minh Nguyen, Roufei Ren, Wenjie Zhang, and Jin Song Dong. 2026. WebTestPilot: Agentic End-to-End Web Testing against Natural Language Specification by Inferring Oracles with Symbolized UI Elements. Proc. ACM Softw. Eng. 3, FSE, Article FSE087 (June 2026), 24 pages. doi:10.1145/3797115
  49. [50]The Trustees of Princeton University. 2026. Editorial11y. Accessed: Oct 1, 2026. https://editorial11y.princeton.edu/
  50. [51]U.S. Department of Justice. 2024. Nondiscrimination on the Basis of Disability; Accessibility of Web Information and Services of State and Local Government Entities. 28 CFR Part 35, Federal Register. https://www.federalregister.gov/documents/2024/04/24/2024-07758/nondiscrimination-on-the-basis-of-disability-accessibility-of-web-information-and-services-of
  51. [52]U.S. General Services Administration. 2018. Applicability & Conformance. Section508.gov. https://www.section508.gov/develop/applicability-conformance/
  52. [53]Eric Velleman and Shadi Abou-Zahra. 2014. Website Accessibility Conformance Evaluation Methodology (WACG-EM) 1.0. W3C Working Group Note. https://www.w3.org/TR/WCAG-EM/
  53. [54]Marek Walgio, Justin Brown, and Vivienne Conway. 2013. Benchmarking web accessibility evaluation tools: measuring the harm of sole reliance on automated tests. In Proceedings of the 10th international cross-disciplinary conference on web accessibility. 1–10.
  54. [55]W3C Web Accessibility Initiative. 2026. About ACT Rules. https://www.w3.org/WAI/standards-guidelines/act/rules/about/. Accessed: 2026-07-03.
  55. [56]W3C Web Accessibility Initiative. 2026. Accessibility Conformance Testing (ACT) Overview. https://www.w3.org/WAI/standards-guidelines/act/. Accessed: 2026-07-03.
  56. [57]Thomas A. Walsh, Gregory M. Kapfhammer, and Phil McMinn. 2017. ReDeCheck: an automatic layout failure checking tool for responsively designed web pages. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (Santa Barbara, CA, USA) (ISSTA 2017). Association for Computing Machinery, New York, NY, USA, 360–363. doi:10.1145/3092703.3098221
  57. [58]WebAIM. 2024. WAVE Web Accessibility Evaluation Tools. https://wave.webaim.org/
  58. [59]Anderson M. Winkler, Gerard R. Ridgway, Matthew A. Webster, Stephen M. Smith, and Thomas E. Nichols. 2014. Permutation inference for the general linear model. NeuroImage 92 (May 2014), 381–397. doi:10.1016/j.neuroimage.2014.01.060DOI
  59. [60]Robert Winn, Paul T. Chiou, and William G. J. Halfond. 2026. Lost in Navigation: Detecting Keyboard Navigation Accessibility Issues in Web Pages. In 2026 IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE Computer Society, Los Alamitos, CA, USA, 682–693. doi:10.1109/ICST69053.2026.00089DOI
  60. [61]Laurence A. Wolsey. 1998. Integer Programming. Wiley, Chichester.
  61. [62]World Wide Web Consortium. 2026. Nu Html Checker. Accessed: Oct 1, 2026. https://validator.w3.org/nu/
  62. [63]Ke Yang, Yao Liu, Sapana Chaudhary, Rasool Fakoor, Pratik Chaudhari, George Karypis, and Huzefa Rangwala. 2025. AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents. In The Thirteenth International Conference on Learning Representations. https://openreview.net/forum?id=oWzdUpOlKxarxiv.org/abs/2410.13825
  63. [64]Mingyuan Zhong, Ruolin Chen, Xia Chen, James Fogarty, and Jacob O. Wobbrock. 2025. ScreenAudit: Detecting Screen Reader Accessibility Errors in Mobile Apps Using Large Language Models. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’25). Association for Computing Machinery, New York, NY, USA, 19 pages. doi:10.1145/3706598.3713797DOI
  64. [65]Mingyuan Zhong, Xia Chen, Davin Win Kyi, Chen Li, James Fogarty, and Jacob O. Wobbrock. 2026. TaskAudit: Detecting Functionality Errors in Mobile Apps via Agentic Task Execution. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26). Association for Computing Machinery, New York, NY, USA, Article 1200, 20 pages. doi:10.1145/3772318.3791415
  65. [66]Mingyuan Zhong, Ajit Mallavarapu, and Qing Nie. 2025. Caption: Generating Informative Content Labels for Image Buttons Using Next-Screen Context. In Adjunct Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology (Busan, Republic of Korea) (UIST Adjunct ’25). Association for Computing Machinery, New York, NY, USA, Article 130, 3 pages. doi:10.1145/3746058.3758413DOI

Paper details

vlnc:W_f4qukxdn
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents