Repository navigation
What should "supported" be in ACD? #2916
Description
Accessibility Compat Data is expanding browser support data by including accessibility support. ACD is using existing testing infra (WPT, ARIA-AT) to inform developers about what is/isn't accessibility supported. We've been working with folks in the Interop Accessibility group, ARIA-AT, Baseline and BCD owners with the goal of integrating accessibility support information into Baseline and BCD. This raises a number of questions:
- How do we evaluate support?
- How do we present support status to developers?
- What do we do with web features that are currently Baseline "widely available" or BCD "supported", but are not accessibility-supported?
Activity
lolaodelola commentedon Sep 29, 2026
Summary: What is "Supported"? Defining the ACD Framework
This summary is to give folks a bit more detail about what we're going to be discussing in the seesion.
If you're attending the breakout of the same name, no need to attend this session.
Background
ACD will produce web accessibility data for developer tooling (BCD, Baseline, IDEs, etc.), documentation and AI agents. Tooling doesn't include accessibility in its support data today, but tool owners and developers want it to. The WebDX CG has added it to their roadmap this year.
I'm basing "supported" on the accessibility-supported definition, which I've paraphrased in two parts:
This is why HTML-AAM tests alone aren't enough. Exposing the right role to the accessibility tree doesn't mean the feature works as intended in the browser.
Data sources
Proposed framework
All tests are included, except:
Here's what that gives us for button:
Questions we want to answer
:activeisn't cleared on Tab in theactive-onblurWPTs. That behaviour isn't in the spec, but other browsers implement it and users expect it.run-intests fail everywhere.spectranaut commentedon Oct 19, 2026
Scheduled for Tuesday at upcoming TPAC: https://github.com/w3c/aria/wiki/TPAC-2026#tuesday-oct-27-2026
lolaodelola commentedon Oct 27, 2026
ACD-what-is-support.pdf
ZoeBijl commentedon Oct 27, 2026
Could you please share the keynote version?
css-meeting-bot commentedon Oct 27, 2026
The ARIA Working Group just discussed
What should "supported" be in Accessibility Compat Data (ACD)?.The full IRC log of that discussion
<jamesn> Topic: What should "supported" be in Accessibility Compat Data (ACD)?<jamesn> github: https://github.com//issues/2916
<sarah> scibe+
<sarah> scribe+
<lola> slides: https://github.com//issues/2916#issuecomment-6439733596
<jugglinmike> s/scibe/scribe/
<sarah> now I'm extra scribe
<sarah> [miscellaneous discussion about slide sharing]
<sarah> cyns: we work in tech, really
<sarah> pkra: [misc tech tips about keynote]
<sarah> aardrian: I use skype, then nobody joins and it's great
<sarah> Jamie: not even you
<sarah> aardrian: I like it when higley scribes
<sarah> aardrian: really?? I know she was going to scribe that s**t
<sarah> lola: good morning everybody, apologies for the delay, [intros]
<sarah> lola: I've been working in web standards for a while now, in accessibility for about 4 years now. I'm now an independent contractor and work for myself, and a project I've been working on for over a year is ACD. Also on [?], TAG, a CG
<sarah> cyns: I'm Cynthia Shelley, I know most of you, in ARIA WG for a long time, work on mapping from browser a11y APIs to AT, at Msoft and Google. Love the idea of this project and testing every layer of the a11y stack and making that info available to web developers and browser engs
<sarah> cyns: web develoeprs assume things work and that's not true
<Jamie> s/develoeprs/developers/
<sarah> lola: a11y compat data, what is the goal. We want to produce compat data for devs in dev tooling. Especially IDEs, docs, AI agents. Doing this by pulling in web feature test results from WPTs, test results from ARIA AT, combining to produce an a11y support status for that web feature
<sarah> lola: in case haven't interacted with BCD, they are datasets on MDN for web features. Just say if the feature is there, not if it works. BCD tries to say if it's implemented, but also run existence tests. Compare spec to web IDL and DOM. If they match, then they say full support. That is not enough for a11y, because we need behavioral tests, so
<sarah> we're leaning into WPTs
<sarah> lola: why do want a11y compat data. Browser tooling does not include a11y in its support data .Been trying to work Web? group to create a baseline. BCD doesn't take into consideration any a11y behavior or data. THen you run into problems later when something is fully supported but has a11y issues
<ZoeBijl> s/why do want/why do we want/
<sarah> lola: BCD is how we want to provide this data. ACD needs to show whether a feature is fully supported, partially, or unsupported
<Jamie> s/THen/Then/
<sarah> lola: broken up def of a11y supported used in the WCAG definition. Part of the def of a11y supported is that the user agents that are available to users have the tech to support it natively in those UAs
<sarah> lola: so e.g. a button, BCD says you can create a button in the DOM. DOesn't say you can press, has focus handling, the text that's supposed to be there is displayed. If any breaks, the UA is not supporting the web feature natively. The specs have the reqs of how the button should be in the browser. So want to use a11y supported as a definition to
<sarah> point to so we can be in accord on that, and have a measurement to point to
<sarah> lola: key part is web tech must be supported in the ATs and accross ATs in how users expect to use the feature, and in languages they want to use it
<front-endian-jane> q+
<front-endian-jane> q-
<sarah> lola: handing to cyns for history
<sarah> cyns: this diagram has a user on one side, web on the other, and ocmputer in the middle. All have to work together to make a11y work. If an author has done everything right, if the browser doesn't then it's not accessible
<front-endian-jane> q+ to ask what the threshold is for saying something is supported by ATs widely if there are some ATs that lack support? What are "all ATs" in this context?
<sarah> cyns: WCAG acknowledges this because you can say it's WCAG compliant, but if it's not accesisble on the user's comp, it's not
<sarah> cyns: WCAG 1, dunno if anyone else remembers 1. It had the concept of UAs, says... [reading slide]
<sarah> cyns: squishy words
<jugglinmike> s/UAs/"until
<sarah> cyns: for a while w3c maintained the list of UAs, it's still online and is very entertaining to read
<jugglinmike> s/"until/"until user agents"/
<sarah> cyns: UAG1 was published in 2022
<jugglinmike> s/"until user agents"/"until user agents..."/
<sarah> cyns: [reading slide, definition of UA behaviors, computers]
<sarah> cyns: group disbanded and folded into AGWG
<sarah> cyns: a user agent that follows UAG2 will improve a11y through it's own UI and ability to communicate to other techs
<sarah> cyns: they say UAs but they mean browsers. They don't apply to AT and exclude it elsewhere
<jugglinmike> rrsagent, make minutes
<RRSAgent> I have made the request to generate https://www.w3.org/2026/10/27-aria-minutes.html jugglinmike
<sarah> cyns: WCAG2x has the concept of a11y supported, which is why we're using those words. We want to measure a11y support in browsers, etc.
<sarah> cyns: only a11y supported ways of using tech can be etc.
<sarah> cyns: more reading slide
<sarah> cyns: authors should consult compilations of uses that are documented to be a11y supported
<sarah> cyns: web features are uses. The things you're trying to do, do they work with AT?
<sarah> cyns: on the combos of browsers, AT, etc. that users actually have
<sarah> cyns: some examples of compilations, but they're volunteer run, not update to date, don't align, have to go looking for them. Web devs don't live there
<sarah> cyns: most web devs, if they think abotu a11y, don't go looking for new ways to figure out a11y, they look at some checklist. They assume if it says supported that it's built in
<sarah> cyns: lots of evidence that web devs and business people believe this is a solved problem and it's not
<sarah> cyns: wcag3 has concept of user support sets
<sarah> cyns: reading dev of user support sets
<sarah> s/user support sets/a11y support sets
<jugglinmike> s/dev /definition /
<sarah> cyns: if' you're a public thing, you're thinking about what your users are using from logs, analytics
<sarah> cyns: default set has not yet been defined
<sarah> cyns: informative doc is needed to say how to create their own user support set
<sarah> cyns: long-present bugs in ATs and UAs is still under discussion
<sarah> cyns: want help on how to define this that is understandable, data-driven, complete
<sarah> lola: ACD has two deps, WPTs and AT. WPTs had web features run per browser. Now there is metadata in the suite. The web features is a shared catalog of features that BCD baseline, now platform tests use to align the support for a specific thing
<sarah> lola: eg there is a web features ID for button and a bunch of sub IDs. So you can have button tests across folders, diff part of the test suite but because of metadata you can find all those tests
<sarah> lola: so you can do interesting things to see how BCD aligns, etc
<sarah> lola: WPTs is organized by tests, have sub-tests. Unfortunately not any way to get sub-tests without whole test results, which complicates things a little. If we want one sub-test result in a file with 15 tests, you have to pull in all. That x1000 is a lot of data you end up throwing away, but working on improving that
<sarah> lola: one web feature can have many tests and subtests. Button web feature for example, there are 23 files with 277 sub-tests, so total of 295 tests including HTML AAM tests for button
<sarah> lola: so why not just HTML AAM. GO back to 11y supported, it's not enough that a feature exposes the correct role to the a11y tree, also needs to work in the UA (browser here)
<sarah> lola: ? runs tests in ATs, but not all SRs are covered. Only have VO in Safari, there are also some SRs that only run in some browsers, e.g. can't run JAWS in Safari. But a missing combo doesn't mean a SR can't be run, such as VO in Chrome or FF
<sarah> lola: ARIA AT uses test plans instead of tests, MUST, SHOULD MAY, missing failing or untestable
<jugglinmike> s/? runs/ARIA-AT runs/
<sarah> lola: recently decided to use the web feature catalog in ARIA AT, so can link it with WPTs, BCD data, baseline data for button, to create something beautiful and interesting
<sarah> cyns: many dimensions
<sarah> lola: button test plan, one test plan with 3 behaviors. Consumer API hasn't been built yet, have flexibility in deciding what that is
<jamesn> q?
<sarah> lola: one question to address in this session, how should we combine results to produce support status to devs?
<sarah> lola: I have one proposal. By proposal, I mean it's unproductive to ask all of you what to do
<JohnJansen> css-aam discussion minutes: https://www.w3.org/2026/10/26-aria-minutes.html#8ea6
<sarah> lola: we are in no way married to this proposal, this is just a thing to respond to. Can figure out what works, what doesn't, what to change. So all tests are included except tentative tests, infra failures, timeouts, tests that don't have user impacts. Don't know how to know what has user impacts. Also exclude disputed behaviors, don't want
<sarah> optional to impact status
<sarah> lola: so if we use that framework to decide on button feature, then the button ACD status would be supported in Chrome and JAWS, then partial for NVDA and Chrome because according to the test the button ARIA AT tests, which are drafts, the NVDA support is partial. So even thought hte browser is fully supported in Chrome, the combo is partial
<sarah> lola: then partial in Safari and VO, because according to WPT Safari is partial didn't dig into why. Then Edge and FF even though the browser status is supported and partial, we don't have ARIA AT results so ACD will say not tested. Not sure if that is right to communicate to devs
<sarah> lola: so first question is that using an exclusion framework means everything included by default. Should we do that the other way, only include what we want. That's obviously more maintenance, benefit of default inclusion means not too much maintenenance, but there will be tests we maybe don't care about and shouldn't impact status. Fundamentally
<sarah> we care about tests that have user impacts
<sarah> q+
<jamesn> ack front-endian-jane
<Zakim> front-endian-jane, you wanted to ask what the threshold is for saying something is supported by ATs widely if there are some ATs that lack support? What are "all ATs" in this
<Zakim> ... context?
<sarah> front-endian-jane: question from a while ago, do you have an exampe of a test that doesn't impact devs, but would be included
<spectranaut_> q+
<sarah> lola: political, trying to figure out what I can say. Don't have an example now because I"m of the opinion that most would be. However, there are some people who believe there are some edge cases that are so small that yes we have WPTs but shouldn't impact dev status
<sarah> lola :because liklihood of dev running into this problem is so small
<sarah> lola: but also a11y is a lot of edge cases
<sarah> front-endian-jane: if edge cases means user is unlikely to encounter, but when encountered has a major impact...
<sarah> front-endian-jane: if it's that fuzzy and people disagree, seems safer to expose and let people have diff opinions. If there are people who do benefit from it but it's excluded, they can't use that
<sarah> front-endian-jane: if it is exposed, an org can choose to ignore it. But if not exposed, there isn't that option
<Matt_King> q+
<sarah> cyns: if this were to change the support for the feature in baseline I can see there are some bugs that people say aren't big enough, because partial or not are a big step. But most a11y things I'd put in the must category
<sarah> cyns: but there is a grey area there
<sarah> cyns: but I'd agree put as much in as you can, but if you have a binary status need flexibility
<front-endian-jane> q-
<front-endian-jane> q+
<sarah> lola: just wanted to say to Jane as well, when I say political I mean with the browsers
<pkra> q+
<spectranaut_> q+ handing/sorting subtests by features, browser repairs tests, listing the failing wpt tests
<spectranaut_> q+ to handing/sorting subtests by features, browser repairs tests, listing the failing wpt tests
<sarah> lola: if tomorrow we change the switch, and say all dev support status needs to come from WPT, over 200 web features would change status for browsers, not even including ATs. That obviously chagnes how devs engage with the data
<Jamie> s/chagnes/changes/
<jamesn> ack sarah
<sarah> lola: so if something they've been using for decades goes to partial support what does that mean, can they trust data. And from the browser perspective, that's a negative signal that has a lot of things that have nothing to do with dev status that feed into it. So really getting to grips with what tests to include
<jugglinmike> scribe+
<jugglinmike> sarah: I was going to agree with you about default inclusion because I think there's also incentive for browsers--if something suddenly goes from "supported" to "not supported" to just sort of yank it out
<jugglinmike> sarah: I think you get faster updates if it's default not-included
<Jamie> q+
<jamesn> ack spectranaut_
<Zakim> spectranaut_, you wanted to handing/sorting subtests by features, browser repairs tests, listing the failing wpt tests
<jugglinmike> cyns: It will cause some drama if things change overnight
<sarah> scribe+
<sarah> spectranaut_: I have a lot of thoughts
<aardrian> q+
<sarah> spectranaut_: curious if you've spent time thinking of how to handle sorting between sub tests and features. Lately thinking a lot about tests that are browser repair situations. How does that factor in to feature's actual supportedness. If someone is using the feature wrong, I don't think browsers should get dinged for that
<sarah> Jamie: are those author MUST?
<sarah> spectranaut_: yes, also some browser repairs that are UA MUSTs too
<sarah> spectranaut_: I still wonder, maybe not important
<sarah> spectranaut_: last thing is, not to add complexity, you're communicating a lot of info in tables and I like what you did. If you're already collecting this data, I wonder if you could point to the failing test. So a dev could follow what fails and decide if they care
<Jamie> s/author MUST/user agent MUST/
<sarah> lola: we can talk later about test/subtest stuff, but in a nutshell in the meetings we've been having with interop a11y, that's why we want to rearrange the tests so each subtest in HTML AAM is a test
<sarah> lola: because there's no way to get subtests out. So more test files but smaller tests
<jugglinmike> q+ the practical role of BCD
<jugglinmike> q+ to ask about the practical role of BCD
<sarah> lola: repair stuff is great question, because BCD, even though they do use the spec, but they are not necessarily documenting, saying the thing is according to spec. They do according to implementation with other browsers. So for instance, if a -- asked about source of truth -- if spec says A but browsers agree on B, that's full support
<sarah> lola: so we need to figure out which part of the convo we want to adopt.
<sarah> spectranaut_: failing tests contrib to partial support?
<sarah> lola: prob not to devs. Most devs have not seen WPTs.
<sarah> cyns: it'd freak them out
<sarah> lola: BCD is already a struggle for devs to understand and read. Baseline is also a struggle
<pkra> q_
<pkra> q-
<sarah> lola: devs are not incapable of understanding things [ed: untrue], but we don't want to make things more complex
<jamesn> ack Matt_King
<sarah> cyns: if devs want to find WPTs, they can
<pkra> a+
<pkra> q+
<sarah> Matt_King: about supported, partial, not. I think partially supported is such a wide band, goes from 1% to 99%. Maybe this is a primary point of the issue, does supported need to be perfection, but also... it's such a hard thing if the whole point, the whole thing we want to do is help web devs decide maybe I don't want to use, maybe I want to use
<sarah> a diff feature. Or, the other way they might think about it, this is the feature I"m using, how risky is it
<sarah> Matt_King: in some places I've worked, all they care about is how risky is it
<jugglinmike> s/a diff feature/Matt_King: a different feature/
<sarah> Matt_King: and they'd think partial is good enough. I'm a little concerned we're going to have, as the nature of teh data now. If so many things in broad use, used in apps where at least the owners would like to think everybody to think they're accessible, there's going to be a bit of a...
<sarah> Matt_King: can't set the bar that something has to be supported to be good if 80% of the web features are partial
<lwarlow> q+
<sarah> lola: that's part of what we want to figure out. Maybe supported unsupported is the wrong language, Maybe language about risk levels. On the flip side, defining what tests are dev impact tests. It's possible, I showed this button table, and mentioned there are 295 button tests. I really really doubt we care about all those tests. If we only care
<sarah> about 3 button tests, I don't think it's out of this world to say all three need to pass. And perfection is those 3 passing, not 295
<sarah> lola: but we need to agree on the framework for deciding which 3 of the 295 matter
<sarah> spectranaut_: I say you just choose which ones
<sarah> Matt_King: in ARIA AT, if you look at our defs of must should may, the bar for must is actually really high, intentionally. If this assertion is not satisfied then it would be impossible for the user to use the feature. However there is a ton of really important stuff that moves to should. I'm concerned we'd draw the line strictly because the line
<sarah> for must is so high
<sarah> Matt_King: so there's certain ARIA features like posinset and setsize where all assertions are should all the time because it's possible to use without it. But so many SR users rely on that in diff ways
<jugglinmike> s/for must/Matt_King: for must/
<sarah> Matt_King: so that's how we set it, maybe it would twist our arms to reconsider that. But I hate getting into things, subjective stuff where the lines are not crisp, the lines are a lot harder
<sarah> lola: should usnpecified behavior be excluded? We currently include it. This goes to spectranaut_'s question. We include unspecified behavior if it's what users expect. E.g. FF has partial support because active is not cleared on tab
<sarah> lola: should tests that fail in every AT or every browser be excluded?
<sarah> lola: example of test that fails in every browser onscreen
<sarah> lola: Edge and FF don't have AT data for ARIA AT, what should that show
<sarah> lola: I'm doing a lot of work talking to diff groups and I'm tired
<sarah> lola: would be great if there was a CG, a task force, part of the ARIA WG. What should the space be? I would like a diverse participant pool, so I'm going to everyone relevant
<sarah> lola: there are known issues like FF and Edge. We don't have data for other ATs like VC, switch
<sarah> lola: dev understanding issues too [ed: again, devs are the problem]
<sarah> cyns: we're testing the computed a11y, testing the SR behavior, not testing APIs
<sarah> Matt_King: that's covered by SRs
<sarah> cyns: other ATs use APIs
<aardrian> q?
<jamesn> ack front-endian-jane
<jamesn> qv?
<sarah> front-endian-jane: going back a sec, it does feel like from a dev perspective, it would be useful to have split support in two levels. Supports as in totally works, and supports as in you can use it but there are issues that won't block. Where it's not strictly necessary, so still kinda supports so would be nice to have that info there
<sarah> front-endian-jane: so you can have tests that feed into that but doesn't drop to partial supports. In my ex, that's a useless label
<aardrian> removing myself from queue, since I was only going to ask for if / when SHOULD could / would be considered. Matt and Jane covered it.
<sarah> front-endian-jane: is there a long-term plan to add other ATs?
<aardrian> q-
<sarah> cyns: yes
<sarah> front-endian-jane: would be especially nice to have the ATs that don't play nicely, the ones that do DOM and don't listen to APIs. No one is testing those in their app and have no idea that a lot of features are unsupported there. Right now if a dev sees this they'd thing SRs are all there is to a11y, and this confirms that bias
<sarah> front-endian-jane: lots of devs think all ATs play nice
<jamesn> ack Jamie
<sarah> front-endian-jane: doing that would be nicer than pulling in more ATs that use the same APIs
<Matt_King> q?
<Matt_King> q+
<sarah> Jamie: I think you were saying something about dealing with test MUST?
<sarah> lola: that's for the ARIA AT, not WPT
<aardrian> ...and Jamie is hitting SHOULDs as well.
<front-endian-jane> q+ to mention that showing things red across the board is helpful.
<sarah> Jamie: Even in ARIA the bar is high for UA must, role exposure, some of those are shoulds. It's tricky because I could see this being a problem in either direction. Part of me wants to say only UA musts, but...
<sarah> lola: for AT side, we could include the shoulds. I think ARIA AT is easier to work with than WPTs, because for the most part the group is Matt and a few volunteers
<sarah> lola: if I talk to Matt long enough I can convince him, I think
<sarah> Matt_King: lol no
<sarah> Jamie: I bring this up because there are a11y WPT tests as well, is there some way we can tag those as well
<sarah> lola: we should talk because this idea of tagging and labelling has come up so much in the last day with diff opinions
<sarah> Jamie: if there's a tech problem blocking this...
<sarah> cyns: more of a political problem
<sarah> Jamie: we have control over WPT, there are people who understand it well
<jamesn> ack jugglinmike
<Zakim> jugglinmike, you wanted to ask about the practical role of BCD
<sarah> spectranaut_: org(?) is paying for improvements to WPT
<jamesn> zakim, close the queue
<Zakim> ok, jamesn, the speaker queue is closed
<spectranaut_> s/org(?) /SCOBB/
<sarah> jugglinmike: asked about venue for dispute resoution. I think that could be WPT but only if you're aggressive in ignoring politics behind these things. If folks recognize the public implication of failures, that'd motivate them to participate more actively. If WPT is the source of truth, they should go to that
<jamesn> ack pkra
<sarah> jugglinmike: could take the form of a WPT staging zone where folks could get notifications before things get merged of impacts to status. That's a lot of infra to build, maybe you go there immediately, maybe the staging zone is the PR, and the end result is getting poeple more engaged in WPT
<sarah> pkra: most has been asked, one thing that hasn't -- you mentioned devs struggle understand BCD. I'm wondering what research you've seen or done around that problem
<sarah> pkra: I only always see at minimum self-selecting research, so you know, so the stupid end of year surveys, blah
<sarah> lola: so there are two surveys run this year. One is not a state of survey, but on MDN. So asking see this banner, the baseline banner, what do you think it means
<sarah> lola: do you think it means the feat is widely available, does it include a11y, were a bunch of options. 30% didn't know what the banner was. That was enough for group maintaining it to say oh no
<sarah> lola: ran quick survey on thurs, just to get a quick idea. You see this full support on BCD, this is how BCD defines full support, what do you assume it means. Options were avail in browser, matches spec, etc
<sarah> lola: 88% interpreted it as implemented according to spec. Which isn't true. Folks doing BCD have more info on historical stuff
<jamesn> ack lwarlow
<sarah> lwarlow: I think for parital support, it would be valuable to have another status of broken rather than not supported. Example -- accent color on pro for android has no contrast algo because someone at google broke it, and it never got caught till I raised a bug. But in BCD it says partial support instead of no support.
<sarah> lwarlow: having a status for your feature detection may say it's supported but it's broken would be nice
<aardrian> +1 to "Broken"
<sarah> lwarlow: for conflict resolution, I think you've looked into this, I've said this should be partial support because it's broken, then someone else comes along and says it's unfair there should be a separate feature
<sarah> lwarlow: I think BCD doesn't use subfeatures enough, I think this could use it more
<jamesn> q- Matt_King
<jamesn> q- front-endian-jane
<sarah> lola: in terms of conflict resolution, I meant which group. BCD has its own meeting and we could do similar, just want to make sure we have the right people in the room
<jamesn> zakim, open the queue
<Zakim> ok, jamesn, the speaker queue is open