Accessibility evidence belongs to the build it was taken on
Automated accessibility tools settle roughly a quarter to a third of WCAG. The rest is judgement: whether alt text describes the picture, whether the focus order makes sense, whether an error message tells anyone what to do. The record of who checked what, and when, is the audit itself, not paperwork wrapped around it. And the field most evidence stores get wrong is the build, not the outcome and not the method.
We hit this the ordinary way. A team keeps a spreadsheet of checks, each with a date and a tester, and a release goes out. Someone asks whether the release is accessible. The spreadsheet can tell you the product was accessible at some point in the past, across some mixture of pages and some mixture of versions. That is a different question, and the difference stays invisible because every row looks equally authoritative. A pass on a page that has since been redesigned is evidence about something else, not weaker evidence about the page that ships today.
A date is not a version#
The instinct is to fix this with freshness: keep checks recent, re-audit quarterly, flag anything older than N months. It does not work, because what invalidates a check is a deploy rather than the passage of time. Two checks a week apart can straddle a release that rebuilt the checkout flow, and two checks eight months apart can bracket a period where nothing in scope changed. Time is a proxy for change, and on a product with a release train it is a bad one.
So in a11ytrail (a small library we maintain, at https://github.com/shipmindlabs/a11ytrail) a check names the build it was taken on, alongside what was checked and by whom:
export type Check = {
/** Success criterion id, e.g. "1.4.3". */
readonly criterion: string;
readonly outcome: Outcome;
readonly method: Method;
/** ISO date the check was performed. */
readonly checkedAt: string;
/** Who performed it, a person or a tool. Anonymous evidence is not evidence. */
readonly checkedBy: string;
/** What was checked: a page, a view, a component. */
readonly scope: string;
/**
* The build the check applies to (a version, a tag, a commit). A result that
* does not name one cannot be tied to what ships today.
*/
readonly build?: string;
readonly tool?: string;
readonly note?: string;
};The build is optional, deliberately. Historical evidence arrives from spreadsheets and PDFs and consultancy reports, and refusing to import it would just push it back out of the system. What stays mandatory is honesty about it: a check carrying an empty string for a build is rejected outright, because a blank that renders as a value is worse than an absent field.
if (check.build !== undefined && !check.build.trim()) {
throw new InvalidCheck(
`check for ${check.criterion} carries an empty build: leave it out rather than recording a build nobody can identify`,
);
}The same validation rejects an automated check that does not name its tool, and a check that does not say who performed it. Every case has the same shape: a field you cannot trace back to something reproducible is not evidence, and letting it in makes the store less trustworthy rather than more complete.
Latest wins is the wrong resolution rule#
Checks are append-only, and they have to be, because correcting a result means recording a later one, and an audit asks what you knew and when. So you need a rule for which check speaks for a criterion on a page. The obvious rule is the most recent one. It breaks as soon as you ask about a specific build.
Suppose checkout was tested for contrast on build 2026.9.0, the release you are about to certify, and it failed. Then someone ran a quick pass on main a week later, after a hotfix, and it passed. Sorting by date hands you the pass. The question on the table is what holds for 2026.9.0, and a result from another build answers something else, probably something more optimistic about code that is not in the release.
So the resolution works in two tiers. Checks taken on the build under assessment are considered first and cover their scopes. Only scopes with no check on that build fall back to their most recent result, which then gets marked stale instead of treated as current:
latestPerScope(criterionId: string, build?: string): readonly Check[] {
const checks = this.for(criterionId);
const onBuild = build === undefined ? [] : checks.filter((check) => check.build === build);
const covered = new Set(onBuild.map((check) => check.scope));
const considered = [...onBuild, ...checks.filter((check) => !covered.has(check.scope))];
const latest = new Map<string, Check>();
for (const check of considered) {
const held = latest.get(check.scope);
if (!held || Date.parse(check.checkedAt) >= Date.parse(held.checkedAt)) {
latest.set(check.scope, check);
}
}
return [...latest.values()].sort((a, b) => (a.scope < b.scope ? -1 : 1));
}Notice the second thing that rule refuses to do: a pass on one page does not speak for another. Recency supersedes within a scope, because that is what re-testing after a fix means. It does not generalise across scopes, because a home page that passes tells you nothing about checkout.
Gaps, defects, and the re-check list#
With the build as part of the question, the output splits into three categories that teams routinely collapse into one.
A criterion checked on the assessed build that failed is a defect, and someone should fix code. A criterion checked on a different build is a caveat, and what it needs is a re-run rather than a ticket for a developer. A criterion nobody checked on a page at all is a gap: nobody looked. Reporting a gap as a failure sends an engineer to fix something that may be perfectly fine, and it inflates the defect count until the whole report is harder to use.
Assessing against a build makes those three visible separately, and turns the second category into a work list:
import { assess, Evidence, statement, toMarkdown } from "a11ytrail";
const claim = assess(evidence, { level: "AA", build: "2026.9.0" });
if (claim.recheck.length > 0) {
// What the release you are about to ship has no evidence for,
// named by criterion and scope, with the build each result came from.
}
const published = statement(claim, organisation, new Date());
if (published.pending.length > 0) {
// The parts no record of tests can answer, listed rather than guessed.
}
console.log(toMarkdown(published));The re-check list is the practically useful artefact. It is the smallest set of checks that would let you say something true about the release in front of you, ordered by criterion and scope, and it is usually much shorter than a full re-audit. It also tells you, honestly, how much of your accessibility position is inherited from a product that no longer exists.
The claim that will not round down#
The last piece follows from the same discipline. WCAG's conformance requirement gives no partial credit: a claim at a level means every criterion at that level is satisfied. A criterion nobody has evaluated leaves the claim incomplete, and incomplete is not one of the statuses a published accessibility statement is allowed to carry. So building the statement from an incomplete claim throws, instead of quietly emitting "partially compliant" because that is the nearest publishable value.
The same refusal applies to the parts of the statement that are judgement rather than record: whether an exemption is a disproportionate burden, what alternative a user is offered, what a failing criterion means for the person in front of it. Those come back as pending and are marked in the output, because a plausible sentence in that slot reads exactly like an answered one.
What it costs to run#
Less than it sounds, and the cost lands in the right place. Recording a build per check is one field in whatever the QA role already fills in, and in CI it is the version string the pipeline already knows. The real cost is cultural. The re-check list will be long the first time, because most existing evidence was taken on builds that have since shipped several times over. That number is not created by the tool. It was true all along, and simply not visible.
The library runs no tests. axe-core, Playwright and a person with a screen reader do that, and building a fifth rule engine would add nothing, since the gap in this field is the record rather than detection. What it does is hold the evidence and tie each result to the build it belongs to, and it refuses to turn an unknown into a claim.
Which is the whole argument, really. Accessibility work goes wrong less often because someone tested badly and more often because a true statement about an old build was quietly reused as a statement about the current one. Naming the build on every check is what stops that substitution from being possible.