Accessibility in CI/CD: what to check on every release and what requires human review
Accessibility checks in continuous integration and delivery (CI/CD) can flag some new barriers before a release. A passing automated check, however, does not confirm WCAG conformance or show that disabled people can use the website successfully. A useful setup therefore combines fast machine checks, deliberate triage and a plan for human evaluation.
Why scanning one page is not enough
Tools such as axe-core and the IBM Equal Access Accessibility Checker evaluate rules that can be determined reliably from the code and rendered interface. They may identify, for example, a control without an accessible name, invalid use of selected ARIA attributes, or certain contrast problems.
They cannot fully assess the meaning of content, the clarity of a process, or whether a screen-reader user can comfortably finish a task. W3C therefore states explicitly that an automated tool cannot determine accessibility or conformance on its own. A green pipeline means only that the selected machine-detectable violations were not found in the state that was tested.
Dynamic states pose another limitation. A closed menu, a modal, errors after form submission, or the next checkout step may not exist in the initial HTML. The check must run after the interaction that actually displays the relevant state. This is one reason why an accessibility scanner alone is not enough.
Divide checks by speed and risk
Accessibility need not be one large task at the end of a pipeline. A practical arrangement can use three layers:
- When a component changes, run fast rules against its representative states. The developer receives feedback on the button, form or dialog while the change is still fresh.
- During an integration or browser test, open important application states and run the selected engine in each one. Include an empty and completed form, a validation error, an open modal and a signed-in state.
- Before a release or on a schedule, exercise a broader sample of key journeys. Align this layer with what the team already runs in CI/CD rather than creating a separate, disconnected pipeline.
Every run needs a defined URL, state, DOM scope, ruleset version and expected outcome. Without these, a new barrier cannot be distinguished from a tool-configuration change.
A baseline must not become a dumping ground
If the website already has known findings, immediately blocking on every result could stop every release regardless of the current change. A temporary baseline—a recorded starting state—can highlight new findings. It must not turn old ones into permanent exceptions.
Give each exception an identifier, reason, owner and review date. When the tool or ruleset version changes, compare results in a controlled run first; a new or more precise finding is not automatically a pipeline defect. Broad selector-based suppression is risky because it may hide future problems in the same part of the page.
Decide what blocks a release
A blocker should be a confirmed, reproducible new finding within the agreed scope, particularly in a critical journey. A tool’s severity label alone is not enough. The team needs to know which rule failed, on which element, after which interaction, and whether missing data or incomplete loading caused the result.
Items marked for human review, known baseline findings and unstable technical failures require triage first. “Informational” must not mean “ownerless”: the result needs a recorded decision rather than a silent skip.
What still requires human review
Human evaluation focuses on what a machine cannot decide reliably: logical reading order and meaning, the quality of text alternatives, keyboard operation, focus movement, error announcements to screen readers, and whether the complete task can be finished. Keyboard accessibility testing provides a practical first filter.
Frequency depends on risk and change. A new checkout needs human review before release; a stable information page may follow a different schedule. CI/CD reduces the risk of reintroducing certain barriers, but it does not replace an audit, user testing or individual legal advice. It is a continuous layer between deeper evaluations, not proof of full conformance. Combine it with a plan that recognises why a one-off audit is not enough.
What you gain
Developers receive specific findings while the relevant change is still fresh. The team can separate a new regression from technical noise, and critical journeys do not wait for the next large audit. Human specialists can spend their time on behaviour and meaning that automation cannot judge.
Next step
Choose one critical journey and list both its visible and dynamic states. For each state, define the automated check, expected result, and human-review items. Trial this bounded pilot in one part of the pipeline without presenting it as a complete WCAG audit.