Accessibility Testing Tools — axe, WAVE, Lighthouse

Automated accessibility tools cannot replace manual testing — they catch around 30–40% of WCAG failures, and the failures they miss are often the most impactful ones. But that 30–40% is real, repeatable, and cheap to find. Integrating the right tools into your workflow means you never ship the easy-to-catch failures and can focus your manual testing time on the harder ones that require human judgement.

The main automated tools

axe DevTools — the standard for automated accessibility testing

axe from Deque Systems is the most widely deployed accessibility testing engine in the world. It is the engine powering Lighthouse, many browser extensions, and automated testing integrations across every major testing framework.

The browser extension (available for Chrome and Firefox) is the fastest way to start: install it, open DevTools, and run a scan on the current page. axe returns a list of findings, each mapped to the WCAG criterion it violates, the HTML element responsible, and a clear description of how to fix it.

axe has a design philosophy of zero false positives: every finding it reports is a genuine issue. Issues it is not confident about are flagged as "Needs review" rather than errors. This makes it suitable for CI use — a failing axe scan is a reliable signal, not noise.

Automation integration is where axe becomes a continuous quality gate. The @axe-core/playwright, cypress-axe, and axe-selenium-java integrations allow you to inject an accessibility scan into any existing automated test. A typical pattern:

// Playwright + axe example (conceptual)
test('checkout page has no a11y violations', async ({ page }) => {
  await page.goto('/checkout');
  const results = await checkA11y(page);
  expect(results.violations).toHaveLength(0);
});

Adding this to your existing end-to-end test suite means every page your automation visits is also accessibility-tested — at no additional cost in test maintenance.

WAVE — the visual learner's tool

WAVE from WebAIM overlays icons and indicators directly on the page: red icons for errors, yellow for alerts, blue and green for structural and positive elements. Instead of a list of findings, you see the issues in their visual context — a missing label indicator appears next to the unlabelled input.

WAVE is particularly effective for:

• Learning accessibility: seeing issues in context builds faster intuition than a text list
• Communicating findings to developers: a screenshot with WAVE overlays is more immediately understandable than an axe report
• Structural overview: WAVE's Structure tab shows the heading hierarchy, landmark regions, and ARIA roles in a readable list, making it easy to spot missing headings or duplicate landmarks

WAVE is available as a browser extension and as a web service at wave.webaim.org (useful for publicly accessible URLs without needing an extension).

Lighthouse — the combined audit

Lighthouse is built into Chrome's DevTools panel (under the Lighthouse tab) and requires nothing to install. It runs a combined audit covering accessibility, performance, best practices, SEO, and Progressive Web App criteria, producing a single report.

For accessibility, Lighthouse uses axe under the hood and produces a score from 0–100. The score is useful for tracking trend over time but not meaningful as an absolute target — a score of 100 still means only that automated checks pass, not that the page is genuinely accessible.

Lighthouse is particularly useful in CI pipelines through Lighthouse CI: it can be configured to fail a build if the accessibility score drops below a threshold, or if new violations are introduced compared to the baseline.

Pa11y — for scripted and batch testing

Pa11y is a command-line accessibility tester designed for CI environments and batch testing scenarios. It runs headlessly against any URL and outputs results in multiple formats (JSON, CSV, HTML report).

Pa11y is useful when you need to:

• Test a large number of URLs in one run (crawl and test an entire site)
• Integrate accessibility testing into a CI pipeline without installing a browser extension
• Script accessibility tests as part of a deployment validation step

The 30–40% limit: what tools cannot catch

Automated tools reliably catch:

• Missing alt text on images
• Missing form labels
• Insufficient colour contrast
• Missing document language
• Invalid ARIA attributes
• Page title absent

Automated tools cannot catch:

• Whether alt text is meaningful (an alt of "image" passes the check)
• Logical tab order (the technical order is measurable; whether it is intuitive is not)
• Whether a screen reader understands a complex widget (ARIA is syntactically valid but semantically confusing)
• Keyboard traps (the element exists and is focusable, but you cannot Tab away from it)
• Cognitive accessibility (language clarity, navigation consistency, timeout warnings)

This is why the next lesson — manual testing — is not optional. Tools clear the floor; human testing determines whether the floor is high enough.

Recommended workflow

1. Daily/per-commit: axe integrated into existing end-to-end tests. Zero violations as a merge requirement.
2. Per-sprint/feature: WAVE or axe browser extension run manually on every new or changed page.
3. Per-release: Lighthouse CI report tracking score over time. Manual keyboard and screen reader test of all new features.
4. Periodically: External accessibility audit by a specialist team or users with disabilities.

⚠️ Common mistakes

• Reporting the Lighthouse score as proof of accessibility. A score of 100 means automated checks pass. It says nothing about keyboard navigation, screen reader usability, or cognitive accessibility. Never cite it as evidence of full compliance.
• Running axe only on the home page. Different pages have different components. A checkout form might have label issues that the home page does not. Test every distinct page and component type.
• Not integrating axe into CI. A tool you run manually before releases will catch issues before releases. A tool integrated into CI catches issues before they reach the main branch. The earlier you catch, the cheaper the fix.

🎯 Practice task

Set up a basic automated accessibility testing layer on any project you have access to.

1. Install axe DevTools and run it on your product's three most-trafficked pages. Count the violations per page.
2. If you have an existing Playwright or Cypress test suite, add one axe check to an existing test using the appropriate integration library.
3. Run Lighthouse on the same three pages and note the accessibility scores. Compare what Lighthouse reports versus what axe DevTools shows — are they the same findings?

Document the current violation count as a baseline. Track it over the next two sprints. A declining violation count is a lagging metric for improving accessibility.