Q14 of 17 · Framework design

How do you keep a test automation framework maintainable over time as the application evolves?

Framework designSeniorframework-designmaintenancetechnical-debtcode-qualityversioning

Short answer

Short answer: Treat the framework like production code: code reviews for test PRs, lint rules, dependency updates on a schedule, test-to-code ratio tracking, and a clear policy for updating Page Objects when the UI changes.

Detail

Practices that prevent framework decay:

1. Test code reviews — same standard as production: Test PRs reviewed for: locator strategy (prefer data-testid over CSS), assertions specificity, no hardcoded waits (sleep), no business logic in step definitions, meaningful test names.

2. Dependency updates on a schedule:

# Check outdated packages weekly (automated PR via Dependabot or Renovate)
npx npm-check-updates -u
npm test  # verify nothing breaks after update

Playwright/Selenium major version upgrades often require Page Object changes. Defer them → your locators break on a day you can least afford it.

3. Flaky test policy: Track flakiness rate per test. Tests that flake more than 2% are tagged @quarantine and moved to a separate CI job. The quarantine job runs but doesn't gate the build. Flaky tests are fixed within one sprint or deleted.

4. Page Object coverage tracking: When a dev ships a new UI page or component without a corresponding Page Object update, tests start using raw selectors. Track this via PR checklists: "Does this feature change require a PO update?"

5. Naming conventions documented and enforced:

  • Page Objects: <PageName>Page.ts / <PageName>Page.java
  • Test files: <feature>.spec.ts
  • IDs: login-btn, not loginBtn (kebab-case for data-testid)

6. Regular framework audits (quarterly):

  • Delete tests that cover deleted features
  • Remove duplicate page objects
  • Update broken locators from the last UI overhaul
  • Review the flaky test quarantine queue

7. Changelog: A CHANGELOG.md for the shared framework library so teams know what changed between versions and whether they need to update their test code.

// WHAT INTERVIEWERS LOOK FOR

Flaky test policy with quarantine. Dependency update cadence. Test code review standards. Quarterly audit. Framing test code as a first-class engineering asset.