Q26 of 37 · API testing
Walk me through your strategy for catching breaking changes in a public API.
Short answer
Short answer: Layered: schema diff in CI on every PR; contract tests against a representative consumer suite; deprecation header + sunset checks; an OpenAPI-driven test that fails on any incompatible change. Frame breaking change as a process problem — checks block merges, communications fire automatically, customers get advance notice.
Detail
Public APIs have customers who depend on every quirk. Breaking changes shipped quietly become incidents; the right strategy makes them impossible to ship by accident.
The five-layer defence:
1. Schema diff on every PR.
- The OpenAPI / GraphQL schema is checked in.
- A tool (
oasdiff,schemathesis,graphql-inspector) compares the PR's schema to main. - Any incompatible change (removed field, renamed parameter, narrowed enum) blocks merge unless tagged
breakingand approved.
- run: oasdiff breaking spec-main.yaml spec-pr.yaml --fail-on-diff
2. Schema-driven test generation.
- Tools like Schemathesis generate property-based tests from the OpenAPI schema.
- They send malformed inputs, edge cases, and confirm responses match the documented schema.
- Catches "API and docs disagree" — the most common breaking-change vector.
3. Contract tests with representative consumers.
- Pact contracts from each major consumer (web, iOS, Android, partner SDK).
- Provider verification in CI fails if the change breaks any consumer's recorded expectations.
- For public APIs without internal consumers, maintain "synthetic consumer" tests that exercise every documented response shape.
4. Versioned snapshot tests.
- For each major version (v1, v2), a frozen test suite that asserts on the exact response shape.
- New versions of the API may change responses; old versions cannot.
- Run continuously against main; failures mean someone changed v1's behaviour.
5. Deprecation + sunset enforcement.
- Every deprecated endpoint carries
DeprecationandSunsetheaders. - Tests assert these are set correctly.
- A separate test fails 30 days before sunset to force review.
Process:
Communicate breaking changes through:
- Changelog entry on every release, with migration guidance.
- Emails to API customers using the affected endpoints (telemetry-driven).
- Deprecation period of 6-12 months for SaaS, longer for partner integrations.
- Blog post for substantial changes.
SemVer or date-based versioning. /v1, /v2 for major; date-based (?api-version=2026-05-10) for finer granularity. Stripe's date-versioning is the gold-standard reference.
Internal escalation. A breaking change PR with the right tag goes through:
- Eng manager review.
- Customer success awareness (any major customer affected?).
- Optional product / legal sign-off for material changes.
Anti-patterns:
- Catching breaks in production telemetry only. By then, customers are paged.
- A single E2E test labelled "the API contract" that someone updates without thought when it breaks. Schema diff is non-negotiable.
- Deprecating endpoints without sunset dates. They linger forever and the surface area grows.
The senior signal: framing this as a process + automation combo. Tooling alone misses the human side; process alone is forgettable. Both are needed.
// WHAT INTERVIEWERS LOOK FOR
// COMMON PITFALL
// Related questions