Provider Verification

Verify that the User Profile service satisfies two consumer contracts — plan provider states, run verification against a failing contract, diagnose the root causes, and recommend fixes and a CI integration strategy.

Role

SDET

Difficulty

Advanced

Time limit

2 hr

Scenario

The User Profile service team has adopted Pact. Two consumer teams have published contracts to the Pact Broker: the Orders service and the Notifications service. The provider team must run verification to confirm the current User Profile service implementation satisfies both contracts before the next deployment. You have been given the two Pact files (summarised below) and a verification run report showing two failures. Your task is to: plan the provider-state setup hooks needed for verification, diagnose both failures, propose fixes, and define a CI verification strategy that prevents deploying a breaking change.

Requirements

1.For each consumer contract, list every provider state that must be set up and describe the exact data precondition each state hook must create or clear
2.Explain how provider verification works mechanically: what the Pact framework does when you run it against the provider, and what role the provider states play
3.Analyse the two provided failure reports: for each failure, identify the root cause (contract mismatch, missing provider state, provider-side bug, or breaking change), state which party must act, and propose a concrete fix
4.Produce a provider verification checklist: a step-by-step list covering environment setup, provider state hooks, running verification, interpreting results, and publishing the 'can-I-deploy' result
5.Build a contract verification matrix: rows are interactions (consumer × state × endpoint), columns are pass/fail for the current run and proposed status after your fixes
6.Define a CI integration strategy: where in the pipeline verification runs, what it blocks (merge? deploy?), and how failing provider verification is communicated back to the consumer team
7.Describe at least two breaking-change scenarios that would cause provider verification to fail, and for each one explain how you would communicate the change to the affected consumer team

Starter data

›Orders service Pact file (summary of 3 interactions): Interaction 1: state='user with ID 42 exists with role customer'; GET /users/42 → 200, { id: 42, username: string, role: 'customer' } Interaction 2: state='user with ID 99 exists with role admin'; GET /users/99 → 200, { id: 99, username: string, role: 'admin' } Interaction 3: state='no user with ID 9999 exists'; GET /users/9999 → 404, {}
›Notifications service Pact file (summary of 2 interactions): Interaction 1: state='user with ID 42 exists'; GET /users/42 → 200, { id: 42, email: string, phoneNumber: string } Interaction 2: state='user with ID 7 exists and has no phone number'; GET /users/7 → 200, { id: 7, email: string, phoneNumber: null }
›Verification run report — FAILURES: FAIL [Orders] Interaction 2: state='user with ID 99 exists with role admin'; GET /users/99 → expected body { role: 'admin' } but received { userRole: 'admin' } — field name mismatch FAIL [Notifications] Interaction 1: state='user with ID 42 exists'; GET /users/42 → expected body includes 'phoneNumber' but field is absent from response — field not yet implemented
›User Profile current response schema: { id, username, email, userRole, created_at, preferences } — note the field was renamed from 'role' to 'userRole' in a recent refactor; phoneNumber field is planned but not yet added
›Provider state hooks are implemented as test setup/teardown functions in the provider's test suite — they run before each interaction is replayed, and must be idempotent (safe to run multiple times)
›Pact Broker URL: https://pactbroker.shopflow-internal.example.com (assume accessible from CI)

Expected deliverables

✓Provider-state setup plan: a table listing each state name, which consumer owns it, what data the hook must create or confirm exists, and what it must clear or reset after the interaction
✓Mechanical explanation of provider verification: a 200–300 word description of what happens when the Pact verification step runs (request replay, response comparison, provider-state lifecycle)
✓Failure analysis for both failing interactions: root cause, responsible party, and concrete fix proposal with enough detail to implement
✓Provider verification checklist: a numbered step-by-step checklist from 'set up test environment' to 'publish verification result to Pact Broker'
✓Contract verification matrix: all five interactions as rows, with columns for consumer name, provider state, endpoint, current status (PASS/FAIL), proposed status after fixes, and root cause note
✓CI integration strategy: where verification sits in the pipeline, what it blocks, and how failures are surfaced to both provider and consumer teams
✓Breaking-change communication plan: two example breaking scenarios with a step-by-step description of how to notify affected consumers and coordinate the migration

Evaluation rubric

Dimension	What reviewers look for
Provider state precision	Are provider state hooks described precisely enough to implement? 'Create a user' is not sufficient. 'Insert a row into the users table with id=42, username=alice, email=alice@example.com, userRole=customer, status=active — or confirm the row already exists and has those values' is implementable. Each hook description must include the table/entity, the field values required, and a note on idempotency (what to do if the row already exists from a previous test run).
Failure root-cause accuracy	For Failure 1 (role vs userRole rename): is the root cause identified as a breaking change introduced on the provider side, not a consumer-side bug? The provider team owns the fix — but the consumer team must also be consulted about a migration path. For Failure 2 (missing phoneNumber): is it correctly identified as a not-yet-implemented feature on the provider, not a consumer error? The fix is to implement the field, not to reject the consumer contract.
Verification mechanics understanding	Does the explanation correctly describe the sequence: provider states are set up by hooks → Pact framework sends the contracted request to the real provider → actual response is compared to the contracted response → result is published to the broker? A common error is describing verification as 'the provider sends requests to the consumer' or 'both sides communicate during verification' — verification replays the contracted request against the provider in isolation; no live consumer is involved.
CI integration strategy completeness	Does the strategy specify where in the pipeline verification runs (PR check? pre-deploy gate?), what it blocks (merge into main? deployment to staging?), and how the 'can-I-deploy' query works? A strategy that only says 'run pact verify in CI' without specifying the blocking point and the feedback loop to consumers is incomplete.
Breaking-change communication	For the role → userRole rename scenario: does the plan include notifying the Orders team before deploying the rename, agreeing a migration window, updating the consumer contract, and only deploying the provider change after the consumer has deployed its update? Unilateral deployment of a breaking change is the most common real-world CDC failure mode — the plan must describe coordination, not just 'fix and deploy'.

Sample solution outline

›Provider-state setup plan (5 states): (1) 'user with ID 42 exists with role customer' → upsert user id=42, username=alice, email=alice@example.com, userRole=customer, status=active; (2) 'user with ID 99 exists with role admin' → upsert user id=99, username=bob, userRole=admin; (3) 'no user with ID 9999 exists' → delete user id=9999 if exists; (4) 'user with ID 42 exists' [Notifications] → same as state 1, also set phoneNumber=+15555550100; (5) 'user with ID 7 exists and has no phone number' → upsert user id=7, username=carol, phoneNumber=null
›Verification mechanics: The Pact framework starts the provider's test server, runs the state-setup hook for the first interaction, then sends the exact HTTP request from the Pact file to the provider. It compares the actual response status and body against the contracted response using the matchers defined in the Pact file. After each interaction the state-teardown hook runs. Results are recorded and published to the Pact Broker as a verification result tagged to the provider version. The 'can-I-deploy' query then checks whether both provider and consumer have verified results for the current versions before allowing deployment.
›Failure 1 analysis: Root cause — provider-side breaking change. The User Profile service renamed the response field from 'role' to 'userRole' in a backend refactor without notifying consumers or running a contract verification check before deploying. The Orders service contract expects 'role'; the provider now returns 'userRole'. Owner: provider team. Fix: either add a 'role' alias to the response (backward-compatible) while migrating consumers, or coordinate a joint deploy where both the Orders consumer contract and the provider are updated simultaneously. Do not simply rename the consumer contract without a consumer deploy.
›Failure 2 analysis: Root cause — feature not yet implemented on the provider. The Notifications service's contract requires phoneNumber in the GET /users/{id} response, but the User Profile service has not yet implemented the field. Owner: provider team (implementation). This is not a consumer error — the contract correctly describes a feature the Notifications team needs. Fix: implement the phoneNumber field on the provider, update the schema, and re-run verification. The contract should not be rejected or changed on the consumer side.
›Verification checklist: (1) Confirm provider test environment is isolated from production data; (2) Confirm Pact Broker is reachable and consumer pact versions are published; (3) Implement or update all 5 provider-state hooks in the provider test suite; (4) Run pact verify pointing to the Pact Broker; (5) Review per-interaction results — identify PASS/FAIL with interaction name; (6) For each FAIL: log root cause, assign owner, track in sprint; (7) Re-run after fixes; (8) Publish verification result to Pact Broker with provider version tag; (9) Run can-I-deploy query for provider version before promoting to staging
›CI strategy: Verification runs in the provider's CI pipeline as a pre-merge check on every PR that touches the User Profile service. A failing verification blocks the PR from merging into main. The verification result is published to the Pact Broker automatically, enabling the can-I-deploy webhook. The consumer teams receive a Slack notification (or Pact Broker webhook) when a provider verification fails against their contract, so they can coordinate a fix before the provider deploys.
›Breaking-change scenarios: (1) Renaming role to userRole — provider must not deploy this rename until all consumers have updated their contracts and deployed. Coordination: notify Orders and Notifications teams in a shared channel, agree a migration window, deploy consumer changes first, then deploy provider change, then verify; (2) Removing the GET /users/{userId} endpoint entirely — any consumer contract that includes this interaction will fail verification immediately. Coordination: publish a deprecation notice 2 sprints ahead, provide a migration path (e.g. new endpoint or versioned path), verify all consumers have migrated before removing the old endpoint.

Common mistakes

Writing provider states that are too vague for a developer to implement ('ensure test data exists') — states must specify the entity, its field values, and idempotency behaviour
Misidentifying Failure 1 as a consumer bug — the consumer contract is correct; the provider introduced the breaking change by renaming a field without coordination
Misidentifying Failure 2 as a consumer error — the Notifications contract is a valid description of a feature the team needs; the fix is to implement the feature, not reject the contract
Describing verification as a live integration test where both services communicate — provider verification is a provider-only test that replays the contracted interactions in isolation against the provider service; no running consumer is involved
Proposing that the provider deploys a breaking change and consumers update their code afterwards — in CDC, the provider must never deploy a change that breaks a live consumer contract; the coordination must happen before deployment
Omitting the 'can-I-deploy' query from the CI strategy — verification confirms the provider satisfies the current contracts; can-I-deploy confirms that a specific provider version can safely be deployed alongside all currently deployed consumer versions
Creating provider state hooks that rely on execution order rather than being idempotent — a state hook that fails if the row already exists will cause intermittent failures when tests are re-run without a full teardown

Submission checklist

Provider-state setup plan with a row for each of the five states, including the entity, field values, and idempotency note
Mechanical explanation of provider verification (200–300 words) that correctly describes the request-replay-and-compare sequence
Failure analysis for both failing interactions: root cause, responsible party, concrete fix proposal
Provider verification checklist with numbered steps from environment setup to publishing the result
Contract verification matrix: all five interactions with consumer, state, endpoint, current status, proposed status, and root cause
CI integration strategy specifying where verification runs, what it blocks, and how failures are surfaced
Breaking-change communication plan for at least two scenarios with coordination steps described

Extension ideas

+Implement the five provider-state hooks in a language of your choice and run a local pact verify step against a mock User Profile server to confirm the pass/fail outcomes match your analysis
+Set up a local Pact Broker using the pact-broker Docker image, publish both consumer Pact files, run provider verification against the broker, and query the can-I-deploy endpoint for the provider version
+Write a provider verification test for a third consumer contract you design yourself — covering a future Analytics service that needs GET /users/{id} to return only id, created_at, and role — and verify it passes against the current provider