Agile Testing

How testing actually works inside an Agile team — what QA does in each ceremony, how to size effort per story, what "done" means, and how the practice extends from CI into production.

Agile Testing Principles

Agile Testing Quadrants (Brian Marick)

A model for thinking about what kind of testing you're doing and why. Two axes: business vs technology, and supporting the team vs critiquing the product.

                  Supporting the team
                          │
   ┌──────────────────────┼──────────────────────┐
   │                      │                      │
   │         Q2           │         Q3           │
   │    Functional        │   Exploratory        │
   │    User stories      │   Usability / UAT    │
   │    Prototypes        │   Alpha / beta       │
   │                      │                      │
   ├──────────────────────┼──────────────────────┤   Business-facing
   │                                             │
   │ Technology-facing                           │
   │                                             │
   ├──────────────────────┼──────────────────────┤
   │                      │                      │
   │         Q1           │         Q4           │
   │    Unit tests        │   Performance        │
   │    Component tests   │   Security / load    │
   │    Contract tests    │   Soak / chaos       │
   │                      │                      │
   └──────────────────────┼──────────────────────┘
                          │
                  Critiquing the product

A balanced team invests in all four. A common smell: heavy in Q1 + Q2, no Q3 (no exploratory) and no Q4 (no perf/security). Bugs sneak through the gap.

Test Pyramid

The cost-and-coverage shape of an Agile test suite. Most tests at the bottom (cheap, fast); fewest at the top (slow, expensive).

            ╱╲
           ╱  ╲       ~10%   E2E       slowest, most fragile
          ╱────╲
         ╱      ╲     ~20%   Integration
        ╱────────╲
       ╱          ╲   ~70%   Unit       fastest, cheapest
      ╱────────────╲

Anti-patterns

The pyramid isn't a law — for some products (libraries, pure-logic services) the right shape is even more bottom-heavy. For others (UI-heavy apps), 60/25/15 is more realistic. The point: be deliberate about the ratio, not accidental.

QA in Scrum Ceremonies

Three Amigos meeting

When a story is unclear, get a developer, a tester, and a product person together — the three amigos. The tester's role is to keep asking "what could go wrong?" and "what's the acceptance criteria for that case?" until the story is concrete enough to estimate.

Story Testing Workflow

The same-sprint flow that healthy Agile teams use. The order matters: testing tasks are spread across the sprint, not stacked at the end.

Story enters sprint
       │
       ▼
QA reviews AC ──── gaps? ──→ raise in standup / Three Amigos
       │
       ▼
QA writes scenarios (shift-left, before dev finishes)
       │
       ▼
Developer builds the feature
       │
       ▼
QA tests on dev branch or feature environment
       │
       ├──── bug found? ──→ communicate immediately (chat / pair > ticket)
       │                    └─ developer fixes ─ QA verifies
       ▼
Regression check (automated suite + targeted manual)
       │
       ▼
Story → Done (DoD met) → demo at review

What gets in the way

• Story arrives in code review with no test scenarios. QA wasn't pulled in early — fix at refinement, not at the PR.
• All testing happens on the last day of the sprint. Story was too big to ship + test in one sprint. Split it.
• "It works on my machine." No shared dev/feature env, or env is broken. Treat env health as a blocker, not a fact of life.
• Bugs filed but never fixed in-sprint. Carryover compounds. Cap WIP on bugs the same way you cap stories.

Definition of Done (DoD) — Testing Criteria

A story isn't done until everything below is true. Treat this as a checklist on the story card — paste it into the description if your tracker doesn't surface it natively.

□ All acceptance criteria verified (manual or automated)
□ Unit test coverage meets team threshold (e.g. ≥ 80 %)
□ Integration tests passing
□ Regression suite passing
□ No open critical or high severity defects
□ Performance benchmarks met (if perf-sensitive)
□ Accessibility checks passed (WCAG AA)
□ Cross-browser / cross-device tested per support matrix
□ Code reviewed and approved
□ Documentation updated (user-facing, API, runbook)
□ Telemetry / logging in place

Some teams also add: feature flag added (if behind one), translations updated, analytics event wired, security review checked off.

The exact list depends on the team — but every team should have an explicit DoD. "We'll know it when we see it" is how regressions ship.

Acceptance Criteria & BDD

INVEST — what makes a good user story

If you can't answer the testability question, the story isn't ready. Send it back to refinement.

Given / When / Then format

The standard structure for acceptance criteria in Agile + BDD teams. Each scenario reads as one observable outcome.

Worked example

Given I am a logged-in user
And my cart is empty
When I add an item to my cart
Then the cart count should increase by 1
And I should see the item in the cart summary

Read top to bottom: the scenario is concrete, observable, and binary. The Then clauses are what the test will assert.

Multiple scenarios per story

Most stories need 3–6 scenarios — at minimum, one happy path plus the obvious failure modes.

Scenario: Add an item to an empty cart
  Given I am a logged-in user
  And my cart is empty
  When I add "Mountain Bike" to my cart
  Then the cart count should be 1
  And the cart summary should list "Mountain Bike"

Scenario: Add an out-of-stock item
  Given I am a logged-in user
  When I attempt to add an out-of-stock item to my cart
  Then I should see "Out of stock" message
  And the cart should remain empty

Scenario: Add an item while logged out
  Given I am not logged in
  When I attempt to add an item to my cart
  Then I should be redirected to the login page
  And the item should be added to the cart after I log in

Converting acceptance criteria to test cases

Each scenario in Given/When/Then form maps directly to a test case. The test runner determines the level:

The same Given/When/Then text can drive a manual test, a Cucumber/SpecFlow scenario, or be paraphrased into a Playwright test() block — pick the level that matches the AC's scope, not always the highest.

Automation of acceptance tests

When AC are written in Gherkin, automation is mostly glue:

The benefit isn't speed of writing — it's that the AC become the test artefact. Product, dev, and QA all see the same Given/When/Then; nobody hand-translates between a Word doc and a code file.

The cost: discipline. If step definitions become a tangled mess of generic When I click {string} steps, you've lost the readability advantage. Keep step phrasing domain-specific, not technology-specific.

Continuous Testing

The pipeline-driven extension of Agile testing — every commit verified through a layered test suite that gets slower as confidence grows.

Every commit triggers automated tests

The pipeline runs the same tests for every PR and every merge. Local "but it works on my machine" loses to CI as the source of truth.

The standard pipeline shape

commit
   │
   ▼
┌──────────┐  fail-fast — runs in seconds
│   lint   │
└────┬─────┘
     ▼
┌──────────┐  fast — isolated, no I/O
│   unit   │
└────┬─────┘
     ▼
┌──────────────┐  medium — DB, message bus, HTTP mocks
│ integration  │
└────┬─────────┘
     ▼
┌──────────┐    slow — full browser, real services
│   E2E    │
└────┬─────┘
     ▼
┌──────────────┐  optional / scheduled — load, soak
│ performance  │
└──────────────┘

Each stage gates the next. A failure in unit aborts before E2E even starts. Cheaper failures find faster feedback.

Fast-feedback budget

If the unit stage takes 20 minutes, devs stop running it locally. If E2E takes 90 minutes, devs stop reading the failures. Slow tests get bypassed — speed is correctness.

Shift-right — testing extends into production

Continuous testing doesn't stop at deploy. The complement of shift-left is shift-right: learn from production.

Feature flags — ship dark, then test

Decouples deploy from release. Code reaches production behind a flag; the flag stays off until tested. Switch on for QA, then internal users, then real users.

deploy (flag off, no behaviour change)
        │
        ▼
flag-on for QA-only cohort   ──────────┐
        │                              │
        ▼                              │
flag-on for 1% of real users           ├──── monitor production
        │                              │
        ▼                              │
flag-on for 100%                       │
        │                              │
        ▼                              │
remove flag from code  ◄───────────────┘

If anything goes wrong at any step: flip the flag off — no rollback, no redeploy.

A/B testing — validate with real users

Run the new version (B) against the old (A) for two cohorts of real users. Compare outcomes:

The QA role isn't to pick the winner — it's to make sure the experiment is measurable (instrumentation present, metrics defined, sample size adequate) and that both arms are equally tested before launch.

Agile Testing Metrics

Metrics in Agile aren't management report fodder — they're feedback for the team. Pick a small set and watch the trend, not the absolute number.

Defect-related metrics

Coverage metrics

Code coverage as a target gameable; as a trend, it's a sensible early warning.

Velocity & process metrics

Automation metrics

Pick the smallest set that drives action

Reporting 12 metrics nobody acts on is a bigger problem than reporting 3 you do. A practical starter dashboard:

1. Escaped defects this release — the only one product cares about.
2. CI build time — fast-feedback budget; team productivity.
3. Flakiness rate — trust in the suite; if it climbs, fix it that sprint.
4. Stories rolled over due to testing — capacity signal.

Add more only when you have a question those four don't answer.