DebuggingIntermediate4-6 min reference
Flaky Test Debugging
A flaky test passes and fails without the code changing. The fix is almost never "add a retry" — it's identifying the category of flake and removing the nondeterminism. This sheet maps symptoms to likely causes and lists the evidence to capture so you debug once instead of re-running blindly.
Flake patterns → likely cause
| Symptom | Usual cause | Fix direction |
|---|---|---|
| Passes locally, fails in CI | Timing / slower machine | Wait on state, not on sleep |
| Fails only when run with others | Shared/ leaked state, order dependency | Isolate data; reset state per test |
| Fails intermittently on one step | Race with async UI / network | Auto-wait for the element/response |
| Fails after a deploy, then "fixes itself" | Environment / data drift | Seed deterministic test data |
| Element not found sometimes | Animation, lazy render, fragile selector | Stable data-testid, wait for visible |
| Fails at a time boundary | Time zone / DST / midnight | Freeze clock; use UTC fixtures |
Evidence to capture (before re-running)
- Trace / video / screenshot at the moment of failure.
- Console + network logs and any server logs with timestamps.
- Seed / run order and the exact CI runner.
- Failure rate — 1-in-50 vs 1-in-2 changes the hunt.
- Whether it reproduces with the suite run in isolation vs full.
When to use
The moment a test "fails for no reason." Resist quarantining or adding blanket retries until you've categorised the flake — a retry hides a real race that will bite users too.
Common mistakes
sleep(2000)instead of waiting for the actual condition.- Tests that depend on each other's data or execution order.
- Blanket retries masking a genuine product race condition.
- Reusing one shared account across parallel tests.
- Asserting on wall-clock time or default time zone.
For code-level fixes (custom waits, fixtures, isolation), see the Snippets library linked below — keep this sheet about diagnosis.
// Related resources