The mobile bug we missed because we only tested on Wi-Fi

This is a case study, details blurred, of a bug that was structurally invisible to us: it could only happen on a slow or flaky connection, and we did all our testing in an office on fast, stable Wi-Fi. The app worked perfectly under the one condition we tested, and broke under the conditions most of our users actually had.

Context

A mobile app with an in-app payment step: tap "Pay", a request goes to the backend, a spinner shows, success screen on completion. Tested thoroughly — on the office Wi-Fi, where every request came back in well under a second.

Symptoms

Support tickets about double charges. Customers swore they tapped Pay once; their statement showed two identical charges minutes apart. It was intermittent and we couldn't reproduce it at our desks — which, again, was the clue we didn't read at first.

Investigation

The pattern in the data: double charges clustered around users on cellular networks, and around times of day that suggested commutes. Reproducing it meant leaving the office (literally) — or throttling the connection to simulate a train. On a slow link, the sequence was:

1. User taps Pay. Request goes out.
2. The connection is slow; the app's request times out client-side after a few seconds and shows "Payment failed, please try again."
3. But the request hadn't failed — it was just slow. It reached the backend and succeeded a moment later.
4. The user, seeing "failed", taps Pay again. Second charge.

The backend happily processed both, because nothing told it they were the same intended payment.

Root cause

Two compounding issues, both hidden by fast Wi-Fi:

• A client timeout shorter than real-world latency. On Wi-Fi, responses were instant, so the timeout never fired. On cellular, it fired constantly — turning slow successes into displayed "failures."
• No idempotency on the payment. A retried payment created a second charge instead of being recognised as the same one. The classic idempotency gap from the 12 API bugs — invisible until something triggers a retry, and a flaky network triggers retries all day.

What the tests missed

We tested the payment flow's logic exhaustively and its network conditions not at all. Every test ran on a fast, stable connection, so the timeout-then-retry path — the entire failure — never executed. There was no test on a throttled/slow link, no test of "what happens when a request times out but actually succeeds," and no idempotency check. It's the exact blind spot behind why mobile bugs escape web-first teams: the network is a variable, and we'd treated it as a constant.

The reusable lesson

Test mobile (and any network-dependent) flows on realistic, hostile networks, not just the office Wi-Fi — throttle to slow 3G, introduce latency and packet loss, drop and restore the connection mid-request. For anything that charges, books, or mutates, assume retries will happen and test idempotency: same intent twice should mean one effect. The conditions you don't test are the conditions your users will find for you.

Network-condition case lessons

• Test on throttled / slow / flaky networks, not just fast stable Wi-Fi
• Simulate request timeout-then-late-success — does the app double-act?
• Set client timeouts to real-world latency, not Wi-Fi latency
• For payments/bookings/mutations, test idempotency: same intent twice → one effect
• Drop and restore the connection mid-request; switch Wi-Fi ↔ cellular
• "Can't reproduce at my desk" on a mobile bug is a prompt to change the network, not close the ticket