How we reduced release testing from two days to four hours

This is a case study, details blurred, of a release-testing process that had grown to two miserable days and how we cut it to an afternoon. The interesting part isn't speed for its own sake — it's that the faster process caught more real bugs, because most of the original two days was spent on testing that protected nothing.

Context

A two-week release cycle ending in a manual regression pass: a couple of hundred test cases, two testers, two days, every release. It was the bottleneck everyone dreaded, it pushed releases late, and — the detail that started the whole project — it had recently let a real bug through anyway. Slow and not catching things is the worst of both worlds.

Symptoms

The pass had the classic signs of effort without proportion: every case weighted the same regardless of risk, large blocks of time re-verifying features that hadn't changed in a year, and testers mechanically marching through a checklist rather than thinking about the release. Two days in, everyone was exhausted, sign-off was a formality, and confidence was low despite the hours spent.

Investigation

We did something simple: we looked at where the two days actually went, and cross-referenced it against where bugs had actually been found over the previous year. The mismatch was stark.

• A huge share of time went to stable, low-traffic features that essentially never changed and essentially never broke.
• Almost no extra time went to the areas that changed most and broke most — they got the same single happy-path case as everything else.
• A big chunk was manual re-checking of deterministic, stable flows — exactly the work automation is good at and humans are wasted on.

The two days weren't testing the release; they were re-testing the whole product uniformly, most of it for no reason.

What we changed

Four changes, in priority order — and notably, we did not try to automate everything or simply test less:

1. Risk-and-change weighting. Each cycle now starts from what changed and what's historically fragile. Risk-based prioritisation gets the modified, high-traffic, high-impact areas deep testing; stable backwaters get a light touch.
2. Automated the boring stable core. The deterministic, unchanging happy-path flows — login, core navigation, the critical money path — became an automated smoke suite that runs in CI in minutes. Humans stopped re-doing it by hand.
3. Pruned dead cases. Cases for features no one used came out. The suite shrank to what protected something real.
4. Time-boxed exploratory testing on the change. Instead of more scripted cases, we put focused exploratory sessions on the actual new work — where the real bugs live.

Root cause of the original slowness

The two days weren't slow because testing is inherently slow. They were slow because the process spent equal effort on unequal risks and made humans repeat work a machine should own. The cost wasn't thoroughness; it was a lack of prioritisation dressed up as thoroughness. Uniform coverage feels rigorous and is actually the opposite — it under-tests the risky 10% to lavish attention on the safe 90%.

The result

Four hours: a fast automated smoke pass plus focused human testing aimed at the change and the fragile areas. And it caught more, because the testers' attention was now spent where bugs actually were instead of spread thin across the whole product. The lesson generalises well beyond this team: the goal of release testing is confidence proportional to risk, not coverage proportional to the size of an old checklist. Shorter was a side effect of testing the right things; the right things were always a fraction of what we'd been doing.

Faster-release case lessons

• Audit where the time goes vs. where bugs are actually found — expect a big mismatch
• Weight testing by risk and change, not uniformly across every case
• Automate the stable, deterministic core (login, navigation, money path) so humans stop repeating it
• Prune cases for features nobody uses; shrink the suite to what protects something
• Spend the freed time on exploratory testing of the actual change, where real bugs live
• Aim for confidence proportional to risk, not coverage proportional to an old checklist