Q16 of 38 · Manual & exploratory
How do you decide what to automate versus keep manual?
Short answer
Short answer: Automate tests that are repetitive, deterministic, fast-feedback critical, and cover stable behaviour. Keep manual the cases that need human judgement, are exploratory, change frequently, or rarely run. The deciding factor is ROI — does the test save more time than it costs to maintain?
Detail
Automation is a cost-benefit decision per test, not a blanket policy. The questions to ask:
Will this test run often? A test that runs every commit is a strong automation candidate; one that runs once per release rarely is. Frequency × per-run cost = total saved time.
Is the behaviour stable? A login flow that changes once a year is high-ROI to automate. A new feature still being designed weekly will rot the test faster than it pays back. Wait until the design stabilises.
Is the assertion deterministic? "Page loads in < 2s" can be automated. "The new design feels intuitive" cannot — that's a human judgement test.
What's the cost of a defect escaping? Critical-path tests (login, payment, data integrity) deserve automation even at high maintenance cost; cosmetic tests probably don't.
What's the maintenance cost? Brittle UI tests with selectors that change on every refactor cost more than they save. API and integration tests are usually cheaper to maintain than UI E2E.
The rough hierarchy: automate smoke, regression of stable areas, data-driven tests (same logic, many inputs), API contract tests, performance/load. Don't automate (yet) exploratory, usability, accessibility judgement calls, anything where the spec is still moving. Manual one-shot for setup verification, audit-trail evidence, and stakeholder demos.
The signal interviewers want is that you don't automate for the sake of automating — you'll explicitly choose not to automate something.