TEST DESIGN
Pairwise Testing.
Black-Box A combinatorial test design technique that guarantees every pair of parameter values appears in at least one test case — covering the majority of real-world defects while dramatically reducing the total number of tests.
What it is
Pairwise Testing (also called All-Pairs testing) is based on the empirical observation that the large majority of software defects are caused by interactions between at most two variables — not three or more. A pure exhaustive test of n parameters with k values each requires k^n test cases. Pairwise testing uses a combinatorial algorithm to produce a much smaller test set in which every possible pair of values across all parameters appears at least once. This gives near-equivalent defect detection to exhaustive testing at a fraction of the test count — typically 50–90% fewer tests. The paired Pairwise Test Case Generator utility automates the algorithm so you never need to calculate the matrix by hand.
When to use it
When to use
- Features with multiple independent input parameters and too many combinations for exhaustive testing (browser × OS × screen resolution, payment method × currency × 3DS flag)
- Configuration matrices for CI pipelines (language version × framework × database)
- API endpoints with several orthogonal query parameters (format × sort × filter × page)
- Form fields with multiple independent options (e.g. shipping type, payment tier, coupon active — all independent)
- When defect history shows interaction defects between pairs of parameters rather than individual parameter failures
When NOT to use
- When parameters are NOT independent — pairwise assumes pair-independence; correlated parameters (e.g. city and country) need explicit rules, not all-pairs coverage
- When 3-way or higher-order interactions are known risk areas — pairwise only guarantees 2-way coverage; use n-wise (3-wise, 4-wise) if higher interaction coverage is required
- Very small combination spaces (2×2 or 3×2) where exhaustive testing is already practical
- When regulatory or safety standards require full exhaustive coverage — pairwise is a risk-based reduction, not 100% coverage
How it works
A pairwise algorithm (typically an orthogonal array or IPOG algorithm) constructs the smallest possible test set in which every combination of any two parameter values appears at least once. The worked example shows a login form with 3 parameters × 3 values each: exhaustive testing requires 27 tests; pairwise reduces this to 9 while still covering every pair.
Login form — Browser (Chrome, Firefox, Safari) × Auth method (Password, SSO, MagicLink) × Remember me (On, Off)
| Test # | Browser | Auth method | Remember me | Result |
|---|---|---|---|---|
| 1 | Chrome | Password | On | Accept |
| 2 | Chrome | SSO | Off | Accept |
| 3 | Chrome | MagicLink | Off | Accept |
| 4 | Firefox | Password | Off | Accept |
| 5 | Firefox | SSO | On | Accept |
| 6 | Firefox | MagicLink | On | Accept |
| 7 | Safari | Password | On | Accept |
| 8 | Safari | SSO | Off | Accept |
| 9 | Safari | MagicLink | On | Accept |
⚠ 9 tests instead of 27 (exhaustive). Every pair is covered: Chrome+Password, Firefox+SSO, Safari+MagicLink, Password+On, SSO+Off, MagicLink+On, etc.
Step by step
List parameters and their values
Identify every independent input parameter. List all distinct values each can take. Parameters with continuous ranges should be discretised first using Equivalence Partitioning (e.g. price field → low/mid/high).
Feed the parameter/value table into the generator
Use the Pairwise Test Case Generator: enter each parameter name and its comma-separated values. The generator produces a test matrix using the IPOG algorithm.
Review the output for parameter independence
Scan the generated matrix for any row where a parameter combination is logically impossible (e.g. country=UK + postal-code-format=US-ZIP). Mark those rows as N/A rather than executing them; the algorithm over-generates for dependent parameters.
Add expected results to each row
For each row in the pairwise matrix, determine the expected outcome. Most rows will have the same expected result (success) — the value of pairwise is finding the one combination that silently fails.
Execute and record actuals
Run each test case. Record any row where actual ≠ expected. A failure in a pairwise test pins the defect to an interaction of two specific values — that pair is the starting point for root-cause investigation.
Pitfalls & what it misses
Assuming pairwise gives full coverage
Pairwise guarantees 2-way (pair) coverage. If a defect only manifests when three specific values occur together (3-way interaction), pairwise may not include that exact combination. Use 3-wise testing for known high-risk 3-way interactions.
Applying pairwise to dependent parameters
If parameter B's valid values depend on the value of parameter A (e.g. state/province depends on country), pairwise will generate invalid combinations. Either pre-filter them out or model dependent parameters as a single combined parameter before running the algorithm.
Skipping EP first on continuous parameters
Feeding raw continuous ranges (age: 1–120) into a pairwise tool creates a combinatorial explosion or forces you to pick arbitrary values. Discretise continuous parameters with EP partitions (low/mid/high) before applying pairwise.
Treating a 'no failure found' pairwise run as full regression
A clean pairwise run means no 2-way interaction defect was found in that specific value set. It does not mean the feature is fully regressed — it still needs boundary values (BVA) and individual-parameter negative tests.
Paired utility
// Related resources