Q2 of 38 · CI/CD & DevOps
How do you parallelise your test suite to keep CI runs under 10 minutes?
Short answer
Short answer: Shard tests across multiple runners by historical duration, run unit tests with the test runner's built-in workers, and parallelise E2E by spec file or by tag. Cache dependencies and Docker layers aggressively.
Detail
Getting CI under 10 minutes for a non-trivial suite is mostly about parallelism and caching. The strategy stacks at three levels.
Inside a job: use the test runner's worker pool. Jest, Vitest, pytest-xdist, NUnit, and most modern runners support N parallel workers per machine. The right N is roughly the number of CPU cores; over-subscribing causes contention.
Across jobs: shard the suite across multiple CI runners. Most CI providers (GitHub Actions matrix, CircleCI parallelism, Buildkite, Jenkins) let you split tests into N shards and run each shard on its own runner. The naive approach is round-robin file assignment; the smarter approach is duration-based sharding — record how long each spec took, then bin-pack so each shard finishes around the same time.
Across pipelines: split by speed, not by type. Run fast unit tests on every push and gate the merge; run slow E2E tests in a separate pipeline that runs on PR open and pre-deploy. This keeps developer feedback fast without skipping coverage.
Caching is the other half of the budget: dependency caches (npm/pnpm/poetry/Maven), build caches (Turborepo, Nx, Bazel, Gradle), Docker layer caches (BuildKit + a registry-backed cache), and browser binary caches for Playwright/Cypress. A clean install on every run easily costs you 3–5 minutes you don't need to spend.
For E2E specifically: tag tests by criticality (@smoke, @regression, @nightly), run smoke on every PR, and run the long tail nightly or on-demand. Most teams don't need full E2E on every commit.
// EXAMPLE
.github/workflows/test.yml
name: tests
on: [pull_request]
jobs:
unit:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- run: npm ci
- run: npx jest --shard=${{ matrix.shard }}/4 --maxWorkers=2
e2e-smoke:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: '20', cache: 'npm' }
- run: npm ci
- run: npx playwright install --with-deps chromium
- run: npx playwright test --grep @smoke --shard=${{ matrix.shard }}/3