Cypress Cloud — Dashboard, Analytics, and Flake Detection

The previous lesson treated Cypress Cloud (formerly the Cypress Dashboard) as the spec-distribution coordinator for parallel runs. That's its most-cited feature, but probably not its most valuable one. The real argument for Cloud is observability: a hosted dashboard that records every test run, replays failures with full DOM snapshots, flags flaky tests automatically, and runs the previously-failed specs first on the next push. This lesson covers the setup, the analytics, and the flake-management workflow that makes large suites maintainable.

Setup

The recording side is one CLI flag and one env var:

npx cypress run --record --key abc-123-record-key

The key comes from your Cypress Cloud project settings — create a project on cloud.cypress.io, generate a record key, store it in your CI's secret store. In CI it's almost always passed via CYPRESS_RECORD_KEY:

- uses: cypress-io/github-action@v6
  with:
    record: true
  env:
    CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}

The first run uploads results, screenshots, videos, and the entire command log. From then on the dashboard tracks every run keyed on the project ID + git SHA + branch.

You don't need parallel: true to record. Recording works on serial runs too — the parallelism is just one of the things record: true enables.

What the dashboard shows

For each run, Cypress Cloud presents:

• Run overview — pass/fail count, duration, parallelisation timing, browser, OS, the commit message.
• Per-spec breakdown — every spec file with its status, duration, retries, and link to the failing test.
• Test replay — for each test (passing or failing), a full video plus the same Time Travel debugging you'd see in the Test Runner. Hover over commands; see the DOM at that moment; expand network requests with their headers and bodies.
• Stack traces and screenshots — click any failure for the assertion error, the captured screenshot, the rendering of the surrounding page.
• Branch comparison — historical view of how the same suite has performed across branches and over time.

A red CI build that used to mean "download the zip, unzip, scrub through video, find the failure" now means "click the link, see the failure with full context in five seconds."

Flake detection — the killer feature

A flaky test is one that sometimes passes and sometimes fails on identical input. They're the bane of every test suite — engineers stop trusting red builds because "it'll pass on retry," and real failures slip through.

Cypress Cloud automates the management of them. Combined with retries:

// cypress.config.ts
export default defineConfig({
  e2e: {
    retries: { runMode: 2, openMode: 0 },
  },
});

Every failed test gets retried up to twice in CI. If a test fails then passes on retry, Cloud marks it as flaky instead of failed. The build is still green (the test eventually passed), but the dashboard surfaces the flake separately:

• Flakiness rate per test, ranked by frequency.
• Most-flaky tests with their last 30 runs.
• Trends over time — is flake getting worse or better?

This is the feature that turns flake from "we'll fix it eventually" into a visible, measurable backlog item. Engineers see the same five flaky tests every week and have to either fix them, quarantine them, or admit the suite isn't trustworthy.

Smart orchestration — failing tests run first

The Cloud knows which tests failed in the most recent run. On the next push, it reorders the specs so the previously-failing ones run first. You see whether the regression is fixed or not in the first minute of CI instead of waiting twelve minutes for the slow CI test grid to crawl through the green suite.

This isn't a config flag — it's automatic when you record. Combined with parallelisation, it means the most useful information lands in the dashboard fastest.

Cypress Cloud features at a glance

Pricing — the trade-off to know

Cypress Cloud has a generous free tier (500 test results per month) and paid tiers for teams. A "test result" is one run of one test on one machine — a parallel run with 4 containers and 60 tests on Chrome + Firefox produces 60 × 2 = 120 test results, not 480.

For projects below ~25,000 test results per month, Cloud is usually a no-brainer for the time it saves. For very large projects, the bill grows; either move to a paid tier or self-host with sorry-cypress (the OSS-compatible service mentioned in the previous lesson).

Alternatives if you don't want Cloud

If your project is small, your budget is zero, or you need to self-host:

• Mochawesome HTML reports (chapter 7) — covers the "stakeholder-readable artifact" use case.
• JUnit XML feeding the CI's native test dashboard (Jenkins, GitLab) — covers per-test pass/fail tracking.
• cypress-split + matrix sharding — covers parallelisation without the orchestration smarts.
• sorry-cypress — Cloud-compatible features on infrastructure you run.

You can replicate most of Cloud's individual features piecemeal. What's hard to replicate is the integration: replay + flake detection + smart orchestration + analytics in one click. Most teams that try the OSS path eventually trade up to Cloud for that integration.

Wiring it into a real project

A complete cypress.yml recording into Cypress Cloud:

name: Cypress
 
on:
  pull_request:
    branches: [main]
  schedule:
    - cron: "0 6 * * *"
 
jobs:
  cypress-run:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        containers: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
 
      - uses: cypress-io/github-action@v6
        with:
          record: true
          parallel: true
          group: "Chrome - PR"
          browser: chrome
          build: npm run build
          start: npm start
          wait-on: "http://localhost:3000"
        env:
          CYPRESS_RECORD_KEY:     ${{ secrets.CYPRESS_RECORD_KEY }}
          CYPRESS_BASE_URL:       ${{ secrets.STAGING_URL }}
          CYPRESS_ADMIN_PASSWORD: ${{ secrets.STAGING_ADMIN_PASSWORD }}
          GITHUB_TOKEN:           ${{ secrets.GITHUB_TOKEN }}

Combined with retries: { runMode: 2 } in cypress.config.ts, this gives you parallel execution, retry-on-failure, automatic flake detection, smart orchestration, and a recorded artifact that stakeholders can review in the dashboard. The whole stack ships with one PR.

A flake-management workflow

A pragmatic process that teams converge on once Cloud is in place:

1. Weekly review. A QA lead opens the Cloud's flaky-tests dashboard once a week and ranks the top five by frequency.
2. Triage. Each flaky test gets one of three labels:
   • Fix this sprint — likely cause is a known anti-pattern (cy.wait(ms), unstable selector, race condition).
   • Quarantine — move to a cypress/e2e/quarantine/ folder excluded from the main run; tracked in the bug tracker for later.
   • Delete — feature is going away or coverage isn't worth keeping.
3. Block on regressions. If a previously-stable test becomes flaky after a code change, the PR that introduced the regression is held until the test is fixed or a quarantine ticket is filed.

Cloud is the signal; the workflow is what turns signal into action. Without the dashboard, "flake" is fuzzy and unmeasurable. With it, flake becomes a tracked backlog you actually close down.

⚠️ Common mistakes

• Recording every push and burning through the free-tier quota by Tuesday. Restrict recording to PRs against main and nightly runs; skip recording on draft-PR pushes or feature branches that haven't been opened for review yet.
• Setting retries: { runMode: 5, openMode: 0 } to "make CI green." Five retries means a test that fails 80% of the time still passes most builds. The dashboard now thinks the test is "flaky" when it's actually broken. Two retries is the right ceiling — anything more is hiding bugs.
• Treating Cloud's flake list as informational and not actionable. The dashboard is only useful if someone fixes the flakes. Without weekly triage, the list grows, the suite drifts, and engineers start ignoring the dashboard the same way they were ignoring red builds before. Cloud doesn't fix tests for you; it surfaces what to fix.

🎯 Practice task

Wire Cypress Cloud into your project end to end. 30-40 minutes.

1. Sign up at cloud.cypress.io. Create a project; copy the project ID into cypress.config.ts (projectId: "abc123"); copy the record key into your CI secret store as CYPRESS_RECORD_KEY.
2. Add record: true to your existing GitHub Actions workflow alongside the matrix from the parallel-execution lesson. Push; confirm the run appears in the Cloud dashboard.
3. Inspect a passing test — click any test in the dashboard; watch the test replay; hover over a cy.get command; confirm Time Travel shows the DOM at that moment.
4. Force a flaky test. Add Math.random() > 0.5 to one assertion. Set retries: { runMode: 2, openMode: 0 }. Run the spec several times. Confirm Cloud marks the test as flaky on runs where it fails-then-passes.
5. Inspect the flake dashboard. Find the most-flaky tests view. Confirm the test you flaked appears with its retry history and links to each failing replay.
6. Smart orchestration drill. Force a different test to fail. On the next push, watch the Actions tab — the failing spec runs first. The first minute of the next CI build tells you whether your fix worked.
7. Stretch: add a second matrix dimension group: "Chrome - PR" and group: "Firefox - PR" (one job per browser). Confirm Cloud shows both groups under the same run with separate pass/fail tallies.

That ends chapter 8. Your suite now lives in CI, runs in parallel, and reports into a dashboard stakeholders can read. The next chapter steps back to framework architecture — folder structure, shared utilities, data factories, and the maintenance habits that keep a 200-spec suite trustworthy after eighteen months.