HOW TO TEST
How to Test Dashboards.
Core Features A complete testing guide for analytics and reporting dashboards: loading and widget rendering, data accuracy, date-range filters, search, sorting, CSV/PDF export, empty/loading/error states, permission scoping, timezones, refresh, responsive layout, and chart accessibility.
Dashboards surface aggregated data that drives business decisions. A bug here is often silent and dangerous: an incorrect calculation is trusted because it looks plausible, an export with missing rows is used in a board presentation, or a chart that displays another tenant's data goes unnoticed. This guide covers the full dashboard testing surface: widget loading and rendering, chart data accuracy against the underlying API, date-range and filter interactions, sorting, CSV and PDF export completeness, empty/loading/error states per widget, permission-scoped data, timezone handling, auto-refresh, responsive layout, and chart accessibility for screen readers. All test cases are written for a Cypress/Playwright engineer who can intercept API calls and compare chart renders against known data sets.
Risks
Incorrect aggregation logic — data silently wrong
Dashboard charts often apply filters, date truncations, and GROUP BY queries that can produce plausible-but-wrong numbers. An off-by-one in the date range (inclusive vs exclusive boundary) or a missing NULL handling condition can cause totals to appear correct while actually being wrong by 5–20%.
Data from another tenant visible to the current user
Dashboard queries that aggregate at the system level without scoping to the authenticated user or organisation expose data from other tenants. In a shared-data aggregation, this may be invisible — both users see the combined total and assume it is their own data.
Export includes rows the user should not see
The CSV export endpoint often uses a separate code path from the UI display. If the export query lacks the same permission scoping as the display query, exported files contain all records, not just those the user is authorised to see.
Stale data displayed due to aggressive caching
Dashboard data is expensive to compute, so it is often cached. Without a clear cache invalidation strategy, users see data that is hours old immediately after an update, making the dashboard misleading for real-time decision-making.
Timezone mismatch causing off-by-one-day errors
A user in UTC+9 viewing a daily chart that stores timestamps in UTC will have a different understanding of 'today' than the server. If timestamps are not converted to the user's timezone before day-bucketing, events appear on the wrong day.
Chart renders but with silently empty data series
If a data series returns an empty array, some chart libraries render an apparently normal chart with an empty area — no zero line, no legend gap, no visual clue. Users assume the data shows 'no activity' rather than a missing data feed.
Export with more than N rows exceeds memory and fails silently
An export query that loads all rows into memory before writing to CSV can exhaust server memory for datasets over a few hundred thousand rows, resulting in a 500 or a corrupt file with no user-visible error.
Test Scenarios
Dashboard loads all widgets within the performance threshold
CriticalperformanceFully automatedNavigate to the dashboard. Assert all widgets render with content (not skeleton loaders) within the defined threshold (e.g. 3 s on a fast connection). Assert no widget is stuck in a loading state.
Chart values match the underlying API response
CriticaldataFully automatedIntercept the API call that populates a specific chart. Assert the displayed values (bar heights, numbers in tooltips, summary tiles) match the values returned by the API — no transformation, rounding, or off-by-one.
Date-range filter changes the displayed data correctly
CriticalfunctionalFully automatedApply a specific date range (e.g. last 7 days). Assert the API request includes the correct date parameters. Assert the chart data changes to reflect only that period. Assert the start and end dates are inclusive as documented.
Widget with no data shows an informative empty state
HighfunctionalFully automatedConfigure a date range or filter that returns no data. Assert the widget shows a 'no data for this period' message — not a blank chart area, a blank page, or a JavaScript error.
Loading state is shown while data is fetching
MediumfunctionalFully automatedThrottle the network to slow-3G or intercept the API call and delay it 3 s. Assert the widget shows a skeleton loader or spinner during the delay — not a blank white area.
Widget shows an error state when the data API fails
HighnegativeFully automatedIntercept the widget's data API call and return a 500. Assert the widget shows an error message with a retry option — not a blank area, a spinner that never resolves, or a JavaScript error.
CSV export contains all rows and correct columns
HighfunctionalFully automatedTrigger a CSV export. Assert the file is downloaded, is valid CSV, has the correct column headers, and contains the expected number of rows matching the displayed count. Assert no rows are truncated.
PDF export renders charts and summary data correctly
MediumfunctionalManual onlyTrigger a PDF export. Assert the PDF is a valid file (not corrupted), opens without errors, and contains the expected charts, summary figures, and correct date range in the title.
Dashboard shows only the authenticated user's or organisation's data
CriticalsecurityFully automatedOrganisation A has 100 events. Organisation B has 50. As Org B's user, assert the dashboard shows 50 — not 150. Cross-validate the chart total against GET /api/analytics which must also return 50 for Org B.
Date grouping respects the user's timezone
HighdataFully automatedA user in UTC+9 creates an event at 2024-01-02T01:00:00+09:00 (which is 2024-01-01T16:00:00Z in UTC). Assert the daily chart shows the event under January 2nd in the user's local timezone, not January 1st.
Auto-refresh updates the dashboard without a full page reload
MediumfunctionalFully automatedIf the dashboard has an auto-refresh interval, create a new event and wait for the refresh cycle. Assert the new event appears in the chart without the user manually reloading. Assert the page does not fully reload (no navigation event).
Dashboard layout is correct on mobile viewport
HighcompatibilityFully automatedResize to 375 × 812 (iPhone 14). Assert widgets stack vertically rather than overflowing horizontally. Assert charts are readable (not cropped). Assert all controls remain accessible.
Charts have accessible text alternatives
HighaccessibilityManual onlyAssert that each chart has an aria-label or role='img' with a descriptive label. For data tables: assert they have proper th elements and a caption. Assert colour is not the only way to distinguish data series.
Detailed Test Cases
Preconditions
- Known dataset: 42 events created on 2024-06-15
- Dashboard has a 'Total events today' tile or bar chart
- API endpoint for the same metric documented: GET /api/analytics/events?date=2024-06-15
Steps
- 1.Intercept GET /api/analytics/* requests using Playwright route interception
- 2.Navigate to the dashboard with date filter set to 2024-06-15
- 3.Capture the API response for the events endpoint
- 4.Assert API response total === 42
- 5.Assert the 'Total events' tile on the dashboard displays '42'
- 6.Assert the bar chart for 2024-06-15 has a bar height proportional to 42 (not 0 or another value)
- 7.Assert the tooltip on hover shows '42 events'
Expected result
Dashboard tile, chart bar, and tooltip all display 42, matching the API response.
Evaluation criteria
- Displayed number must exactly match the API response value (no rounding unless documented)
- Chart bar must be non-zero and proportional
- Tooltip value must match the displayed tile value
Edge Cases
Dashboard with 0 rows of data for the entire time range
A brand-new organisation with no historical data. Every widget should show an empty state, not a JavaScript error or a chart axis with NaN labels.
Date range spanning a DST transition
A date range like 2024-03-09 to 2024-03-11 spans the US daylight saving time transition. Day-bucketing must handle the 23-hour and 25-hour days correctly rather than dropping or duplicating the boundary day.
CSV export initiated while filtering — export should respect active filters
If a user applies a category filter and then exports, the CSV must contain only the filtered rows — not the full dataset. Many export implementations ignore client-side filter state.
Chart with a single data point
A bar chart with exactly one bar, a line chart with exactly one point, or a pie chart with 100% in one category. Some chart libraries crash or render incorrectly with degenerate cases.
Chart with very large numbers (billions) causing label overflow
Y-axis labels for values in the billions can overflow the chart's left margin, overlapping the chart area or getting clipped. The chart should abbreviate (1B, 2.5M) rather than truncate.
Dashboard accessed by a user with read-only access to a subset of resources
If a user can see Projects A and B but not C, the dashboard totals must reflect only A and B. A query that aggregates all records and then post-filters for UI display (rather than pre-filtering in the query) may still expose Project C's total.
Auto-refresh firing while the user is editing a filter
If auto-refresh fires while the user has the date picker open, it resets the displayed data to the previous period while the user is still selecting a new range, causing confusing state changes.
PDF export with charts that require JavaScript rendering
Server-side PDF generators that use headless browsers to render charts require JavaScript to complete before capturing the screenshot. If the capture fires before the chart JavaScript finishes, the PDF contains blank chart areas.
Dashboard widget order changed by a concurrent admin
If an admin reorders dashboard widgets while another user has the dashboard open, the user's layout should update on next load — not immediately (which could be disorienting) — or show a 'layout updated' banner.
Automation Ideas
API-vs-UI data accuracy assertion
Intercept the chart data API call with Playwright, extract the JSON response, and compare the total/series values to what is rendered in the DOM. Run for every chart on the dashboard as a parameterised test.
Tools: playwright, cypress
CSV row count and column validation
Trigger a CSV export, parse the downloaded file (Playwright's download event), count the rows, and compare to the displayed count. Assert column headers match the expected schema.
Tools: playwright
Empty and error state injection via route interception
Use Playwright route interception to return { data: [], total: 0 } for empty state and HTTP 500 for error state. Assert correct widget rendering for each. Run as a suite of 2-minute UI contract tests.
Tools: playwright, cypress
Permission scoping with multiple org accounts
Create Org A and Org B in the test environment via API. Add known event counts. Log in as each org's user and assert the dashboard shows the org-specific count. Run as a security regression on every release.
Tools: playwright
Responsive layout visual regression
Screenshot the dashboard at 375px, 768px, and 1440px viewports. Compare against baseline images. Flag any widget overflow or layout breakage. Run nightly.
Tools: playwright, backstopjs, percy
Load performance benchmark
Use Playwright's Performance API to capture LCP, FCP, and TTFB for the dashboard with a large dataset. Fail the test if LCP > 3,000 ms. Run as a nightly performance gate.
Tools: playwright, lighthouse, grafana
Timezone boundary test via API date injection
Insert events with timestamps at exactly midnight UTC in the test database. Authenticate as users with different timezones and assert each user sees the event on the correct local date. Fully automatable via the API and the dashboard date-filter assertion.
Tools: playwright, cypress
Common Bugs
Date range uses exclusive end date — last day's data missing
The API query uses WHERE date >= startDate AND date < endDate (exclusive end). A user who selects 'Jan 1 to Jan 31' sees data through Jan 30 only. The final day's data is silently excluded.
Impact: Up to one full day of data omitted from every dashboard view and export.
Chart renders 'NaN' or blank when a data series has null values
A data series with one null value in the middle causes the entire chart to display NaN labels or a blank rendering. The null handling must coerce to 0 or skip the point, not propagate as NaN.
CSV export ignores the active category filter
The export function calls the API without the filter parameters the user has set in the UI. The exported CSV contains all records, not just the filtered subset the user intended to export.
Dashboard caches data for 1 hour — misleads users into acting on stale numbers
Aggressive server-side caching means the dashboard can show data that is up to 60 minutes old. Users who make decisions based on 'live' numbers are acting on stale information. The cache age should be visible.
Y-axis does not start from zero — visual amplification of small changes
A chart where the Y-axis starts at 990 instead of 0 makes a drop from 1,000 to 995 look like a 50% decline. Dashboard charts must default to zero-baseline unless there is a clear documented reason.
Permission check on the display query but not on the export query
The dashboard display fetches data with a WHERE userId = ? clause, but the export endpoint uses a separate query without that clause. Exported CSV files contain all users' data.
Impact: Complete data exposure for all users via the export endpoint.
Auto-refresh resets the user's active date range
Auto-refresh re-initialises the dashboard component state, resetting the date picker to the default range. A user who had selected a custom date range loses their selection every refresh cycle.
Chart tooltips show the wrong value on hover
A chart with multiple data series renders tooltips that show the wrong series' value on hover, due to incorrect index mapping between the DOM element and the data array. Particularly common after a data reload that changes the series order.
Summary tiles not updated when the date range changes
The summary tiles (total count, average, top item) are loaded once on page load and are not re-fetched when the date range is changed. The chart updates; the summary tiles remain stale.
Useful Tools
End-to-end dashboard tests: route interception for data stubbing, download handling for CSV assertions, viewport emulation for responsive tests.
Dashboard E2E with cy.intercept for simulating empty/error states and cy.readFile for CSV content assertions.
Performance audits: LCP, FCP, TBT, and accessibility scores for the dashboard page.
Visual regression testing for dashboard layout across viewports — catches overflow and widget misalignment regressions.
Cloud-based visual regression for dashboard snapshots across browsers and viewports.
WCAG 2.1 AA audit of the dashboard — chart aria labels, colour-contrast for data series, and focus management.
Monitor dashboard API response times and error rates during load. Spot latency regressions before they affect production.
Load test the dashboard's data API endpoints under concurrent users to find performance degradation thresholds.
// Related resources