Q31 of 38 · Performance

How do you test the performance impact of caching (Redis, CDN, database query cache) in a load test?

PerformanceMidperformancecachingrediscdncold-cachewarm-cacheload-testing

Short answer

Short answer: Run the test in two modes: cold-cache (cache cleared before run) and warm-cache (cache seeded or pre-warmed). The difference in p95 between the two modes tells you the cache's contribution to performance.

Detail

Caching is one of the largest performance multipliers in most systems, but a load test that always runs warm-cache will produce results that do not reflect the worst-case post-deploy scenario (when the cache is empty and every request is a cache miss).

For a meaningful test:

  1. Cold-cache run: flush Redis and invalidate CDN before starting. This measures worst-case origin load — what happens when you deploy and cache is empty.
  2. Warm-cache run: run a short warm-up pass before measuring, or seed the cache explicitly. This measures steady-state performance.

The ratio between the two results is your "cache multiplier." If warm-cache p95 is 80 ms and cold-cache p95 is 2,000 ms, you know the system is critically cache-dependent — a thundering-herd scenario during a deploy will overwhelm the origin until the cache warms.

For CDN testing, use a tool that can route requests to the CDN and measure cache hit rate alongside latency. A CDN test where 100% of requests miss the cache is not representative.

// WHAT INTERVIEWERS LOOK FOR

The cold-cache vs warm-cache distinction and what each measures. The thundering-herd risk when the cache is empty post-deploy. Measuring cache hit rate alongside latency.