Cohort analysis
// Definition
Cohort analysis groups users by a shared characteristic at a specific point in time — usually when they first signed up or first performed an action — and tracks their behaviour across subsequent periods. It is the standard method for measuring retention without the distortion that new-user growth introduces: rising daily active users can mask worsening retention if you acquire users faster than you lose old ones. A cohort retention chart has signup week on one axis, weeks since signup on the other, and retention rate as the cell value. The signature of product-market fit is a retention curve that flattens above zero rather than trending toward zero. For QA engineers, cohort analysis resembles regression tracking across release cycles: both ask whether a population maintains expected behaviour over time, just at different scales.
// Related terms
AARRR metrics (Pirate metrics)
AARRR — Acquisition, Activation, Retention, Referral, Revenue — is a funnel framework coined by Dave McClure for measuring product lifecycle health. Acquisition: how do users find you? Activation: do they experience core value in the first session? Retention: do they return? Referral: do they tell others? Revenue: does the interaction generate money? Each stage surfaces different problems and often has different owners: marketing drives acquisition, product owns activation and retention. The framework's value is forcing you to look at the whole funnel — optimising acquisition while ignoring a leaky Retention stage is expensive failure. QA engineers naturally think in pipelines and staged execution; AARRR maps cleanly onto that mental model. The new skill is instrumenting each stage with real behavioural metrics rather than coverage percentages or defect counts.
North Star metric
A North Star metric is the single number that best captures the core value a product delivers to customers — the one that, if it grows sustainably, predicts long-term business health. Spotify's was time-in-app; Airbnb's was nights booked. The North Star sits above vanity metrics (page views, sign-ups) and below revenue — it measures the value exchange, not the financial outcome. Choosing the wrong metric drives the wrong behaviour at scale: an engagement metric optimised without guardrails can increase usage while eroding user wellbeing. Supplement the North Star with counter-metrics to prevent gaming. For testers becoming PMs, this is a genuine mindset shift: QA thinks pass/fail (binary, per-release, reversible), while product thinks directional (continuous, long-horizon, probabilistic). Learning to act confidently on a trend rather than a verdict is the core adjustment.
Product-market fit
Product-market fit is the degree to which a product satisfies strong market demand — a state where users acquire, retain, and refer at rates that sustain growth without heavy marketing spend. Marc Andreessen defined it as "being in a good market with a product that can satisfy that market." Sean Ellis's heuristic: if more than 40% of surveyed users would be "very disappointed" if the product disappeared, PMF is likely. Before PMF, optimise for learning, not efficiency — every process and metric should target understanding what the market actually wants. After PMF, optimise for growth. For QA engineers pivoting to product, PMF reframes the entire quality conversation: before PMF you are validating the concept, not production polish; after PMF, reliability and quality become competitive differentiators that directly affect retention.