🟦 AI Product QA

Model answer

The core shift is that there's no single correct output, so exact-match assertions are wrong by construction — they'd flag every valid rephrasing as a failure. Instead I assert properties a good answer must satisfy: structural ones (valid JSON, required fields, length bounds), content ones (mentions the required facts, stays grounded in the provided context, doesn't include forbidden or unsafe content), and behavioural ones (refuses out-of-scope requests). For subjective quality I use a graded rubric — often an LLM-as-judge or human rating against criteria — and assert a score threshold across an eval set rather than pass/fail on one example. I'd run multiple samples per input to characterise the distribution, since one input has a range of outputs, and watch variance, not just the mean. I'd also pin what should be deterministic (temperature 0 for extraction tasks) so I'm only tolerating non-determinism where it's inherent. The principle is testing the qualities of a correct answer, not a canonical answer.

Model answer

An eval set is the AI feature's test suite, so I'd treat curation seriously. Coverage: typical cases that represent real usage, edge cases (ambiguous inputs, long context, mixed languages, empty/garbage), and adversarial cases (injection attempts, requests for unsafe content). Each item has a graded success criterion — a reference answer, a rubric, or a checkable property — so scoring is consistent. I'd version the set and keep it frozen as a baseline so I can detect regression across model/prompt changes, while growing it as new failure modes appear in production. On data sourcing I'm careful: I don't pull real user prompts into the eval without consent, both for privacy and because raw production data can be unrepresentative or sensitive; I prefer synthetic or consented, de-identified examples. I'd balance the set so no single category dominates the aggregate score, and report per-category results, not just an overall number, so a regression in one slice isn't masked. The eval set plus its grading is the artifact everything else measures against.

Model answer

Hallucination in RAG is a grounding failure, so I test whether the answer is actually supported by what was retrieved. For each answer I'd verify claim-level groundedness — every factual statement traces to a passage in the retrieved context — often using an automated faithfulness judge plus spot human checks. Citations must resolve to real, relevant sources, not fabricated or mismatched ones. A crucial case: when the retrieved context doesn't contain the answer, the correct behaviour is to decline or say it doesn't know, not to invent something, so I seed questions whose answer is absent from the corpus and assert a graceful refusal. I'd test context the model should ignore (irrelevant retrieved passages) and conflicting sources. I'd also separate the failure modes — was it bad retrieval (the right doc wasn't fetched) or bad generation (the doc was there but ignored) — because the fix differs. The assertion is faithfulness to retrieved context, with honest abstention as a valid answer.

Model answer

I'd treat the safety guardrail as the system under test and attack it like an adversary. Direct injection: inputs that say 'ignore previous instructions', attempts to extract or leak the system prompt, and role-play framings that try to bypass refusals. Indirect injection is the one people miss — malicious instructions hidden inside content the model ingests, like a retrieved document, a webpage, or a user-uploaded file — so I plant instructions in that data and assert the model treats it as data, not commands. I'd test for forbidden-output elicitation (unsafe, harmful, or policy-violating content) and assert consistent refusal, plus that refusals don't over-trigger on benign requests. I'd maintain an adversarial suite that grows as new jailbreak patterns emerge, and run it on every model/prompt change since resistance can regress. I'd also check that a successful injection can't escalate — e.g. trigger an unauthorised tool call or data access. The mindset is adversarial and ongoing, because the attack surface evolves.

Model answer

I treat a model or prompt change exactly like a code change: it needs regression testing against a baseline. I'd run the frozen, versioned eval suite on both the current and the new version and compare scores per category, not just the aggregate, because a model can improve overall while regressing badly on one slice — and the average hides it. I'd flag any statistically meaningful drop and specifically diff the cases that previously passed and now fail. Because outputs are stochastic, I'd run multiple samples and compare distributions, not single runs, to avoid chasing noise. I'd watch the non-quality dimensions too — latency, token cost, and refusal rate can all regress on a version bump. For higher-stakes changes I'd do an A/B or shadow run in production with online metrics before full rollout. And I'd keep the eval set current so it reflects real failure modes. The discipline is baseline-versioned, per-slice regression with distributional comparison.

Model answer

RAG is a two-stage pipeline, and evaluating only the final answer conflates two different failures, so I test the stages separately. For retrieval, I build a set of queries with known relevant documents and measure precision and recall — did the retriever fetch the right passages, in the right rank order, with sufficient recall to contain the answer — independently of generation. That isolates retrieval problems: bad embeddings, chunking, or indexing. For generation, I feed gold/ideal context and evaluate whether the model uses it faithfully, which isolates generation problems like ignoring context or hallucinating despite having the answer. With both, an end-to-end failure becomes diagnosable: if retrieval recall is low, fix retrieval; if context was present but the answer ignored it, fix generation. I'd also test the interaction — good retrieval with distracting passages, and the no-relevant-document case where the system should abstain. Decomposing the metrics is what makes the failures actionable.

Model answer

For AI products, an answer that's correct but too slow or too expensive is still a failed feature, so I test cost and latency as budgets with thresholds. Latency: I'd assert p50 and p95 response times against an SLA, measure how they scale with prompt/context length and output length, and test streaming so perceived latency is acceptable even when total time is high. Cost: I'd assert per-request token usage stays within budget, watch for prompts that balloon with long context or tool-call loops, and track aggregate cost under realistic traffic. Resilience: a slow or failed model call must hit a timeout and a defined fallback — a cached response, a smaller/faster model, or a graceful degraded message — rather than hanging, so I test the timeout and fallback paths explicitly. I'd also test rate-limit handling and retries with backoff. These become monitored SLOs in production, not just pre-release checks. The point is that cost and latency are quality attributes with budgets, fallbacks, and alarms, exactly like correctness.

Model answer

A static pre-release eval set is necessary but not sufficient, because real inputs, the model, and the world all drift, so I'd build evaluation as a continuous production capability. I'd sample live traffic and grade it on a rolling basis — automated judges for scale plus human review on a sampled and on flagged cases — and track quality, safety, groundedness, cost, and latency as SLOs with dashboards and alarms. I'd add drift detection: shifts in input distribution, output quality, refusal rate, or user signals (thumbs-down, regenerations, escalations) trigger investigation. Safety gets tighter SLAs and fast alerting because a guardrail regression is high-harm. New production failure modes feed back into the frozen eval set so the offline suite keeps catching them. I'd guard rollouts with A/B or shadow evaluation before full ramp. Human-in-the-loop stays for ambiguous and high-stakes cases. I'd frame this to the team as a product-shipping discipline — owning quality over time — which is the complement to the technique-level depth in our Testing-AI-systems topic. The strategy is ongoing measurement, not a one-off gate.