Q17 of 21 · AI for testing
How do you prevent AI from baking bad patterns into your test suite?
Short answer
Short answer: Define and enforce your conventions before adding AI to the workflow: a style guide for the test suite, lint rules for common anti-patterns, and few-shot examples in your prompts that show the AI what good output looks like. Review AI output against the style guide, not just for functionality.
Detail
AI generates code by pattern-matching against its training data and your examples. If your codebase has cy.wait(3000) in 40 tests, AI will generate cy.wait(3000) for the 41st. If you prompt with no conventions, it picks reasonable-looking ones that may not match your team's standards.
Prevention strategies:
Negative examples in the prompt: "Do not use hard waits. Do not use CSS selectors that include component class names. Do not assert on text that is likely to change with content updates."
Lint rules as a safety net: ESLint or custom rules that flag cy.wait with a number argument, raw CSS class locators, or empty assertions catch what slips past prompt engineering.
Reviewed example tests as few-shot context: paste 2–3 exemplary tests from your suite into the prompt. The model imitates concrete examples far more reliably than text descriptions of conventions.
AI-generated tag in PR: when a test is AI-generated, tag it so reviewers know to apply extra scrutiny to patterns, not just functionality.
See AI refactoring for test code for patterns to reverse existing bad practices.