Review and Stretch Goals

If you have worked through all five chapters and completed the capstone framework, stop and appreciate what you have built. A working Selenium + TestNG framework is not a toy — it is the kind of artifact that takes junior engineers months to produce on the job. You have thread-safe parallel execution, data-driven tests reading from external files, automatic retry logic, screenshot-on-failure reporting, and a CI pipeline that runs without human intervention. This lesson is the self-assessment: a checklist of what a complete framework looks like, the stretch goals for pushing further, and a map of where to go next.

Self-assessment checklist

Work through this list against your GitHub repository. Be honest — "mostly works" is not the same as "works cleanly."

Foundation

• pom.xml has correct <scope>test</scope> on TestNG and all test-only dependencies
• maven-surefire-plugin pinned to 3.2.5 or higher
• src/test/resources/ contains smoke.xml, regression.xml, and cross-browser.xml
• Running mvn clean test -DsuiteXmlFile=smoke.xml from a fresh clone produces BUILD SUCCESS

Test structure

• All @Test methods have a description attribute — no unnamed tests
• All @Test methods are tagged with at least one group (smoke, regression, wip)
• @BeforeMethod(alwaysRun = true) is absent — alwaysRun belongs on @AfterMethod
• @AfterMethod(alwaysRun = true) guards every driver.quit() call with a null check

Data-driven

• At least one @DataProvider reads from a JSON file via getClassLoader().getResourceAsStream()
• At least one @DataProvider reads from an Excel file via Apache POI
• Neither provider uses new FileReader("path") — classpath loading only
• Each data row produces a separate, individually-named result in the TestNG report

Advanced TestNG

• DriverManager uses ThreadLocal<WebDriver> with driver.remove() in teardown
• RetryAnalyzer MAX_RETRIES is 2 or less
• RetryTransformer is registered in testng.xml — no retryAnalyzer = on individual tests
• RetryResultCleaner removes initial failures for tests that eventually pass
• parallel="classes" or parallel="methods" is enabled in regression.xml with thread-count="2"

Reporting

• TestListener.onTestFailure captures a screenshot to reports/screenshots/
• ExtentReports or Allure generates a report after every run
• test-output/, reports/, and screenshots/ are all in .gitignore
• reports/extent-report.html (or Allure results) is self-contained and opens without a server

CI/CD

• .github/workflows/testng.yml runs on push and PR
• Chrome flags --headless=new --no-sandbox --disable-dev-shm-usage are applied in CI
• Reports are uploaded with if: always()
• Secrets (base URL, credentials) are injected via GitHub Actions secrets — not hardcoded in XML or Java
• Nightly regression is scheduled with a cron trigger

If five or more items are unchecked, pick the most impactful gap and fix it before moving to stretch goals. A framework that passes the checklist is far more valuable than one with extra features that doesn't run reliably.

Stretch goal 1 — @Factory for multi-environment runs

Create EnvironmentFactory.java:

@Factory(dataProvider = "environments")
public LoginTest(String env, String baseUrl) {
    this.env     = env;
    this.baseUrl = baseUrl;
}
 
@DataProvider(name = "environments")
public static Object[][] environments() throws Exception {
    // Load from src/test/resources/environments.json
    // [{"env": "staging", "baseUrl": "..."}, {"env": "production", "baseUrl": "..."}]
    return DataReader.fromJson("environments.json");
}

A factory-suite.xml with parallel="instances" thread-count="2" runs the same tests against two environments concurrently. The report shows results grouped by environment — immediately obvious which environment has a failure.

Stretch goal 2 — IMethodInterceptor for priority-first execution

Add @Severity(Level.CRITICAL) to your 5 smoke tests and @Severity(Level.HIGH) or @Severity(Level.MEDIUM) to the rest. Implement SeverityInterceptor from Chapter 4. Register it in regression.xml.

The practical payoff: when a CRITICAL test fails 2 minutes into a 30-minute regression run, you know immediately rather than after the full suite completes. A fast signal on the most important tests is one of the things that separates a thoughtful framework from a script that just runs tests.

Stretch goal 3 — Custom @Owner annotation and report grouping

Every unchecked item in a failure report triggers a "who owns this?" conversation. Eliminate that friction:

@Test(groups = {"smoke"})
@Owner("alice")
public void validLoginSucceeds() { ... }

In SummaryReporter.generateReport(), group failures by owner:

Map<String, List<ITestResult>> byOwner = new HashMap<>();
for (ITestResult r : ctx.getFailedTests().getAllResults()) {
    Method m = r.getMethod().getConstructorOrMethod().getMethod();
    Owner o = m.getAnnotation(Owner.class);
    String owner = o != null ? o.value() : "unassigned";
    byOwner.computeIfAbsent(owner, k -> new ArrayList<>()).add(r);
}
// Write one section per owner in the HTML report

The report now says: "alice: 2 failures, bob: 0 failures, unassigned: 1 failure." Teams know immediately who to contact.

Stretch goal 4 — Allure trend history

Set up allure-maven and configure the GitHub Actions workflow to carry allure-history between runs:

- name: Get Allure history
  uses: actions/checkout@v4
  with:
    ref: gh-pages
    path: gh-pages
 
- name: Copy history
  run: cp -r gh-pages/history allure-results/history || true
 
- name: Generate Allure report
  run: mvn allure:report
 
- name: Deploy to GitHub Pages
  uses: peaceiris/actions-gh-pages@v3
  with:
    github_token: ${{ secrets.GITHUB_TOKEN }}
    publish_dir: target/site/allure-maven-plugin

After 5–10 CI runs the Allure dashboard shows a trend line: pass rate over time, average duration, flakiest tests. This is the kind of observability that makes QA teams genuinely useful at sprint retros.

Where to go next

What you have built

The framework you have completed demonstrates:

• TestNG mastery: all ten lifecycle annotations, three parallel modes, group-based suite management, @DataProvider from three external formats, @Factory for multi-instance runs, IAnnotationTransformer, ITestListener, IRetryAnalyzer
• Selenium discipline: ThreadLocal<WebDriver> for thread safety, @AfterMethod(alwaysRun = true) for leak-free teardown, headless execution in CI
• Data engineering: classpath-safe file loading, POJO-backed JSON parsing, Excel row-to-Object[][] conversion, a DataReader utility that keeps test classes clean
• CI/CD integration: GitHub Actions with secrets, headless Chrome flags, parallel job structure, nightly scheduling, artifact upload with if: always()

Show this repository when a recruiter asks "can I see some of your automation work?" The checklist above is a reasonable proxy for what a senior QA engineer would use to evaluate it. Pass the checklist and you have a portfolio piece that stands up to scrutiny.

Final thought

The TestNG features covered in this course are the same ones used in every large-scale Java automation project. The skills transfer directly: the next time you join a company with a Selenium + TestNG suite, you will recognise the patterns immediately — BaseTest, DriverManager, listeners, retry, data providers — and you will be able to contribute from day one. That readiness is the real deliverable.

The Selenium with Java course covers the browser-interaction side in depth — locator strategy, waits, the Page Object Model, Selenium Grid, and cross-browser testing — if you want to continue building on this foundation.