TestNG Listeners — ITestListener, IReporter

Every test run generates events: a test starts, a test passes, a test fails, a suite finishes. Listeners let you hook into those events and add behaviour — capture screenshots on failure, post results to Slack, write a custom HTML report, log timing data — without touching a single test method. This is the separation that makes test infrastructure maintainable: test logic stays in @Test methods, cross-cutting behaviour lives in listeners. This lesson covers the two most important listener interfaces (ITestListener and IReporter), the screenshot-on-failure pattern that appears in almost every Selenium framework, and how to register listeners so they apply globally without repeating code.

ITestListener — react to individual test events

ITestListener fires for each test method throughout its lifecycle:

package com.mycompany.tests.listener;
 
import com.mycompany.tests.util.DriverManager;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.testng.ITestListener;
import org.testng.ITestResult;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
 
public class TestListener implements ITestListener {
 
    @Override
    public void onTestStart(ITestResult result) {
        System.out.printf("▶ STARTED  : %s%n", result.getName());
    }
 
    @Override
    public void onTestSuccess(ITestResult result) {
        long duration = result.getEndMillis() - result.getStartMillis();
        System.out.printf("✅ PASSED   : %s (%dms)%n", result.getName(), duration);
    }
 
    @Override
    public void onTestFailure(ITestResult result) {
        System.out.printf("❌ FAILED   : %s%n", result.getName());
        System.out.printf("   Cause    : %s%n", result.getThrowable().getMessage());
        captureScreenshot(result);
    }
 
    @Override
    public void onTestSkipped(ITestResult result) {
        System.out.printf("⏭ SKIPPED  : %s%n", result.getName());
        Throwable cause = result.getThrowable();
        if (cause != null) {
            System.out.printf("   Reason   : %s%n", cause.getMessage());
        }
    }
 
    @Override
    public void onStart(org.testng.ITestContext context) {
        System.out.printf("🚀 Suite block starting: %s%n", context.getName());
    }
 
    @Override
    public void onFinish(org.testng.ITestContext context) {
        System.out.printf("🏁 Suite block finished: %s — %d passed, %d failed, %d skipped%n",
            context.getName(),
            context.getPassedTests().size(),
            context.getFailedTests().size(),
            context.getSkippedTests().size());
    }
 
    private void captureScreenshot(ITestResult result) {
        WebDriver driver = DriverManager.getDriver();
        if (!(driver instanceof TakesScreenshot ts)) return;
 
        try {
            File screenshot = ts.getScreenshotAs(OutputType.FILE);
            String dirPath = "screenshots";
            Files.createDirectories(Paths.get(dirPath));
            String filename = result.getName() + "_" + System.currentTimeMillis() + ".png";
            Files.copy(screenshot.toPath(),
                       Paths.get(dirPath, filename),
                       StandardCopyOption.REPLACE_EXISTING);
            System.out.printf("   Screenshot: %s/%s%n", dirPath, filename);
        } catch (IOException e) {
            System.err.println("Screenshot failed: " + e.getMessage());
        }
    }
}

The interface has default (empty) implementations for all methods — override only what you need. The most-used methods are onTestFailure (screenshots), onTestStart (logging), and onFinish (summary).

ITestResult gives you everything about the test: getName(), getThrowable(), getStartMillis() / getEndMillis(), and getParameters() (the DataProvider arguments, if any).

IReporter — generate custom reports after the suite

IReporter fires once after all tests complete. Use it to generate any format of report: HTML, JSON, CSV, a Slack message:

package com.mycompany.tests.listener;
 
import org.testng.*;
import org.testng.xml.XmlSuite;
import java.io.*;
import java.nio.file.*;
import java.util.*;
 
public class SummaryReporter implements IReporter {
 
    @Override
    public void generateReport(List<XmlSuite> xmlSuites,
                               List<ISuite> suites,
                               String outputDirectory) {
 
        StringBuilder sb = new StringBuilder();
        sb.append("<!DOCTYPE html><html><head><title>Test Summary</title></head><body>");
        sb.append("<h1>Test Run Summary</h1>");
 
        for (ISuite suite : suites) {
            sb.append("<h2>Suite: ").append(suite.getName()).append("</h2>");
 
            for (Map.Entry<String, ISuiteResult> entry : suite.getResults().entrySet()) {
                ITestContext ctx = entry.getValue().getTestContext();
                int passed  = ctx.getPassedTests().getAllResults().size();
                int failed  = ctx.getFailedTests().getAllResults().size();
                int skipped = ctx.getSkippedTests().getAllResults().size();
                int total   = passed + failed + skipped;
 
                sb.append("<h3>").append(ctx.getName()).append("</h3>");
                sb.append(String.format(
                    "<p>Total: %d | Passed: <span style='color:green'>%d</span> | " +
                    "Failed: <span style='color:red'>%d</span> | " +
                    "Skipped: <span style='color:orange'>%d</span></p>",
                    total, passed, failed, skipped));
 
                // List failed tests with their error messages
                if (failed > 0) {
                    sb.append("<ul>");
                    for (ITestResult result : ctx.getFailedTests().getAllResults()) {
                        sb.append("<li><b>").append(result.getName()).append("</b>: ")
                          .append(result.getThrowable().getMessage())
                          .append("</li>");
                    }
                    sb.append("</ul>");
                }
            }
        }
 
        sb.append("</body></html>");
 
        try {
            Path reportPath = Paths.get(outputDirectory, "custom-summary.html");
            Files.writeString(reportPath, sb.toString());
            System.out.println("Custom report written to: " + reportPath);
        } catch (IOException e) {
            System.err.println("Could not write report: " + e.getMessage());
        }
    }
}

After mvn test, open test-output/custom-summary.html to see the generated report.

Registering listeners

Two ways to register listeners — prefer testng.xml for global application:

In testng.xml (recommended — applies to every test):

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Regression">
    <listeners>
        <listener class-name="com.mycompany.tests.listener.TestListener"/>
        <listener class-name="com.mycompany.tests.listener.SummaryReporter"/>
    </listeners>
    <test name="All Tests">
        <packages>
            <package name="com.mycompany.tests.tests"/>
        </packages>
    </test>
</suite>

With @Listeners annotation on a test class (applies only to that class):

@Listeners({TestListener.class, SummaryReporter.class})
public class LoginTest extends BaseTest { ... }

Use testng.xml registration for cross-cutting concerns like screenshots, logging, and reports — these should apply everywhere. Use @Listeners only when a listener is specific to one test class.

The listener lifecycle

ISuiteListener.onStartSuite begins. Use for suite-level setup…

ITestListener.onStartA <test> block begins. context.getName()…

ITestListener.onTestStartA single @Test method is about to run. r…

onTestSuccess / onTestFailure / onTestSkippedOne of these fires after every @Test. on…

ITestListener.onFinishThe <test> block is complete. context ha…

IReporter.generateReportCalled once after ALL suites finish. All…

Other useful listener interfaces

Interface	When it fires	Common use
`ISuiteListener`	Suite start / finish	Suite-level infrastructure
`IInvokedMethodListener`	Before / after every method (including config methods)	Timing, tracing
`IAnnotationTransformer`	During test discovery	Attach retry analyser globally (next lesson)
`IMethodInterceptor`	After discovery, before first test	Re-order or filter methods at runtime
`IRetryAnalyzer`	After test failure	Retry logic (next lesson)

⚠️ Common mistakes

Calling DriverManager.getDriver() in onTestFailure without a null check. If the test failed in @BeforeMethod before DriverManager.initDriver() was called, getDriver() returns null. A bare ((TakesScreenshot) null).getScreenshotAs(...) throws NPE and loses the original failure in a cascade of teardown errors. Always check driver instanceof TakesScreenshot before attempting the screenshot.
Registering the same listener twice — once in testng.xml and once via @Listeners. TestNG deduplicates at the class level, but if you register two instances of the same class, some events fire twice. One screenshots/testLoginFailed_1234.png becomes two identical files. Register once — in testng.xml for global listeners.
Writing blocking I/O in onTestFailure without a try-catch. A failed screenshot write (disk full, permission denied) throws an unchecked exception inside the listener. TestNG surfaces this as a listener error, obscuring the original test failure. Always wrap file operations in try-catch inside listeners.

🎯 Practice task

Wire up a complete listener. 30–40 minutes.

Implement TestListener with onTestFailure screenshot capture (using DriverManager.getDriver()). Register it in testng.xml. Deliberately fail a Selenium test. Confirm a screenshot appears in screenshots/.
Verify onTestStart / onTestSuccess / onTestSkipped. Add console output to all three. Create a dependency chain where one test skips. Confirm each fires at the right moment.
Implement SummaryReporter. After mvn test, open test-output/custom-summary.html. Add the failed test's duration (from getEndMillis() - getStartMillis()) to the failure list.
Test listener order. Register two ITestListener implementations. Confirm TestNG calls them in the order they appear in testng.xml. Swap the order — observe the output changes.
ITestContext in onFinish. Print the exact number of passed, failed, and skipped tests to the console after each <test> block. Run a suite with two <test> blocks — confirm the counts are per-block, not global.
Stretch — IInvokedMethodListener. Implement IInvokedMethodListener and add timing logs for every @BeforeMethod and @AfterMethod invocation (not just @Test). This is the right place to measure driver startup cost per test.

Next lesson: retry logic — IRetryAnalyzer and IAnnotationTransformer to automatically retry flaky tests without annotation noise.