Running Tests and Reading Results

You've written feature files. Now you run them and read what they tell you. Karate gives you three ways to run tests — Maven, IntelliJ, and the command line with options — and two output formats to read results: the console and an HTML report that it generates automatically. This lesson covers both, plus the debugging techniques that save time when a scenario fails.

Running from Maven

The standard command:

mvn test

Maven compiles the runner class, discovers it via JUnit 5, and Karate executes every feature file wired to that runner. If you have multiple runners across packages, they all run. Output appears in the console in real time.

To point at a specific environment:

mvn test -Dkarate.env=staging

This sets the karate.env variable that karate-config.js reads to switch base URLs and credentials.

To run only scenarios tagged @smoke:

mvn test -Dkarate.options="--tags @smoke"

Tags go on individual scenarios or whole features:

@smoke
Scenario: Health check
  Given path 'health'
  When method get
  Then status 200

Running from IntelliJ

Two options. First: right-click the runner class (e.g., UsersRunner.java) and select Run. This runs every feature file the runner references. Second: right-click directly on a .feature file and select Run. This runs all scenarios in that one file.

With the Karate IntelliJ plugin installed, a green play button appears next to each Scenario: line inside the .feature file. Click it to run a single scenario in isolation — the fastest feedback loop when you're debugging a specific test.

Run configurations in IntelliJ can be saved with VM options (e.g., -Dkarate.env=staging) so you don't retype them each time.

Console output

When tests run, Karate prints each step with its result:

---------------------------------------------------------
feature: classpath:users/users.feature
---------------------------------------------------------
Scenario: Get all users returns a list        # users/users.feature:8
  * url baseUrl                               passed
  * header Accept = 'application/json'        passed
  Given path 'users'                          passed
  When method get                             passed
    response time: 142 ms
  Then status 200                             passed
  And match response == '#array'              passed
---------------------------------------------------------

Failed steps include the expected value, the actual value, and the full path of the assertion that broke:

FAILED  match response.name == 'Bob'
actual:   Alice
expected: Bob

You don't need to add any logging configuration for this output — it's on by default.

The HTML report

After every run, Karate writes a report to target/karate-reports/. Open karate-summary.html in a browser:

The scenario page is the most valuable. For every method call, Karate captures the exact request it sent (URL, headers, body) and the exact response received (status, headers, body). When a test fails, you don't need to reproduce it manually — the report already has the full conversation.

A Cucumber-compatible JSON report is also generated in target/karate-reports/ automatically. Third-party tools like Cluecumber and the Cucumber Reports plugin can consume it for trend dashboards and historical data.

Debugging with print

When a scenario fails and the HTML report doesn't give enough context, add a print step:

Scenario: Debug a failing assertion
  Given path 'users', 1
  When method get
  Then status 200
  * print response
  And match response.name == 'Alice'

* print response dumps the full response object to the console as pretty-printed JSON. It's a quick way to see exactly what the API returned before your assertion runs. Remove print steps before committing — they're debugging scaffolding, not permanent test code.

For structured logging:

* karate.log('User ID from response:', response.id)

karate.log() accepts any JavaScript expression and writes the result to the console alongside the other step output.

What a clean run looks like

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

HTML report: (click to open)
target/karate-reports/karate-summary.html

What a failure looks like:

Tests run: 4, Failures: 1, Errors: 0, Skipped: 0

BUILD FAILURE
...
org.opentest4j.MultipleFailuresError:
  * match failed: response.name
    actual:   Bob
    expected: Alice

The failure message names the exact field, shows the actual value, and shows what you expected. No stack trace digging required.

⚠️ Common mistakes

• Leaving * print response in committed tests. Print statements are for debugging sessions, not for production test suites. They add noise to the console output and slow down large parallel runs. Pull them before committing.
• Not opening the HTML report after CI failures. The console shows the assertion error message, but the HTML report shows the full request and response that caused it. A 401 Unauthorized that looks like a body assertion failure in the console is obvious in the report — the response body is { "error": "Unauthorized" }, not the user object you expected. Always check the report.
• Running mvn test with karate.env unset in CI. Without -Dkarate.env, karate-config.js receives null for karate.env. If your config has if (env == 'staging') branches that set the correct URLs, those branches won't match and tests will point at the default — which may be a local URL that CI can't reach. Always pass the environment flag explicitly in your CI pipeline command.

🎯 Practice task

Run your feature file three different ways and read the output in each format. 25–35 minutes.

1. From the terminal, run mvn test in your project root. Watch the console. Identify the steps, response times, and the final pass/fail summary.
2. Open target/karate-reports/karate-summary.html in a browser. Click through to the feature page, then to a scenario. Find the HTTP request and response body in the scenario detail.
3. Force a failure. In one scenario, change a match assertion to an incorrect value (e.g., assert name == 'WrongName'). Run again. Read the failure message in the console — note it shows expected vs actual. Open the HTML report and find the failed step; confirm the report shows the same information with the full response body alongside it. Restore the correct value.
4. Add * print response as the first assertion step in one scenario. Run. Find the printed JSON in the console output. Remove the print.
5. Add a @smoke tag to one scenario. Run mvn test -Dkarate.options="--tags @smoke". Confirm only the tagged scenario ran (the console shows Tests run: 1).
6. Stretch: open target/karate-reports/ and find the cucumber.json file Karate generated alongside the HTML. Open it — it's a machine-readable version of the same run data. This is the file that third-party dashboard tools consume.

Next chapter: the full Karate HTTP vocabulary — GET, POST, PUT, DELETE, query parameters, headers, and chaining multiple calls in a single scenario.