Test summary report
End-of-cycle test results report. Coverage, pass/fail counts, defects found, and risk assessment.
Test Summary Report ā Product Name / Sprint or Release
Report ID: TSR-XXX Prepared by: Name Date: YYYY-MM-DD Sprint / Release: Sprint X / v0.0 Report period: Start date ā End date
Executive Summary
2ā4 sentences. State the overall quality signal: what was tested, what the pass rate was, whether the product is ready for the next stage, and the biggest risk or finding.
Test Coverage
| Test Type | Planned | Executed | Passed | Failed | Blocked | Pass Rate |
|---|---|---|---|---|---|---|
| Functional | X | X | X | X | X | X% |
| Regression | X | X | X | X | X | X% |
| Exploratory sessions | X | X | ā | ā | ā | ā |
| Performance | X | X | X | X | ā | X% |
| Total | X | X | X | X | X | X% |
Defects Summary
| Severity | Raised | Fixed | Closed | Open |
|---|---|---|---|---|
| Critical | X | X | X | X |
| High | X | X | X | X |
| Medium | X | X | X | X |
| Low | X | X | X | X |
| Total | X | X | X | X |
Open Defects
List defects that remain open at report date. Include ID, title, severity, owner, and status.
| ID | Title | Severity | Owner | Status |
|---|---|---|---|---|
| BUG-XXX | Title | Sev | Name | Status |
Test Environment Notes
Any environment issues that affected test results ā instability, downtime, data resets, etc.
Exploratory Testing Findings
Summarise findings from exploratory sessions that are not captured in formal test case results.
Risk Assessment
What risks remain? What coverage gaps exist? What should the next stage watch out for?
| Risk | Description | Likelihood | Impact | Mitigated? |
|---|---|---|---|---|
| Risk | Description | H / M / L | H / M / L | Yes / No |
Recommendation
Go / No-go / Conditional go ā and the reasoning.
Appendix: Not-executed Test Cases
List test cases that were planned but not executed, with reason.
| Test Case ID | Reason Not Executed |
|---|---|
| TC-XXX | Reason |
Test Summary Report ā Novu Bank v3.2 / Sprint 24
Report ID: TSR-2024-032 Prepared by: Priya Mehta Date: 2024-03-13 Sprint / Release: Sprint 24 / v3.2 Report period: 2024-03-06 ā 2024-03-13
Executive Summary
Sprint 24 tested the v3.2 feature set (Open Banking integration, biometric re-authentication, notifications centre) plus a full regression run. Of 247 test cases executed, 215 passed (87%). Eleven defects were raised; nine are resolved. Two High-severity defects remain open but are targeted for fix before the 2024-03-14 go/no-go decision. The product is conditionally ready for UAT, pending resolution of NOVU-2041 and NOVU-2042.
Test Coverage
| Test Type | Planned | Executed | Passed | Failed | Blocked | Pass Rate |
|---|---|---|---|---|---|---|
| Functional (feature) | 112 | 110 | 93 | 15 | 2 | 84.5% |
| Regression (Appium) | 210 | 210 | 205 | 5 | 0 | 97.6% |
| Exploratory sessions | 3 | 3 | ā | ā | ā | ā |
| Performance | 4 | 4 | 3 | 1 | 0 | 75% |
| Total | 329 | 327 | 301 | 21 | 2 | 87.0% |
Defects Summary
| Severity | Raised | Fixed | Closed | Open |
|---|---|---|---|---|
| Critical | 0 | 0 | 0 | 0 |
| High | 3 | 1 | 1 | 2 |
| Medium | 5 | 5 | 5 | 0 |
| Low | 3 | 3 | 3 | 0 |
| Total | 11 | 9 | 9 | 2 |
Open Defects
| ID | Title | Severity | Owner | Status |
|---|---|---|---|---|
| NOVU-2041 | Open Banking: no timeout message when provider redirect exceeds 30s | High | Ayo Adeyemi | In Progress ā fix ETA 2024-03-14 08:00 |
| NOVU-2042 | Open Banking: state_mismatch error on back-button is not user-friendly | High | Marcus Obi | In Review ā PR raised 2024-03-12 |
Test Environment Notes
- Build delivery delay (Day 3): The v3.2 build wasn't available in QA until 14:00 on 2024-03-06 (planned: 09:00). Two test cases were blocked until 2024-03-07.
- Appium flaky tests quarantined: NOVU-AUTO-118, 119, 124, 131 (biometric re-auth flow) were quarantined on 2024-03-10 after failing on 4 of 18 nightly runs. Root cause investigation is in Sprint 25.
- Performance environment stable: No outages recorded. Load test ran successfully across all 18 runs.
Exploratory Testing Findings
Three exploratory sessions were conducted (EXP-2024-045, 046, 047). Key findings:
- NOVU-2041 (High): Provider redirect timeout produces an infinite spinner with no user feedback. Filed from EXP-2024-047.
- NOVU-2042 (High): Browser back-button mid-consent produces a raw
state_mismatcherror string. Filed from EXP-2024-047. - Partial scope connection (positive): When a user declines transaction history permission, the app correctly shows a "Partial connection" state with a clear explanation ā not filed as a defect.
- Tampered OAuth state (positive): App correctly rejects tampered
stateparameter with a user-friendly error and no stack trace exposure.
Risk Assessment
| Risk | Description | Likelihood | Impact | Mitigated? |
|---|---|---|---|---|
| NOVU-2041 fix not ready in time | Timeout fix is in progress but not yet in QA | Medium | High | Partial ā fix ETA confirmed |
| Mid-redirect bank session expiry | Behaviour is undefined in spec; no test coverage | Low | Medium | No ā product to clarify in Sprint 25 |
| Quarantined flaky tests | 4 automation tests excluded from pass-rate calculation | Low | Low | Partial ā quarantined, root cause pending |
Recommendation
Conditional Go ā subject to the following conditions being met before 2024-03-14 14:00:
- NOVU-2041 fix deployed and verified in QA environment
- NOVU-2042 PR merged, deployed, and smoke-tested
If both conditions are met, UAT can proceed on 2024-03-14 as planned. If not, recommend a 24-hour hold.
Appendix: Not-executed Test Cases
| Test Case ID | Reason Not Executed |
|---|---|
| TC-1092 | Blocked pending build ā executed on Day 2 (included in count above) |
| TC-1108 | Blocked pending environment access ā resolved; test executed 2024-03-07 |
// Related templates
Test closure report
Sign-off report when testing completes. Final results, lessons learned, and recommendations.
Defect triage notes
Triage meeting agenda, defect prioritisation grid, and decisions log. For weekly QA syncs.
Test plan
Scope, approach, schedule, risks, entry/exit criteria. IEEE 829-style structure.