Test Execution & Evidence Pack

Execute a set of 15 test cases across a practice application, record definitive pass/fail/blocked results with traceable evidence notes, cross-reference defects, and produce a stakeholder-ready coverage summary report for a product-owner sign-off meeting.

Role

Manual QA engineer

Difficulty

Intermediate

Time limit

~2 hr

Scenario

Your team has just completed a sprint and the QA sign-off is required before the product owner approves the release. You have been given 15 test cases to execute against the sprint's features. Your task is to execute each test case, record a definitive result (Pass, Fail, Blocked, or Not Run), write an evidence note for each Pass, file a bug reference for each Fail, document the blocker for each Blocked test, note the reason for any Not Run test, and produce a one-page coverage summary report that the PO can use to make the go/no-go release decision at the end of the sprint.

Requirements

1.Execute each of the 15 provided test cases and record a definitive result: Pass, Fail, Blocked, or Not Run — every test case must have a result and a brief note; a blank result is not acceptable
2.For every Pass result, write a one-sentence evidence note describing what you observed that confirms the expected result was met (e.g. 'Order confirmation page displayed with the correct order summary and a THANK YOU message'); 'it passed' is not an evidence note
3.For every Fail result, file a bug reference in the execution log: bug ID, a one-line summary of the defect, and the severity; the full bug report is a separate artefact, but the cross-reference in the log is mandatory
4.For every Blocked test case, record the specific blocker (e.g. 'build not deployed to staging', 'test account password reset not completed', 'dependent feature TC-008 fails, preventing TC-011 from running') and note whether the block is known to the team
5.For any Blocked or Fail result from a previous execution cycle that is being retested in this cycle, write a retest note recording: previous result, fix applied by the developer, retest result, and date
6.Produce a one-page coverage summary report suitable for a PO sign-off meeting containing: total test count, Pass/Fail/Blocked/Not Run counts, P1-only pass rate, count of open defects by severity, outstanding risks, and a go/no-go recommendation with a brief rationale

Starter data

›TC-001 (P1) Login with valid credentials — expected: dashboard/product list loads | use standard_user / secret_sauce on saucedemo.com
›TC-002 (P1) Login with invalid password — expected: error message displayed | use standard_user / wrong_password
›TC-003 (P1) Add item to cart — expected: cart badge increments to 1 | add any item on the inventory page
›TC-004 (P1) Remove item from cart — expected: item removed, cart badge decrements | add then remove Sauce Labs Backpack
›TC-005 (P1) Proceed to checkout with items in cart — expected: checkout step 1 (shipping info) shown
›TC-006 (P2) Proceed to checkout with empty cart — expected: error message or redirect to product list | navigate directly to /checkout-step-one.html without adding items
›TC-007 (P1) Complete full checkout — expected: order confirmation ('THANK YOU FOR YOUR ORDER') shown | use First Name: Test, Last Name: User, ZIP: 12345
›TC-008 (P2) Apply promo code at checkout — expected: discount reflected in order total | note: saucedemo.com does not have promo codes
›TC-009 (P1) Logout — expected: session ends, redirect to login page
›TC-010 (P2) Profile edit — update display name — expected: name saved and shown in header | note: saucedemo.com does not have a profile edit feature
›TC-011 (P2) Password change — expected: new password accepted on subsequent login | note: saucedemo.com does not support password change
›TC-012 (P1) Forgot password — expected: password reset email received | note: saucedemo.com does not have a forgot-password flow
›TC-013 (P3) Pagination on product list — expected: next page loads with different products | note: saucedemo.com shows all 6 items on one page — pagination not applicable
›TC-014 (P2) Search — valid keyword — expected: relevant products shown | note: saucedemo.com does not have a search field
›TC-015 (P3) Sort by price (low to high) — expected: products reordered with cheapest first | use the sort dropdown on the inventory page (saucedemo.com)

Expected deliverables

✓A test execution log for all 15 test cases with columns: ID, title, priority, result (Pass/Fail/Blocked/Not Run), evidence note or bug reference or blocker note or not-run reason
✓Evidence notes for all Pass results — one sentence per test case describing the specific observation that confirmed the expected result
✓Bug references for all Fail results — bug ID, one-line summary, severity — cross-referenced in the execution log
✓Blocker notes for all Blocked results — specific blocker description, whether the block is known to the team, and the expected resolution path
✓Not Run reasons for all Not Run results — specific reason (practice-store limitation, out-of-scope for this sprint, prerequisite test failed, etc.)
✓A one-page coverage summary report with total/pass/fail/blocked/not-run counts, P1 pass rate, open defect count by severity, outstanding risks, and a go/no-go recommendation

Evaluation rubric

Dimension	What reviewers look for
Execution rigour	Is every test case given a definitive, correctly assigned status? The Blocked vs. Fail distinction is critical: Blocked means the test could not run (environment, data, or dependency issue); Fail means it ran and produced the wrong result. Are Pass/Fail results grounded in actual observations, not assumptions? A test marked Pass because 'it usually works' is not execution rigour.
Evidence quality and traceability	Does each Pass evidence note describe a specific, observable system response that confirms the expected result? 'TC-007 PASS — order confirmation page loaded with header THANK YOU FOR YOUR ORDER and a Back Home button' is traceable. 'It passed, looked correct' is not. Is the connection between a Fail result and its bug reference explicit (bug ID in the log row)?
Clear pass/fail/blocked/not-run status	Is the status column consistent and complete? No test case is left blank. Blocked statuses include a specific blocker (not 'couldn't test'). Not Run statuses include an explicit reason (not 'skipped'). A status of 'N/A' without explanation is not acceptable — the tester must decide whether the test is Blocked (a recoverable blocker) or Not Run (out of scope or environment limitation) and record why.
Coverage communication	Does the summary report give a stakeholder everything they need for a go/no-go decision without reading the full execution log? The most important metrics — P1 pass rate, count of open Critical/High defects — must be prominent. A report showing '14 Pass, 1 Not Run' without distinguishing which priority the Not Run test was is not fully informative. The PO needs to know: are all P1 tests green?
Stakeholder-readiness	Is the summary report written for a non-QA stakeholder? Test-case IDs should be referenced by ID only in the summary (not repeated in full). Defects should be described in plain terms (e.g. 'Checkout allows empty cart — Medium severity, fix scheduled for next sprint') not in QA jargon. The go/no-go recommendation must be a clear sentence: 'PASS — all P1 test cases pass; 1 Medium defect accepted as known issue with workaround documented' or 'BLOCK — TC-007 fails with a Critical defect in the order confirmation flow'.

Sample solution outline

›TC-001 PASS — logged in with standard_user / secret_sauce; inventory page loaded with 6 products and the Swag Labs header; cart badge not present (empty cart state correct)
›TC-002 PASS — entered standard_user / wrong_password; red error banner displayed: 'Username and password do not match'; no login occurred
›TC-003 PASS — clicked 'Add to cart' on Sauce Labs Backpack; cart badge in header incremented from empty to '1'
›TC-004 PASS — added Backpack then clicked 'Remove'; cart badge disappeared (0 items); cart page shows empty cart message
›TC-005 PASS — with Backpack in cart, clicked Cart icon then 'Checkout'; checkout step 1 form (First Name, Last Name, ZIP) displayed correctly
›TC-006 FAIL — navigated to /checkout-step-one.html with empty cart; checkout step 1 form loaded without error; expected: redirect to product list or 'Your cart is empty' error; BUG-001 (Medium): Checkout accessible with empty cart — no guard prevents empty-cart checkout
›TC-007 PASS — entered First Name: Test, Last Name: User, ZIP: 12345; clicked Continue; order summary shows Sauce Labs Backpack $29.99, Tax $2.40, Total $32.39; clicked Finish; THANK YOU FOR YOUR ORDER page displayed with a green checkmark and Back Home button
›TC-008 BLOCKED — saucedemo.com does not have a promo code feature; blocker: practice-environment limitation (not a product defect); known to team: yes (documented in test plan); resolution: this test case requires a real e-commerce environment with discount-code support
›TC-009 PASS — clicked hamburger menu → 'Logout'; redirected to login page; attempting to navigate back with browser Back button returns to login (session correctly ended)
›TC-010 / TC-011 / TC-012 NOT RUN — saucedemo.com does not have profile editing, password change, or forgot-password features; reason: practice-environment limitation; note: these are P1/P2 tests in a real product and would require a full-featured staging environment
›TC-015 PASS — selected 'Price (low to high)' from sort dropdown; products reordered: Sauce Labs Onesie $7.99, Sauce Labs Bike Light $9.99, Sauce Labs Bolt T-Shirt $15.99, Test.allTheThings() T-Shirt $15.99, Sauce Labs Backpack $29.99, Sauce Labs Fleece Jacket $49.99 — correct ascending order
›Summary: 15 TCs total; 8 PASS, 1 FAIL (TC-006, Medium), 1 BLOCKED (TC-008), 5 NOT RUN (practice limitation); P1 pass rate: 6 of 7 P1 tests executable → 6 PASS (100% of executable P1); 1 Fail (TC-006 Medium); 0 Critical/High open defects; Recommendation: CONDITIONAL PASS — all executable P1 tests pass; TC-006 (Medium) accepted as known issue; 5 tests NOT RUN due to practice-environment limitations (not product defects)

Common mistakes

Leaving test cases without a result — every test case must have a status; a blank or 'TBD' result means the test was not executed, which is itself a Not Run status that needs a reason
Writing 'PASS' without an evidence note — 'it worked' or 'looks correct' is not an evidence note; an evidence note must describe the specific system response that was observed to confirm the expected result
Confusing Blocked with Fail — a test is Blocked when it cannot run due to an external dependency (environment not ready, test data missing, prerequisite test failed); it is a Fail only when it ran and produced the wrong result; recording a Blocked test as a Fail inflates the defect count
Not filing a bug reference for Fail results — the evidence pack must cross-reference the defect record; 'it failed, I'll file it later' is not a complete execution log; a stakeholder reviewing the pack needs to see the bug ID and severity to assess risk
Reporting a pass rate that excludes Blocked and Not Run without acknowledging the gap — '8/8 PASS (100%)' calculated only over executed tests while 7 tests were not run is misleading; the summary must state the total count, the executed count, and the not-run count separately
Producing a summary report that repeats every test case row — the summary is for the PO, not for the QA team; it must be a high-level view with counts, rates, risks, and a recommendation; the full execution log is the supporting detail, not the summary

Submission checklist

Test execution log for all 15 test cases with ID, title, priority, result, and an evidence/bug/blocker/reason note on every row
Every test case has a definitive status (Pass/Fail/Blocked/Not Run) — no blank rows
All Pass results have a specific one-sentence evidence note
All Fail results have a bug reference (bug ID, one-line title, severity) in the log
All Blocked results have a specific blocker description and whether the block is known to the team
All Not Run results have an explicit reason
Coverage summary report with total/pass/fail/blocked/not-run counts and P1-only pass rate
Go/no-go recommendation stated clearly with a one-sentence rationale

Extension ideas

+Re-execute the same test suite logged in as problem_user (saucedemo.com) — this user has injected UI defects; identify which test cases fail, file bug references, and update the summary report
+Design a retest cycle: assume BUG-001 (empty-cart checkout) has been fixed by the developer; write the retest procedure, document the expected fix, and record the retest result
+Convert the coverage summary report into a slide-deck format suitable for a 5-minute sprint review presentation — identify which data points matter most to a non-technical PO and which can be omitted