Performance Test Report — Product Name v0.0

Document ID: PTR-XXX Author: Name Date: YYYY-MM-DD Related Plan: PTP-XXX

---

1. Executive Summary

• Overall verdict: PASS / FAIL / CONDITIONAL PASS
• Key finding 1
• Key finding 2
• Key finding 3
• Recommendation: proceed / hold / fix before release

---

2. Test Configuration

Item	Detail
System under test	Product name and version
Environment	Environment name and key specs
Load generation tool	Tool and version
Test dates	Start date — end date
Executed by	Name / team
Related plan	PTP-XXX

Note any deviations from the test plan.

---

3. NFR Results

NFR	Target	Observed	Pass / Fail	Notes
NFR 1	Target value	Observed value	Pass / Fail	Context
NFR 2	Target value	Observed value	Pass / Fail	Context
NFR 3	Target value	Observed value	Pass / Fail	Context
NFR 4	Target value	Observed value	Pass / Fail	Context

---

4. Detailed Results

4.1 Load Test Results

[Chart placeholder — insert p50 / p95 / p99 response time chart and RPS chart]

Metric	Value
Duration	X min
Peak RPS achieved	X
p50 response time	X ms
p95 response time	X ms
p99 response time	X ms
Error rate	X %
Peak concurrent VUs	X

Observations: describe what happened during the run.

4.2 Stress Test Results

[Chart placeholder — show the ramp and the point where degradation began]

Metric	Value
Breaking point (RPS)	X
Failure mode	e.g. HTTP 503 / timeout / connection refused
First sign of degradation	X RPS / X VUs

Observations.

4.3 Spike Test Results

Metric	Value
Spike load	X RPS
Error rate during spike	X %
Time to p95 recovery after spike	X s

Observations.

4.4 Soak Test Results

Metric	Value
Duration	X h
p95 at start of run	X ms
p95 at end of run	X ms
Drift (increase)	X ms / X %
Memory usage trend	Stable / Growing at X MB/h

Observations.

---

5. Bottlenecks Observed

Component	Symptom	Suspected cause	Evidence	Severity
Component	e.g. p95 spikes above 2 s	e.g. DB connection pool exhaustion	e.g. Datadog: pool wait > 1 s at 800 RPS	Critical / High / Medium / Low

If no bottlenecks were observed, state that explicitly.

---

6. Comparison to Baseline

Metric	Baseline (run PTR-XXX)	This run	Change	Regression?
p95 response time (load)	X ms	X ms	+/- X ms	Yes / No
Peak RPS sustained	X	X	+/- X	Yes / No
Error rate (load)	X %	X %	+/- X %	Yes / No

Baseline was run on YYYY-MM-DD against version X.X in environment Y.

---

7. Recommendations

Fix now (blocking release)

• Issue description and suggested fix

Fix next sprint

• Issue description and suggested fix

Fix eventually (backlog)

• Issue description and suggested fix

Accept (known limitation)

• Issue description and rationale for acceptance

---

8. Appendix

• Raw k6 / JMeter output: location or link
• Grafana dashboard snapshot: link or attachment
• Test scripts: repository path
• Tool versions: list of tool versions used

Performance Test Report — AcmeCorp Checkout API v2.4

Document ID: PTR-2024-011 Author: Jordan Osei Date: 2024-04-22 Related Plan: PTP-2024-011

---

1. Executive Summary

• Overall verdict: CONDITIONAL PASS — three of four NFRs passed; one failed (error rate during soak).
• Checkout initiation p95 met the 800 ms SLA under all test types including the 500 RPS sustained load test.
• A database connection pool bottleneck caused error rates to spike to 0.4% (target: < 0.1%) during the 3-hour soak run at 500 RPS.
• No regressions against the v2.3 baseline for throughput or response time NFRs.
• Recommendation: hold release pending DB connection pool tuning. Estimated fix: 1 day. Re-run soak test before release sign-off.

---

2. Test Configuration

Item	Detail
System under test	AcmeCorp Checkout API v2.4
Environment	perf.acmecorp.internal — 3 × t3.large (ECS Fargate), RDS db.r6g.large
Load generation tool	k6 v0.50, 3 distributed generators in eu-west-1
Test dates	2024-04-15 – 2024-04-18 (soak overnight 2024-04-17)
Executed by	Jordan Osei, QA
Related plan	PTP-2024-011

Deviations from plan: The stress test peak was capped at 1 200 RPS (not 1 500 RPS) because the test DB saturated at 1 200 RPS, masking any API-layer breaking point. Documented as a risk in PTP-2024-011.

---

3. NFR Results

NFR	Target	Observed	Pass / Fail	Notes
p95 — checkout initiation	< 800 ms	612 ms	Pass	Measured during 15-min load test at 500 RPS
p95 — order status poll	< 300 ms	187 ms	Pass	Measured during 15-min load test at 500 RPS
Throughput	≥ 500 RPS sustained	503 RPS	Pass	Stable over full 15-min window
Error rate	< 0.1%	0.4% during soak	Fail	HTTP 503 from DB connection pool; load and stress runs passed at 0.02%

---

4. Detailed Results

4.1 Load Test Results

[Chart placeholder — p95 time series stayed below 700 ms throughout; RPS held at 502–505]

Metric	Value
Duration	15 min
Peak RPS achieved	505
p50 response time	310 ms
p95 response time	612 ms
p99 response time	730 ms
Error rate	0.02%
Peak concurrent VUs	420

Load test passed all NFRs comfortably. No anomalies observed during the run.

4.2 Stress Test Results

[Chart placeholder — p95 increased linearly to 1 200 RPS, then spiked sharply]

Metric	Value
Breaking point (RPS)	1 200 RPS (DB saturation)
Failure mode	HTTP 503 — RDS connection pool exhausted (max_connections hit)
First sign of degradation	~900 RPS — p95 climbed above 1 000 ms

The API layer itself held; Datadog showed API container CPU at 65% at 1 200 RPS with headroom remaining. The DB instance was the limiting factor. Full API-layer stress test should be re-run after the connection pool fix with a larger DB instance.

4.3 Spike Test Results

Metric	Value
Spike load	1 000 RPS for 60 s
Error rate during spike	0.05%
Time to p95 recovery after spike	38 s

Spike test passed. ECS autoscaling added one new task within 45 s, which contributed to the 38 s recovery time. Recovery is within the 2-minute acceptance criterion.

4.4 Soak Test Results

Metric	Value
Duration	3 h
p95 at start of run	615 ms
p95 at end of run	643 ms
Drift (increase)	28 ms (4.5%)
Memory usage trend	Stable — no leak detected

The 28 ms p95 drift is within acceptable tolerance. However, at the 2 h 10 min mark, error rates climbed from 0.02% to 0.4% and remained elevated for the remainder of the run. Datadog confirmed RDS DatabaseConnections metric saturated at the default pool ceiling (max 100). The async payment orchestration layer introduced in v2.4 holds connections open longer than the v2.3 synchronous path, causing pool exhaustion under sustained load.

---

5. Bottlenecks Observed

Component	Symptom	Suspected cause	Evidence	Severity
RDS PostgreSQL connection pool	Error rate rose to 0.4% after 2 h 10 min of soak; HTTP 503 returned to clients	v2.4 async orchestration layer holds DB connections open longer than v2.3 synchronous path, exhausting the 100-connection pool ceiling	Datadog: DatabaseConnections = 100 / 100 from 02:10 onwards; CheckoutAPI logs: "connection pool timeout"	High

---

6. Comparison to Baseline

Metric	Baseline (PTR-2024-007, v2.3)	This run (v2.4)	Change	Regression?
p95 response time — load test	589 ms	612 ms	+23 ms	No — within 5% variance
Peak RPS sustained	501 RPS	503 RPS	+2 RPS	No
Error rate — load test	0.01%	0.02%	+0.01 pp	No
Error rate — soak test	0.03%	0.4%	+0.37 pp	Yes — new failure mode introduced by v2.4

---

7. Recommendations

Fix now (blocking release)

• Increase RDS connection pool max_connections and tune the async orchestration layer's connection lifecycle to release connections promptly after each DB operation. Estimated effort: 1 day (Dev Lead: Priya Kim). Soak test re-run required before sign-off.

Fix next sprint

• Upgrade the test environment DB instance to db.r6g.xlarge to allow full API-layer stress testing without DB masking the results.

Fix eventually (backlog)

• Introduce a connection pool monitoring alert (PagerDuty) that fires when DatabaseConnections exceeds 80% of the ceiling, so future pool saturation is caught before it causes customer-facing errors.

Accept (known limitation)

• The 38 s spike recovery time (vs. a notional 30 s target) is acceptable given autoscale adds a task within 45 s by design. No action required.

---

8. Appendix

• Raw k6 output: s3://acmecorp-perf-results/checkout-v24/2024-04-15—18/
• Grafana dashboard snapshot: grafana.acmecorp.internal/d/checkout-perf-v24
• Test scripts: github.com/acmecorp/perf-tests/tree/checkout-v2.4
• Tool versions: k6 v0.50.0, Grafana 10.3.1, InfluxDB 2.7