🖥️

Real incidents need a real screen.

Open senioreng.dev on your laptop for the full experience.

Live·00:00elapsed

Incident Workspace

Service Metrics

Environment

Production

Error Rate %

9%

avg · 1h

33%25%17%8%0%
1%
1%
1%
1%
1%
1%
1%
1%
1%
1%
1%
1%
1%
1%
1%
1%
1%
1%
33%
33%
33%
33%
33%
33%
-60m-45m-15mNow

P95 Latency ms

62ms

avg · 1h

200ms150ms100ms50ms0
62ms
63ms
61ms
62ms
61ms
63ms
62ms
61ms
63ms
62ms
61ms
62ms
63ms
61ms
62ms
63ms
62ms
61ms
62ms
63ms
62ms
61ms
63ms
62ms
-60m-45m-15mNow

Request Volume k req/min

14k

avg · 1h

16k12k8k4k0
14k req/min
13k req/min
14k req/min
15k req/min
13k req/min
14k req/min
14k req/min
15k req/min
13k req/min
14k req/min
15k req/min
13k req/min
14k req/min
13k req/min
14k req/min
15k req/min
14k req/min
13k req/min
14k req/min
15k req/min
14k req/min
13k req/min
14k req/min
14k req/min
-60m-45m-15mNow

Stripe API Success Rate %

99%

avg · 1h

100%75%50%25%0%
99%
100%
99%
100%
99%
100%
99%
100%
100%
99%
99%
100%
99%
100%
99%
100%
100%
99%
99%
100%
99%
100%
99%
99%
-60m-45m-15mNow

payment-service

3 pods — traffic round-robined evenly across all instances

DEGRADED

Payments/min

420

Failing/min

140

Revenue at risk/hr

$180k

Production Incident

Payment Failures. 33% of Requests

On-Call Alert

1 in 3 payment requests are failing immediately after today's v4.2.0 deployment.

Nova Payments processes $180k/hour across 4,500 B2B customers. Since the v4.2.0 deployment 20 minutes ago, exactly 33.3% of payment requests are failing. Stripe's status page is all green. The error rate hasn't moved since it started.

You are on-call. Investigate the available telemetry, identify the root cause, and restore full payment processing.