The 3 AM Incident
03:12 AM. Your phone goes off:
text
[PAGE] payments-api: success rate 0%, ALL payments failing
Someone deployed PAY-1847: Make worker count configurable at 23:47 and went to bed. Now every payment is failing, and it's your problem.
This challenge starts broken. There's nothing to build, only a production incident to end. Find it, fix it.
System Architecture
Your task
Restore payments: find out why payments-api is down and fix it.
You edit the payments-api repo, code and config, already cloned in the editor:
- The Service forwards
payments-api:80 → 8000, and the readiness probe calls GET:8000/health. - Commit + push → the platform rebuilds the image and redeploys the payments API.
Success Criteria
When you submit, the Judge watches the next 60 seconds of live traffic. Both must pass:
- Payments are flowing: a steady rate of successful POST
/payresponses. - No failed payments: no errors, refused connections or half-recovered flapping.
Where to look
- Logs:
payments-apiis your crime scene. What does the service say when it starts, or tries to? - Metrics: is the pod even running? A restart loop has a very particular heartbeat.
Submit once customer payments have been green for a full minute. Submissions are free, so your best score counts.
- Read the traceback like it's pointing at something. It is.
- "It starts fine now" and "it's serving traffic" are different claims. Verify the second one.
- The platform contract is written down in
README.mdin the repo. Compare it with what the service says it's doing.