Rate Limiting a Fragile Backend

Rate Limiting a Fragile Backend

Your orders API is fragile: past ~15 concurrent requests it stops queueing and starts failing, every request return 500 status while the overload lasts, and the team that owns it can't fix that soon. Most of your traffic is a well-behaved client (~3 req/s), but every 15 seconds an integration partner slams the API with a burst of 60 requests, driving it past the ceiling and taking it down for everyone, the legit client included. You can't fix the backend. You own the proxy in front of it.

System Architecture

Your task

Add rate limiting in nginx so the burst client is throttled at the proxy, the backend's concurrency ceiling is never breached, and the legit client sails through untouched, every burst.

You edit only nginx.conf:

• Every request carries an X-Client-Id header ($http_x_client_id in config). All traffic shares one source IP, so think about what your limit is keyed on.
• Commit + push → the platform rebuilds the image and restarts nginx.

Success Criteria

We watch about 1 minute of live traffic. You pass if all three are true:

• The legit client gets no errors: no 500s, and your limit never blocks it (no 429 or 503).
• The backend never gets overloaded: the orders API returns zero 500s, to anyone.
• Legit orders still get through: at least 2 per second. Blocking everyone doesn't count.

Rejecting the burst client (429/503) is the design working, it's expected and not penalized.

Where to look

• Logs: the orders API logs OVERLOADED every time too many requests reach it at once. Once your limit works, these lines stop.
• Metrics: request and error rates per client. In the broken state the backend's 500s climb with every burst; once you tame it they fall to zero.

Submit once a few bursts have passed with no new OVERLOADED lines and no 500s