Rate Limiting (Deep Workflow)#

Rate limits balance fairness, availability, and abuse prevention. Design explicitly: who is throttled, what resource is limited, and how clients should back off.

When to Offer This Workflow#

Trigger conditions:

• Protecting public APIs, auth endpoints, or expensive operations
• Multi-tenant “noisy neighbor” isolation
• Retry storms after incidents causing cascading 429/502

Initial offer:

Use six stages: (1) threat & fairness model, (2) dimensions & keys, (3) algorithms & config, (4) distributed enforcement, (5) client protocol & UX, (6) observability & tuning). Confirm enforcement layer (API gateway vs app middleware vs edge).

---

Stage 1: Threat & Fairness Model#

Goal: Distinguish legitimate bursts (batch jobs, mobile retries) from abuse; align limits with product tiers and SLAs.

Exit condition: Written policy: free vs paid limits, partner caps, burst allowances.

---

Stage 2: Dimensions & Keys#

Goal: Choose stable limit keys: authenticated user id > API key > IP (with shared-NAT caveats).

Practices#

• Per-tenant and global limits; separate expensive routes (exports, search)

---

Stage 3: Algorithms & Config#

Goal: Token bucket / leaky bucket for smooth bursts; sliding window for strict per-minute caps; consider concurrency limits separately from request rate.

---

Stage 4: Distributed Enforcement#

Goal: Central store (Redis, etc.) with atomic increments; handle multi-region (sticky routing vs shared counters); mind clock skew.

---

Stage 5: Client Protocol & UX#

Goal: Consistent 429 responses with Retry-After; document exponential backoff + jitter; optional X-RateLimit-* headers for transparency.

---

Stage 6: Observability & Tuning#

Goal: Metrics on throttles by route and actor class; alerts on abnormal deny spikes (attack vs misconfigured client).

---

Final Review Checklist#

• Policy matches tiers and fairness goals
• Limit keys stable and hard to spoof
• Algorithm matches burst vs sustained semantics
• Distributed correctness considered
• Client-facing 429 behavior documented
• Metrics and tuning loop defined

Tips for Effective Guidance#

• Coordinate with authentication—anonymous IP limits are coarse.
• Don’t throttle health checks in ways that break monitors.
• GraphQL: consider query cost / depth limits, not only HTTP count.
• WebSockets: separate connection caps from message rate limits.

Handling Deviations#

• Edge/CDN: limits may differ from origin—document both layers.