API Rate Limits Basics
API rate limits cap how many requests a client can send to a service within a time window. Many services publish limits as requests per minute (RPM) and also add a burst limit that restricts short spikes. RPM controls sustained traffic, while burst limits protect the service from sudden load spikes that can happen during retries, batch jobs, or UI refresh storms.
In practice, a “429 Too Many Requests” response often appears when either the sustained RPM window is exceeded or the burst allowance is consumed. Some APIs also return headers that describe the remaining quota and the time until reset. For example, you might see headers like Retry-After or X-RateLimit-Remaining, though exact names vary by provider. On one project I reviewed in late 2024, the provider’s docs showed RPM but the burst behavior lived in a separate “burst” section, and the first load test failed because the client sent 200 requests in a second.
Rate limiting is not only about protecting servers; it also shapes client design. If your client sends requests in tight loops without pacing, you can hit burst limits even when the average RPM looks safe. If your client retries aggressively on timeouts, retries can multiply traffic and push you into throttling during partial outages.
Common Pain Points
Teams often misread rate limits as a single number that applies uniformly to every request pattern. RPM limits apply to a time window, but the windowing method (fixed window, sliding window, token bucket) changes how spikes behave. Burst limits exist because average RPM does not prevent short spikes from overwhelming downstream systems.
Another frequent issue is retry logic that ignores throttling signals. Many clients retry on network errors and 5xx responses, but they treat 429 as just another transient failure. That approach can worsen load and extend the outage window, especially when multiple workers retry at the same time. A mild frustration I’ve seen repeatedly: the code uses exponential backoff for 5xx but uses a fixed 100 ms delay for everything else, which turns a small hiccup into a burst of retries.
Dependencies also matter. If your client calls multiple endpoints, each endpoint may have its own limits, or the provider may enforce a shared limit across the whole API. Some services also rate limit by API key, by user, or by IP address. When you test only one endpoint, you can miss cross-endpoint coupling that appears in production traffic.
Supporting technologies influence how quickly requests arrive. HTTP keep-alive, connection pooling, and concurrency settings can change the effective request rate. If a load balancer retries upstream requests, the provider may see more traffic than your application counts. Even logging can contribute: synchronous logging to a remote sink can slow down request handling and change concurrency patterns, which then changes how often you hit burst limits.
Designing Around Limits
Read Headers And Reset Times
Start by capturing the exact response headers your API returns on both success and throttling. Look for fields that indicate remaining quota and the reset time, and honor Retry-After when present. If the API returns a “limit” header but not a “reset” header, you can still infer reset timing by measuring when 429 responses stop during a controlled test, though that inference is less precise.
When you build a client, log the limit context per request: endpoint, method, status code, and any rate-limit headers. In one integration I audited, the team used version 2.3.1 of their HTTP wrapper and lost headers due to a custom middleware that stripped “unknown” headers; the client then retried blindly and triggered burst limits.
Use Backoff With Jitter
For 429 responses, use backoff that respects the provider’s guidance. If Retry-After is present, wait at least that long before retrying. If it is absent, use exponential backoff with jitter to spread retries across clients. Jitter matters because synchronized retries from multiple workers can recreate the spike that caused throttling.
Set a maximum retry count and a maximum total retry time. A common outcome target is to reduce repeated 429s to near zero during steady load tests, while keeping end-to-end latency within an acceptable range for your use case. If your API calls are part of a user-facing workflow, cap retries more aggressively than for background jobs.
Throttle Client-Side With Concurrency Caps
Client-side pacing prevents burst-limit failures. Use a rate limiter in your application that meters requests before they leave the process. Token bucket algorithms map well to burst limits because they allow short bursts up to a capacity while enforcing a refill rate that matches RPM.
Also cap concurrency. Even with a metered rate, high concurrency can cause request bursts at the network layer when threads unblock together. In Node.js, for example, a library like Bottleneck can enforce both rate and concurrency; in Python, you can use a token-bucket limiter or a leaky-bucket approach. If you use a load test tool such as k6, verify the actual request arrival rate at the client, not just the configured scenario rate.
Test Throttling In Staging
Run a staging test that intentionally pushes near the limit, then observe the transition from success to 429. Measure how many requests are allowed in a burst and how quickly the allowance resets. If the provider uses a sliding window, the reset behavior differs from a fixed window, and your pacing strategy should match what you observe.
Keep the test bounded. Use a dedicated test API key and a short duration. If you share an API key across environments, you can affect other teams and trigger throttling for unrelated workloads.
Educational Case Examples
Example 1: Batch Job Retries
A data team runs a nightly batch that fetches records from a health-adjacent API. The job processes 50 items concurrently and retries on timeouts with exponential backoff, but it does not treat 429 specially. During a partial upstream slowdown, timeouts increase, retries multiply traffic, and the provider returns 429 due to burst consumption. The fix changes the retry policy: 429 triggers backoff that honors Retry-After, and the batch reduces concurrency from 50 to 10 while adding client-side token-bucket pacing. After the change, the job completes with fewer throttling events and a more predictable runtime.
Example 2: UI Polling Storm
A web app polls an endpoint every second to refresh a dashboard. Users open multiple tabs, and the app also triggers a refresh after certain actions, creating short spikes. The provider enforces 60 RPM and a burst limit that allows only a small number of requests per second. The app hits 429 even though the average over a minute seems within 60. The fix adds a shared in-browser scheduler so multiple tabs coordinate, and it increases the polling interval with jitter when the server returns rate-limit headers. The team also records the number of 429 responses per session to confirm the improvement.
Rate Limit Checklist
| Decision Point | What To Look For | Typical Symptom | Practical Adjustment |
|---|---|---|---|
| RPM Limit | Requests per minute per key/user | 429 after sustained load | Meter requests to match RPM |
| Burst Limit | Short spike allowance per second or per window | 429 during spikes despite safe average | Add token bucket capacity and pacing |
| Reset Timing | Headers or observed reset behavior | Retries keep failing for too long | Honor Retry-After or measure reset |
| Retry Policy | 429-specific handling and max retries | 429 storms during partial outages | Backoff with jitter; cap retry time |
Step-by-step checklist
- Record the exact rate-limit documentation for the API key scope you use (per key, per user, per IP).
- Capture response headers on both success and 429 responses, including any reset or retry guidance.
- Set client-side pacing using a token bucket or equivalent limiter that matches RPM and burst behavior.
- Cap concurrency so threads do not release into the network at the same moment.
- Use 429-aware retries: honor Retry-After, add jitter, and stop after a bounded retry budget.
- Run a staging test that ramps traffic until you observe the first 429, then back off and measure recovery time.
Common Mistakes
One mistake is treating burst limits as “nice to have.” If your client sends requests in short batches, burst limits can fail even when your average RPM stays under the published number. Another mistake is ignoring concurrency. A limiter that meters per-process rate can still produce bursts if multiple workers start at the same time and share the same schedule.
Teams also mis-handle timeouts. If you retry timeouts without checking whether the upstream is already overloaded, you can amplify traffic. A safer pattern distinguishes between connection failures, read timeouts, and explicit 429 responses, then applies different retry delays.
Another practical error is failing to test with realistic request payload sizes. Some APIs throttle based on request cost rather than count, or they throttle differently for endpoints that trigger heavier processing. If the docs mention “requests” but also mention “operations” or “units,” you need to map your calls to the provider’s cost model.
Finally, avoid promotional “rate limit bypass” approaches. If you rotate keys to evade limits, you can violate provider terms and create operational risk. If you need higher throughput, the correct path is usually a documented plan change or a request for higher limits, not hidden workarounds.
FAQ
What Does Burst Limit Mean?
Burst limit caps short spikes of requests, often measured over a shorter interval than the RPM window. You can hit burst limits even when your average RPM stays within the published number.
Why Do I Get 429 With Low Average RPM?
Your client may send requests in tight bursts due to concurrency, retries, or synchronized polling. Burst limits and windowing behavior can trigger 429 even when the minute-level average looks safe.
Should I Retry After 429?
Retry after 429 with a delay that respects Retry-After when present. Use exponential backoff with jitter and cap retries so you do not create a retry storm.
How Can I Measure Rate Limit Reset Time?
In staging, ramp traffic to trigger 429, then observe when successful responses resume. If the API provides reset headers, use them; otherwise, measure recovery time with controlled tests.
Do Rate Limits Apply Per Endpoint?
Some APIs enforce limits per endpoint, while others enforce a shared limit across the whole API or across a group of endpoints. Check the provider’s documentation for the scope of the limit.
Author's Insight
Rate limits behave like a contract between your client and the service’s capacity, and the contract includes both sustained and spike behavior. RPM alone does not predict throttling outcomes when concurrency, retries, and windowing rules interact. The most reliable approach is to read the provider’s headers, build 429-aware retry logic, and test throttling in a controlled staging environment. When documentation is incomplete, measuring reset behavior and burst tolerance with bounded load tests gives the clearest operational signal.
Key Takeaways
- RPM controls sustained traffic; burst limits control short spikes.
- 429 responses often reflect burst consumption, retry amplification, or shared limits across endpoints.
- Honor Retry-After and use backoff with jitter for 429, with a bounded retry budget.
- Use client-side pacing and concurrency caps, then validate with staging tests that intentionally approach the limit.
