Your Customers Find Your Rate Limits the Day Their Traffic Spikes

A customer's integration has run quietly for months. Then they onboard their largest account, start a backfill job, and within minutes their logs fill with 429 Too Many Requests. Their code retries immediately, from every worker at once, and the retries get rejected too. By the time their on-call engineer works out what's happening, a nightly sync has failed, their own customers are asking questions, and a ticket lands in your queue marked urgent: "Your API is down."

Your API isn't down. It's doing exactly what it was configured to do. The customer just had no way to know what that was, because your rate limit documentation is one sentence on an FAQ page: "Requests are rate limited to ensure fair usage."

Rate limits are part of your API's contract. If the only way customers learn them is by exceeding them in production, you've handed them an incident instead of a design constraint.

Why rate limits stay undocumented

Having spent years on the engineering side, I understand how this happens. The reasons are all reasonable on their own:

• The limits live in infrastructure, not in the API code. They're configured in a gateway, a load balancer, or a middleware layer that whoever writes the docs never looks at.
• Nobody wants to commit to a number. If you publish a limit, customers may treat it as a guarantee, and you lose the freedom to tune it. So the docs say "fair usage" and stop there.
• The real limits are more than one number. There's a per-key limit, a stricter limit on expensive endpoints, a concurrency cap, and different tiers by plan. Writing that down honestly takes work, and it exposes inconsistencies nobody planned.

The standard doesn't fill the gap for you. RFC 6585, which defines the 429 status code, says the response "SHOULD include details explaining the condition" and may include a Retry-After header saying how long to wait. It also states that it does not define how the server identifies the user or how it counts requests. That's left to you, which means it's also left to your documentation.

The six questions your rate limit docs must answer

A developer planning an integration, or an architect sizing a batch job, needs these answers before writing any code:

1. What is counted, and against what? Per API key, per user, per IP address, per organization? Do all endpoints share one bucket, or do some have their own? Do failed requests count?
2. What are the actual limits? Publish the numbers, by plan if they vary. If you need flexibility, say that limits can change and how you'll announce changes. A number with a caveat is far more useful than no number.
3. What is the window, and how does it reset? A fixed hourly window behaves very differently under bursty traffic than a sliding window or a token bucket does.
4. How can I see where I stand? Which response headers report the limit, the remaining quota, and the reset time, and in what units?
5. What happens when I go over? The status code, the response body, whether Retry-After is sent, and what the client should do next. Also say what happens to a client that keeps sending requests anyway.
6. How do I get more? A plan upgrade, a request form, or a named contact. Customers who are growing fast are your best customers, and this is often the question they care about most.

GitHub's REST API documentation is a good model. It states the primary limit (5,000 requests per hour for authenticated users), lists each rate limit header and what it means, and describes secondary limits, such as a cap of 100 concurrent requests, that can trigger even when a client is under its hourly limit. It tells developers how long to wait in each case, and it warns plainly that continuing to make requests while rate limited "may result in the banning of your integration." A developer who reads that page can design for the limits instead of discovering them.

Document the retry, not just the limit

The most expensive part of a rate limit incident usually isn't the first 429. It's the retries. When many clients hit a limit together and retry on the same schedule, they come back together and get rejected together. Marc Brooker's widely cited AWS Architecture Blog post, "Exponential Backoff And Jitter," showed that adding randomness to backoff substantially reduces both wasted client work and server load, and concluded that jittered backoff "should be considered a standard approach for remote clients."

Your customers won't know that unless your docs tell them. Give them a short, correct example:

import random, time

def call_with_backoff(send_request, max_attempts=5):
    for attempt in range(max_attempts):
        response = send_request()
        if response.status_code != 429:
            return response
        retry_after = response.headers.get("Retry-After")
        if retry_after and retry_after.isdigit():
            delay = int(retry_after)          # the server told us; obey it
        else:
            delay = random.uniform(0, min(60, 2 ** attempt))  # back off with jitter
        time.sleep(delay)
    raise RuntimeError("Still rate limited after retries")

Retry-After can be either a number of seconds or an HTTP date, so a production version should handle both. If you ship SDKs, build this behavior into them so most customers get it right by default, and document that the SDK does it so nobody wraps it in a second retry loop.

Know where the headers are heading

Rate limit headers today are mostly vendor-specific; GitHub's x-ratelimit-* headers are one convention among several. The IETF HTTP APIs working group is drafting a standard, RateLimit header fields for HTTP, which defines a RateLimit-Policy field describing the quota policy and a RateLimit field reporting the quota that remains. It is still an Internet-Draft, not a published RFC, so don't tell customers you follow "the standard" yet. If you're designing or revising your headers now, though, read the draft before inventing new names.

Audit your rate limit docs this week

You can find most of the gaps without a rewrite:

• Find the real limits. Pull the actual configuration from your gateway or middleware and compare it with your docs. Any difference is a bug in one or the other.
• Search your support tickets for "429," "rate limit," "throttle," and "too many requests." Read a dozen, and note which of the six questions each customer was trying to answer.
• Exceed your own limit with a test key. Look at the status, headers, and body you get back. Could a developer who has never seen your code tell what happened and what to do?
• Check your SDKs. Do they retry on 429, honor Retry-After, and add jitter? Does the documentation say so?
• Give rate limits their own page. They apply to the whole API, and as I argued in Your API Reference Is Not Your API Documentation, cross-cutting concerns need their own guide rather than a line repeated on every endpoint.

Why this takes an engineer

A good rate limit guide means reading gateway configuration, understanding how a token bucket behaves under bursty load, testing what the API actually returns, and then explaining all of it to a developer who just wants their sync job to finish. That's engineering work and writing work in the same document. I spent decades as a software engineer before I became a technical writer, so tracing a limit from the config file to the response header to the page that explains it is familiar territory.

If your customers keep discovering your rate limits through production incidents, I can help. I'll test how your API behaves at its limits, compare that with what your docs say, and write the guide that lets developers plan for your limits before they hit them. You can reach me through the contact page.