Your Webhooks Work as Designed, and Your Customers Still Think They're Broken
A customer's integration has been live for three months. It passed every test your sandbox could throw at it. Then their finance team notices that a few dozen orders in their system never moved to "paid," even though the payments went through on your side. Their engineers dig in and find the cause: your webhooks were delivered, their handler timed out on a slow database write, your platform retried, and a second delivery arrived after a later event had already updated the record. Nothing crashed. Data just quietly stopped matching.
The ticket that lands in your support queue says "your webhooks are unreliable." Your engineers look at the delivery logs and see that everything worked exactly as designed. Both statements are true, and the gap between them is a documentation problem. Your webhook behaved correctly. Your docs never told the customer what correct behavior looks like.
Webhooks are a contract, and most docs only describe half of it
When you document a normal API endpoint, the developer is in control. They send a request, they get a response, and they can retry on their own terms. Webhooks reverse that relationship. Your system calls their code, on your schedule, under your rules for timeouts, retries, and ordering. The developer is writing a server for a client they can't see.
That makes the delivery rules part of the contract. Yet most webhook documentation I read covers only the payload: a list of event types and a sample JSON body for each one. That's the half developers can discover by logging a request. The other half, the part that only shows up under load or during an outage, is how delivery actually behaves.
The six questions your webhook docs must answer
Before a developer can build a handler that survives production, they need clear answers to these questions. If your docs don't answer them, your customers are guessing, and some of them are guessing wrong.
-
Can the same event arrive more than once? For almost every webhook system the answer is yes, because retrying after an ambiguous failure is the only safe choice for the sender. Say so plainly, and tell developers which field to use as an idempotency key. Stripe's documentation does this well: it states that endpoints "might occasionally receive the same event more than once" and tells developers to log the event IDs they've processed.
-
Do events arrive in order? If you don't guarantee ordering, say so, and give an example of events that can arrive out of sequence. Then tell developers what to do, such as fetching the object's current state from the API instead of trusting the delivery order.
-
What counts as a failed delivery, and what happens next? Which status codes count as success? How long do you wait before timing out? Do you follow redirects? How many times do you retry, over what period, and with what backoff? A developer can't size a queue or plan for a maintenance window without this.
-
How do I verify that a request came from you? Document the exact string that gets signed, the algorithm, the header format, and the timestamp tolerance. Warn developers that many web frameworks parse and re-serialize the body before handler code sees it, which breaks signature checks. This one warning prevents a whole category of support tickets.
-
What happens if my endpoint is down for a day? Do you turn off endpoints after repeated failures? Do you notify anyone? Can the customer replay missed events, or list recent events through the API to reconcile? This is the question their operations team will ask, usually during an incident.
-
How do payloads change over time? If your API is versioned, which version shapes the event payload? Can new fields appear without notice? Tell developers to ignore fields they don't recognize, and tell them how you'll announce changes that aren't additive.
Show the handler, not just the payload
A sample payload tells a developer what they'll receive. A sample handler tells them how to receive it safely, and that's where they'll make the mistakes that turn into tickets. Your webhook guide should include a short, realistic handler that demonstrates the delivery rules instead of just describing them:
@app.post("/webhooks")
def receive_webhook(request):
# verify the raw bytes, not parsed JSON
raw_body = request.get_data()
if not verify_signature(raw_body, request.headers, SECRET):
return "", 400
event = json.loads(raw_body)
# duplicates are normal, not errors
if already_processed(event["id"]):
return "", 200
# do the slow work after responding
queue.enqueue(process_event, event)
mark_processed(event["id"])
return "", 200
Every line in that handler answers one of the six questions. A developer who copies it starts with the right instincts: verify the raw body, expect duplicates, and respond fast. This is one place where documentation and engineering overlap completely. You can't write this example well unless you understand why each line is there, and the developer can't use it well unless the surrounding prose explains it.
Don't invent rules you can borrow
If you're designing or revising a webhook system, look at Standard Webhooks, an open specification guided by a technical steering committee with members from companies including Zapier, Twilio, Svix, Supabase, and Kong. It defines three headers (webhook-id, webhook-timestamp, and webhook-signature), signs the message ID, timestamp, and body together with HMAC-SHA256, and recommends using webhook-id as an idempotency key. It also gives guidance on retry schedules, how senders should treat redirects, and a recommended request timeout of 15 to 30 seconds.
Adopting a published standard doesn't replace your documentation, but it makes the documentation shorter and easier to trust. Developers who have integrated with another system that follows the standard already know how verification works, and you can point to the spec instead of re-explaining it.
Audit your webhook docs in an afternoon
You don't need a rewrite to find the gaps. Try this:
-
Read your webhook docs against the six questions above. Mark each one as answered, partly answered, or missing.
-
Register a test endpoint that returns a 500 error. Watch what your platform actually does, then compare it with what your docs say. Any difference is a bug in one or the other.
-
Search your support tickets for "duplicate," "missing," "signature," and "out of order." Each cluster points at a question your docs leave open.
-
Ask your support engineers what they explain most often. Whatever they repeat in tickets belongs on the page.
Webhooks are one of the cross-cutting topics that an endpoint-by-endpoint reference has no natural place for, which I wrote about in Your API Reference Is Not Your API Documentation. They deserve their own guide, written by someone who understands both the delivery mechanics and the developer on the other end.
If your support queue keeps filling with webhook tickets that turn out to be "working as designed," I can help. I'll test your webhook delivery the way a customer's integration would, compare it against your docs, and write the guide that tells developers what correct behavior looks like before production does. Get in touch and tell me where things stand.