Your Integration Passed in the Sandbox and Broke on Launch Day

A customer's engineering team spends six weeks building against your API. Every call works in the sandbox. Their QA team signs off, their launch date goes on the calendar, and on the morning they switch to production keys, the integration falls over. A field the sandbox never validated is required in live mode. The webhook endpoint they tested was never registered for production. A payment that always succeeded in testing now goes to manual review, and nothing in their code expects that state. By lunch, a ticket arrives with the five words every API company eventually learns to dread: "It worked in the sandbox."

Nobody on that customer's team did anything wrong. They tested exactly what you gave them. The problem is that your sandbox and your production environment behave differently, and the only record of those differences is in your engineers' heads.

That gap has a real cost. The customer's launch slips, your solutions engineers spend a week on calls, and the revenue that depended on the integration going live starts later than anyone planned. For a platform with partners building on it, every partner pays that cost separately.

The sandbox is a second product

Most API documentation treats the sandbox in a single paragraph: use these test keys, point at this base URL, and you're ready to build. That framing assumes the sandbox is production with fake data. It almost never is.

Having built systems on the engineering side for decades, I know how the differences accumulate, and none of them are mistakes:

• Downstream services are stubbed. The sandbox can't call the real bank, carrier, or fraud service, so it calls a simulator that always answers quickly and usually answers yes.
• Validation is looser. Teams relax checks in test environments so developers can get started, and production keeps the strict rules.
• Some workflows exist only in production. Account verification, compliance review, manual approval, and production-only features have nothing to simulate them, so the sandbox skips them.
• Limits and data differ. Rate limits, timeouts, retention, and data volumes are set separately for each environment, often by different people.

Each of these is a reasonable engineering decision. Together they mean your sandbox is a second product with its own behavior, and it needs its own documentation.

Good vendors document what the sandbox won't show you

The strongest API providers are blunt about this. Stripe's testing documentation says sandbox transactions "don't move funds," and it publishes specific test card numbers that trigger specific outcomes: 4000000000000002 returns a generic decline and 4000000000009995 returns an insufficient-funds decline. It also warns that "the rate limiter is stricter in test environments than in live mode" and tells developers not to load test there.

Twilio's test credentials documentation is just as direct about the limits of its test mode. Test credentials work only for a few resources, such as sending messages, making calls, and buying phone numbers; requests to anything else return 403 Forbidden. Calls made with test credentials don't execute TwiML, and messages and calls "don't trigger status callbacks." Twilio also publishes "magic" phone numbers that reliably produce specific errors, such as a destination that can't receive SMS.

Notice what these pages do. They don't just explain how to use the sandbox. They tell developers where the sandbox stops telling the truth, so the developer knows what still has to be verified in production.

What your sandbox documentation must cover

A developer planning an integration needs answers to these questions before they write tests:

1. What is simulated, and what is real? Name every downstream dependency the sandbox stubs out, and say what the stub returns by default.
2. What behaves differently? Validation rules, rate limits, timeouts, data retention, and whether sandbox data is ever reset. If you don't know, that's the first thing to find out.
3. How do I trigger failures on purpose? Publish the test values that produce declines, timeouts, pending states, and each documented error. A sandbox that only succeeds teaches developers to write code that only handles success.
4. What exists only in production? Review queues, verification steps, approval delays, and features that can't be tested at all. For each one, say what state the API returns and how long it can last.
5. How do events behave? Whether webhooks fire in the sandbox, whether they fire at realistic times, and whether endpoints must be registered separately for each environment.
6. What changes at go-live? Keys, base URLs, webhook secrets, OAuth redirect URIs, and any account settings that must be configured again.

Item 3 deserves special attention. If your error messages are well written, as I argued in Your API's Error Messages Are the Documentation Developers Actually Read, developers should be able to see every one of them in the sandbox before a real customer does.

Give customers a go-live checklist

The single most useful page you can add is a go-live checklist: a short, ordered list a developer works through on the day they switch environments. It turns the six answers above into actions, for example:

• Replace test keys with live keys, and confirm the live keys have the same scopes.
• Register production webhook endpoints and store the production signing secret.
• Run one real, low-value transaction end to end and confirm every expected event arrives.
• Confirm the code handles the production-only states, such as a pending review, that the sandbox never returned.
• Check the account settings that don't carry over from the sandbox.

Your solutions engineers almost certainly walk customers through a version of this list on launch calls already. Writing it down lets every customer use it, including the ones who never get a call.

Find your gaps this week

You can do most of this audit without hiring anyone:

• Search your support tickets for "sandbox," "test mode," "worked in testing," and "live keys." Each ticket points at a difference nobody documented.
• Ask your solutions engineers what they warn customers about before launch. Their warnings are your missing documentation.
• Compare the configurations. Put the sandbox and production settings for validation, limits, and timeouts side by side. Every difference either belongs in the docs or should be removed.
• Run your own quickstart in both environments. Note everything that behaves differently, including the things you had to fix by hand.

Why this takes an engineer

Documenting a sandbox well means reading environment configuration, tracing which services are stubbed, deliberately breaking things to see what the API returns, and then explaining all of it to a developer who just wants to launch on schedule. It's investigative engineering work as much as writing, and it's the kind of work I've done from both sides of the API.

If your customers keep finding the differences between your sandbox and production on launch day, I can help. I'll test both environments, document where they differ, and write the go-live guide your customers need before their launch, not after it. You can reach me through the contact page.