Your API Documentation Is Drifting, and Your Support Queue Knows It

A developer copies the example from your docs, pastes it into their terminal, and gets a 400 Bad Request. They check their API key. They check their headers. They read the page again. Twenty minutes later, they open a support ticket, and your engineer spends another twenty minutes discovering that the customer_id field was renamed to account_id two releases ago. The docs never heard about it.

Nobody wrote a bad page. The page was correct on the day it shipped. Then the API kept moving, and the documentation stood still.

This is documentation drift, and it is one of the most expensive problems in developer-facing products, precisely because it is so quiet. No alarm goes off when a docs page stops being true. The only signal is a slow rise in integration tickets, a few more abandoned trials, and partners who start asking your solutions engineers instead of reading the docs, because they have learned the docs can't be trusted.

Drift is a process failure, not a writing failure

In an earlier post, I argued that your API reference is not your API documentation, and that you need conceptual guides, tutorials, and recipes on top of the reference. That's still true. But every one of those layers has the same weakness: it was written against a version of the API that no longer exists.

Teams usually respond to drift by blaming the writing, then commissioning a rewrite. Six months later, the rewrite has drifted too. The problem was never the prose. It was that nothing in the release process connected a code change to the pages it invalidated.

As a software engineer, I spent years on the other side of this. A change that broke a unit test couldn't merge. A change that broke the docs merged every day, because the docs weren't part of anything that could fail.

Where drift hides

Drift doesn't spread evenly. In my experience, it concentrates in a few predictable places:

• Code samples. These are the first thing developers copy and the last thing anyone retests. A sample that worked against v1 of your SDK may not even compile against v3.
• Hand-written example payloads. The OpenAPI Specification says an example SHOULD match its schema. "Should" is doing a lot of work there. Nothing in the spec checks your examples against what your server actually returns.
• Error documentation. New error codes get added in code and never make it to the errors page. Developers hit a code you've never documented, and they have nowhere to go.
• Defaults and limits. Rate limits, page sizes, timeout values, and retention windows change through configuration, not code review, so nobody thinks of them as docs changes.
• Narrative guides. Tutorials and conceptual pages are the hardest to test and the easiest to forget. They are also the pages new developers read first.

Make the docs part of the build

The cure for drift is not more careful writers. It's making stale documentation fail loudly, the same way broken code does. You can start on most of this without hiring anyone.

1. Generate the reference; don't hand-maintain it. If your OpenAPI file is written by hand alongside the code, it will drift. Generate it from the code, or generate the code from it, but pick one source of truth and make the other derived.

2. Test the spec against the running API. Contract-testing tools close the gap between what the spec promises and what the server does. Schemathesis, for example, generates tests from an OpenAPI or GraphQL schema, runs them against your API, and flags responses that violate the schema. It runs from the command line, pytest, or GitHub Actions, so it fits into the CI pipeline you already have.

3. Execute your code samples. Every sample on your docs site should live in a file that your CI system actually runs against a sandbox environment. Some ecosystems build this in: Rust's rustdoc extracts code examples from documentation and runs them as tests. For HTTP APIs, the pattern is simple enough to build yourself: keep samples in a samples/ directory, include them into the docs at build time, and run them in CI.

docs/
  guides/create-subscription.md   # includes ../samples/create_subscription.py
samples/
  create_subscription.py          # executed in CI against the sandbox

If the sample breaks, the build breaks, and someone fixes it before a customer finds it.

4. Put docs in the definition of done. Add a line to your pull request template: "Which docs pages does this change affect?" An honest "none" is a fine answer. What matters is that someone had to ask the question before merging, not after the tickets arrived.

5. Use the deprecation signals you already have. OpenAPI lets you mark an operation or a parameter as deprecated. Use it, and pair it with a changelog entry that tells developers what to use instead and when the old behavior goes away. A deprecation that exists only in a Slack thread is not a deprecation; it's a surprise scheduled for later.

Audit what you already have

Automation stops new drift. It doesn't fix the drift you've accumulated. For that, you need an audit, and the most efficient way to run one is to follow the evidence:

• Pull the last quarter of integration-related support tickets and tag every one where the docs were wrong, missing, or out of date. The pages that show up most often are your starting list.
• Walk through your getting-started guide from a clean machine with a fresh account, exactly as written. Write down every step where you had to know something the page didn't tell you.
• Compare your published error reference against the error codes your API actually emits. The difference is a list of pages to write.

None of this is glamorous, and that is exactly why it doesn't get done. Engineers are busy shipping features, and the docs are always "next sprint."

Why this takes an engineer's eye

Fixing drift requires reading the code, not just the docs. Someone has to know whether a field rename was intentional, whether a changed default is a bug or a decision, and whether a sample fails because the docs are wrong or because the API is. That requires someone who is comfortable in your repository, your CI configuration, and your OpenAPI file, and who can then turn what they find into clear, trustworthy prose.

That combination is what I bring to API documentation work: decades of hands-on software development, and the writing discipline to make the result something developers actually trust.

If your integration tickets keep climbing after every release, or your partners have stopped trusting your docs, I'd be glad to take a look. You can reach me through the contact page.