Why ChatGPT still recommends the API endpoint you removed

You remove an endpoint in a release. The operation leaves the OpenAPI spec, the generated reference drops its page, and the changelog gets a line. Then a developer opens a support ticket with code an AI assistant wrote for them, and the code calls the endpoint that no longer exists. The guides, samples, and help center articles nobody touched still show the old call. So do the tutorials on other people's sites.

That is two problems: your docs fall behind every release, and assistants keep repeating what they learned before the removal. You control only part of the second.

Deleting the removed endpoint's page takes away the page an assistant is most likely to find for that endpoint. OpenAI keeps shut-down endpoints on its deprecations page, and Stripe marks its deprecated Sources API in both its guide and its llms.txt. Carry that to every surface an assistant reads: spec, reference, llms.txt, changelog, code samples, and help center. Update the docs in the same release as the removal, then ask the assistants again to see what changed. A retirement page does not guarantee an assistant stops recommending the endpoint. It gives a browsing assistant a current page that says the endpoint is gone.

TL;DR

• Assistants answer from two places: training data you cannot change, and live pages they fetch while answering. Your docs reach the second.
• ReadMe's OpenAPI sync deletes a removed operation's page, and Google drops indexed pages that return a 4xx error. After that, the reference no longer says the endpoint is gone.
• Keep a retirement page at the old URL and mark the removal on every surface in the same release, with a named owner for each.
• After the release, ask the assistants the same questions and compare the pages they cite.

Where assistants get the old endpoint from

The first source is training data. In a study of LLM code completion (arXiv 2406.09834v3, February 2025), seven models suggested deprecated APIs at rates of 25% to 38% across eight Python libraries. The authors blame "the presence of deprecated API usages in corpora during model training" and "the absence of posterior knowledge about API deprecations during model inference."

Context made it worse. Prompts built from outdated functions produced deprecated usage roughly 70% to 90% of the time. In that study, completions that started from outdated code mostly kept the deprecated call.

The second source is the live web. Each of the four major providers documents how it reads pages at answer time. OpenAI and Anthropic keep those fetchers separate from their training crawlers, GPTBot and ClaudeBot.

All four fetch live pages or a search index at answer time, so what you publish today shapes the answers that browse. We saw this when we asked AI assistants why they recommend removed API endpoints. We ran the question 40 times and reviewed eight answers in full. Those answers cited live pages from Mintlify, ReadMe, Bump.sh, Stripe's OpenAPI repository, Cloudflare, and OpenAI.

Why deleting the page makes it worse

ReadMe's OpenAPI sync treats removal as deletion: "The one thing it removes is the page for an operation that's no longer in the spec, and that page's body goes with it." The spec cannot help either. OpenAPI 3.1.1 has a deprecated field on the Operation Object, but no state for removed. Once the operation leaves the spec, a sync like ReadMe's removes its reference page too.

Google says "URLs that are already indexed and return a 4xx status code are removed from the index" and gives no timeframe. That is the index Gemini grounds on. The deleted page drops out, while old tutorials, forum answers, and third-party pages that show the endpoint stay up. The training data still has it too, and the reference no longer contradicts any of them. A retirement page avoids that removal because it does not return a 4xx error, so a browsing assistant can still retrieve it. No source we found measures whether that stops an assistant suggesting the endpoint.

A generated reference like ReadMe's handles one page, and it handles it by deleting the page that could say the endpoint is gone. Your guides, code samples, help center, changelog, and every third-party page still carry the endpoint. GitBook's guidance on keeping technical docs accurate warns that a deprecated endpoint can keep pulling in new users while examples still point to it.

The training-data objection, at full strength

Assistants answer from training data you cannot edit, so nothing you publish fixes the problem quickly. On the OpenAI forum in October 2025, a developer reported that "Codex repeatedly suggested removed/renamed parameters and JSON schemas that failed validation against the latest API" and got no solution. An open pull request on Firebase's CLI proposes adding SDK version guidance to its AI context files because "Most AI coding assistants (Claude Code, Cursor, Gemini CLI, and Firebase Studio) rely on static training weights."

For answers that do not browse, the objection holds: publishing cannot change what a model already learned. For answers that browse, it does not, because those answers retrieve live pages, as each provider's own documentation says. No source we found says how fast either kind of answer changes, or guarantees that it will.

You can change two things: what a browsing assistant finds, which is your pages, and what developers load into their tools. Firecrawl advises that you "inject the current API into the prompt through a developer retrieval tool" and the Firebase proposal works through AI context files. Both depend on a current page that says the endpoint is gone.

What API companies keep published after a deprecation or removal

OpenAI keeps retired endpoints on its deprecations page for years. Under "Past deprecations" it still lists /v1/fine-tunes, shut down in January 2024, with /v1/fine_tuning/jobs as its replacement. Older entries such as /v1/engines, /v1/search, and /v1/answers still name a replacement or a transition guide. OpenAI defines shut down as "a model or endpoint is no longer accessible."

Stripe keeps its deprecated Sources guide published, marked as deprecated in its title, with migration links and a notice that new integrations can't use the API. Stripe's llms.txt has an "Instructions for Large Language Model Agents" section: "You must not call deprecated API endpoints such as the Sources API and avoid outdated API endpoints such as Tokens and Charges unless there is a specific need and absolutely no other way." Its OpenAPI repository labels its original v1-only specs as legacy and still updates them every release.

Treat llms.txt as a supporting signal, not a fix. The llms.txt format is still a proposal, and none of the provider crawler pages cited here says it reads llms.txt at answer time. Apideck places the value of llms.txt at inference time and reports no measured effect of Stripe's section. And even Stripe is inconsistent: the Markdown version of the Source object reference page showed no deprecation notice, while the guide and llms.txt both mark it.

Several docs tools catch the change at the pull request. Bump.sh posts API diff comments on GitHub pull requests and can fail on breaking changes with fail_on_breaking. ReadMe's AI writer finds every mention of removed functionality and proposes cleanup. Mintlify's drift automation drafts updates from merged pull requests for review. Cloudflare's docs repository comments on potential redirects when a pull request deletes content files. None of these pages describes publishing a lasting statement that the endpoint is gone.

The removal playbook

The playbook adds that statement and makes it reach every surface in one release. For the CI wiring that catches an endpoint change before merge, follow our guide to keeping your API reference in sync with code. For the spec, llms.txt, and 410 response examples, follow our step-by-step guide to stopping AI assistants from recommending removed API endpoints.

Owners and inputs

The engineer who removes the operation owns the spec and the API response. The technical writer owns the docs surfaces and the check after the release. Before the release, the writer needs four inputs: the spec diff, the replacement endpoint, the migration guide, and a list of every page that mentions the old endpoint path across the docs, help center, READMEs, and sample repositories.

In this split, engineering owns two of the seven surfaces and the writer owns four, so most of a removal is docs work that has to land in the same release.

HTTP standards already describe the API response row. RFC 9110 defines 410 Gone as "access to the target resource is no longer available at the origin server." RFC 9745 (March 2025) defines a deprecation link relation that points to human-readable documentation about the deprecation. RFC 8594 defines the sunset link relation for retirement policy. Your retirement page is the documentation those links point to. Also check that robots.txt lets the answer-time fetchers in the first table read it. OpenAI says its search systems can take about 24 hours to adjust after a robots.txt update.

Cadence and checkpoints

1. At the deprecation announcement, mark the operation deprecated, add the notice to its reference page, add the llms.txt instruction, and add the changelog entry.
2. On the pull request that removes the operation, make the docs impact decision GitBook recommends for every pull request. Checkpoint: a linked docs pull request exists before the removal merges.
3. On release day, confirm the retirement page is live at the old URL and the sweep of guides, samples, and help center articles is merged.
4. After the release, ask the assistants again and compare the answers.

One worked cycle

Say your team is retiring /api/v1/users for a v2 endpoint. At announcement, the engineer sets deprecated on the operation, and the writer adds a reference notice, an llms.txt line naming the v2 replacement, and a changelog entry. When the removal pull request opens, the writer opens a linked docs pull request. It turns the reference page into a retirement page, replaces the v1 call in the quickstart, the guides, and the sample repository, and updates the help center article on listing users. The engineer makes the old route return 410 with a link to the retirement page. Both pull requests merge in the same release. A week later, the writer reruns the assistant questions and compares the answers with the ones recorded before the release.

How to check whether assistant answers changed

None of the guides we reviewed compares assistant answers from before and after a removal. Build the check yourself:

1. Before the release, write down the questions developers ask about the removed endpoint, in their words.
2. Ask each assistant every question with browsing on, more than once. Record whether the removed endpoint appears and which pages the answer cites.
3. After the release, ask the same questions in the same assistants and compare.

An answer that cites your retirement page means the page is being retrieved. An answer that cites a third-party tutorial still showing the old call gives you a page to ask its author to update. An answer with no citations may come from training data, which your pages do not reach. Date every run, because no source we found says how long answers take to change.

Where EkLine fits

For the docs side of this fix, use EkLine Docs Agent. The sweep is where the playbook is hardest: the guides, READMEs, and knowledge-base articles a generated reference never touches. Docs Agent keeps READMEs, API references, guides, and knowledge-base pages current in three ways. When a pull request opens, it checks the diff against your docs and, on a high-confidence match, drafts the update as a documentation pull request. On a schedule you set, a drift audit finds customer-facing changes that merged without documentation and fixes the highest-priority gap. And it takes on-demand requests, like the example in its update docs: "We deprecated the /api/v1/users endpoint. Update the API reference to mark it as deprecated and point to the v2 replacement." In GitHub, mentioning @ekline-ai in a pull request comment starts a session that drafts the docs and can open a docs pull request.

Before: the removal merges, the reference page disappears, and the guides keep the old call until a developer reports it. After: on a high-confidence match, the removal pull request produces a docs pull request covering the reference and the pages around it, and your writer reviews and approves it, because "Every update arrives as a pull request on your documentation repository." At Socure, one technical writer merged 348 documentation updates in seven weeks with no new hires.

EkLine also runs the check after the release. It asks ChatGPT, Claude, Perplexity, and Gemini the questions your developers and buyers ask, shows which pages they cited, and lets you "Re-run the same questions and tasks after you ship and compare." EkLine cannot change what a model already learned. It keeps your pages current and shows whether the answers that browse have caught up. For drift beyond removed endpoints, see our guide to documentation drift.

Book a demo to see what AI assistants currently say about your API, or get started with EkLine.

---

Sources

• arXiv: LLMs Meet Library Evolution
• OpenAI: Overview of OpenAI Crawlers
• Anthropic: crawler help article
• Perplexity Crawlers
• Google common crawlers
• Grounding with Google Search
• Google Search: HTTP status codes
• ReadMe: Sync docs with OpenAPI
• OpenAPI Specification v3.1.1
• GitBook: Keep technical documentation accurate
• OpenAI Community: Codex suggests deprecated API
• firebase-tools pull request 11350
• Firecrawl: Why do LLMs hallucinate deprecated APIs?
• OpenAI: Deprecations
• Stripe: The Sources API
• Stripe: llms.txt
• stripe/openapi
• The /llms.txt file
• Apideck: Stripe's llms.txt instructions section
• Stripe: The Source object
• Bump.sh: Continuous integration
• ReadMe: AI writer and documentation drift
• Mintlify: How to stop documentation drift
• Cloudflare: Contributions
• RFC 9110: HTTP Semantics
• RFC 9745: Deprecation header
• RFC 8594: Sunset header
• EkLine: Keep your API reference in sync with code
• EkLine docs: Stop AI assistants from recommending removed API endpoints
• EkLine: Documentation drift
• EkLine docs: Keep documentation in sync with code
• EkLine docs: Update and review documentation
• EkLine docs: GitHub pull requests
• EkLine homepage
• EkLine case study: Socure