When should a website expose an API for AI agents?

Agents already use your website

Before deciding to build anything, it helps to know that AI agents don't need an API to reach you. Several providers now operate agents that browse ordinary web pages and act on a user's behalf:

Google documents Google-Agent, used by “agents hosted on Google infrastructure to navigate the web and perform actions upon user request”. Like Google's other user-triggered fetchers, it generally ignores robots.txt, because a person asked for the visit.
OpenAI's cloud browser for ChatGPT signs its requests so sites can verify them. OpenAI's own allowlisting guide notes that at launch it could not sign in to websites or complete payments.
Google's guidance for site owners describes browser agents gathering information by “inspecting the DOM structure, and interpreting the accessibility tree”.

That last point is the important one. A browser agent reads roughly what a screen reader reads. If your content is in the server HTML, your headings make sense, and your forms have real labels, an agent can already find facts and fill in a simple request. That's Levels 1–3 in our machine readiness levels, and for many businesses it's the right place to stop.

Match the interface to the task

The deciding factor is what an agent would need to do. Reading is cheap to support. Committing an action on someone's behalf is not.

Recommended interface by agent task
What the agent needs to do	Examples	What we'd recommend
Read facts about you	Hours, prices, service area, policies	Well-built HTML, facts in plain text, structured data that matches it. No API.
Submit a simple request	Callback request, contact form	An accessible, labeled form, and bot rules that don't block verified agents. No API yet.
Query live data	Availability, stock, a price quote	A read-only API with an OpenAPI description, listed in an API catalog.
Commit an action	Book, order, cancel, pay	An authenticated API with scoped permissions, safe retries and an explicit confirmation step.
Be called directly by AI clients	Assistants that connect to tools	An MCP server over the same operations — once the API exists and is stable.

Signs an API is worth building

The task is structured and repeated. Checking availability, quoting a standard job, looking up an order — tasks with clear inputs and a single right answer.
The data changes faster than pages are re-read. Stock levels, appointment slots and dynamic prices go stale in any crawled copy. An API answers from the source.
The web flow is fragile for software. Multi-step wizards, date pickers built from custom widgets, and flows that depend on client-side state are where browser agents fail. Fix them for humans first; an API is the robust path for software.
You can say who is allowed to do what. If an agent acts for a customer, you need to know which customer and which actions they've permitted. If you can't answer that, you're not ready for a transactional API.
You can operate it. An API is a product: documentation, versioning, monitoring, rate limits and support. An abandoned API is worse than none, because agents will keep calling it.
There's evidence of demand. Verified agent sessions reaching your booking or quote flows, partners asking for integration, or customers asking whether their assistant can book. Rising AI crawler volume is not evidence — see what crawler traffic doesn't prove.

Signs it isn't — yet

Your business is mainly information: a firm, a practice, a publisher. Clear pages serve agents well.
Each sale needs human judgment, a site visit or a custom quote. An agent can submit the enquiry through your form.
Volume is small enough that a person handles every request anyway.
No one would own the API after launch.

What an agent-facing API should include

Agents are unforgiving clients: they read your description literally, retry on failure, and can't ring you to ask what an error meant. Most of what they need is existing, standardized practice.

A machine-readable contract

Describe the API with the OpenAPI Specification (3.2.1 at the time of writing). Write operation descriptions as if the reader has no other context — because an agent hasn't. State units, time zones, currencies and what each status code means.

Discovery at a well-known address

RFC 9727 (June 2025) defines /.well-known/api-catalog: a single URL that lists a publisher's APIs as a linkset, pointing to each API's description, documentation and status. It's cheap to publish and removes guesswork.

An API catalog for a fictional equipment-rental business SYNTHETIC

GET /.well-known/api-catalog HTTP/1.1
Host: fernhill-rental.example

HTTP/1.1 200 OK
Content-Type: application/linkset+json;
  profile="https://www.rfc-editor.org/info/rfc9727"

{
  "linkset": [
    {
      "anchor": "https://api.fernhill-rental.example/availability/v1",
      "service-desc": [
        { "href": "https://api.fernhill-rental.example/availability/v1/openapi.json",
          "type": "application/json" }
      ],
      "service-doc": [
        { "href": "https://fernhill-rental.example/developers/availability",
          "type": "text/html" }
      ],
      "status": [
        { "href": "https://api.fernhill-rental.example/availability/v1/health",
          "type": "application/json" }
      ]
    }
  ]
}

Errors a machine can act on

Return errors as problem details (RFC 9457, application/problem+json) so a client can tell “sold out” from “bad date format” from “try later” without parsing prose. For rate limits, use 429 Too Many Requests, which RFC 6585 says may include a Retry-After header — and include it, so a well-behaved agent backs off instead of hammering you.

A rate-limit response with problem details SYNTHETIC

HTTP/1.1 429 Too Many Requests
Retry-After: 30
Content-Type: application/problem+json

{
  "type": "https://fernhill-rental.example/problems/rate-limited",
  "title": "Too many availability checks",
  "status": 429,
  "detail": "Limit is 60 requests per minute per client. Retry after 30 seconds."
}

Safe retries

Agents retry. HTTP already distinguishes idempotent methods such as PUT and DELETE from POST, which isn't (RFC 9110). For anything that creates a booking or charge, make a repeated request harmless — for example by accepting a client-generated key and returning the original result for duplicates. An Idempotency-Key header was proposed in an IETF draft that expired without becoming an RFC, so treat that specific header as a convention, not a standard.

Permissions and a confirmation step

Use OAuth with narrow scopes, so a customer can let an agent check availability without letting it cancel bookings. Split consequential actions in two: one call prepares a quote or hold, a second commits it. That gives the agent — and the person behind it — a clear moment to confirm, and gives you an audit trail.

Where MCP fits

The Model Context Protocol is an open protocol that lets AI applications connect to servers offering tools, resources and prompts. It doesn't replace an API; in practice an MCP server is a thin layer over operations you already expose. Two things in the specification are worth knowing before building one:

It places responsibility on the AI application: hosts “must obtain explicit user consent before invoking any tool”. Your design should still assume a call might not have been reviewed, and keep the prepare-then-commit split.
Authorization is optional in the protocol, but for HTTP-based servers the specification builds on OAuth — including RFC 9728 protected resource metadata — so a server with proper OAuth in place is most of the way there.

Our default order is API first, MCP second, and only when the clients your customers actually use can connect to it.

Experimental — watch, don't depend on

WebMCP Experimental — a proposed JavaScript API for web pages to register tools that browser agents can call. It's a draft Community Group report, explicitly “not a W3C Standard nor is it on the W3C Standards Track”. Worth prototyping; not worth rebuilding a site around.
Web Bot Auth Internet-Draft — HTTP message signatures that let agents prove who operates them. OpenAI's cloud browser already signs requests this way, and Cloudflare uses it to verify bots, but the IETF protocol is still a working-group draft. We cover how to use it in measurement in measuring humans, AI referrals and agents separately.

A sensible sequence

Make the website work for a browser agent: server-rendered content, labeled forms, bot rules that let verified agents through.
Measure verified agent sessions separately from crawlers and humans, so you can see real demand.
If live data matters, publish a read-only API with an OpenAPI description and an API catalog.
Add authenticated write operations with scopes, safe retries and a confirmation step when agents are genuinely trying to transact.
Wrap it in an MCP server if your customers' AI clients can use one.

Each step is useful on its own, and each is cheaper to get right than to retrofit.