WebBotAuth.net

What is Web Bot Auth?

Web Bot Auth is a standard that lets automated HTTP clients (bots, crawlers, and AI agents) prove their identity by signing their requests with a cryptographic key, instead of a User-Agent string that any client can fake.

How it works

A bot operator generates an Ed25519 key pair and publishes the public key (as a JWK) at a well-known URL on their domain: 

/.well-known/http-message-signatures-directory

When the bot makes an HTTP request, it signs that request using RFC 9421 HTTP Message Signatures. The signature covers selected headers and pseudo-headers (method, path, authority, and so on) and includes a tag=web-bot-auth parameter so verifiers know which signature scheme this is.

The server (or an intermediary) can verify the signature by fetching the bot's JWKS directory, finding the key whose RFC 7638 thumbprint matches the signature's keyid, and checking the Ed25519 signature. If the signature verifies, the request genuinely came from whoever controls that key, with no User-Agent trust required.

Why this matters

User-Agent strings are trivially forged. Web Bot Auth shifts identity verification to a cryptographic proof: only the holder of the private key can produce a valid signature. This makes it possible for site operators to distinguish legitimate bots from impersonators and to build per-bot access policies based on verifiable identity.

Key components

  • JWKS directory - the JSON Web Key Set published at /.well-known/http-message-signatures-directory, containing the bot's Ed25519 public key(s).
  • RFC 9421 signature - the HTTP Message Signature attached to each request, produced with the corresponding private key.
  • Agent Card (optional) - a metadata document linked from the directory that describes the bot's purpose, contact information, and rate expectations. Checked by this tool as advisory information.

What this checker does

This tool fetches a domain's published directory and inspects it against the spec: are the keys well-formed? Do the kid values match their RFC 7638 thumbprints? Is there no private key material exposed? When you also paste request headers, it verifies the RFC 9421 signature. The result is a structured report with a letter grade.

See how grading works for the full scoring reference.