AI can't recommend what it can't read.

A plain text file at the root of a site, like robots.txt, that tells a large language model what the site is and which pages are worth reading. It is a proposed standard rather than a requirement: no assistant refuses a site for lacking one, and a site with a good one is easier to summarise correctly. Ours is at /llms.txt if you want a working example.

Being readable is step one — an assistant can't suggest a site it can't parse. Step two is being somewhere assistants already read and trust. This checker handles the first; listing on wayb handles the second.

Almost always because the words are not in the HTML. Assistants fetch a page the way curl does — they do not run your JavaScript — so a React, Vue or Next client-side app hands them a title, an empty root element and nothing to read. The fix is to server-render or prerender: the text has to be in the bytes on the wire, not assembled in the browser afterwards. If you can read your page with "view source" rather than the inspector, an assistant can too.

Check robots.txt for a Disallow aimed at GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot or Google-Extended. Blocking them is usually inherited from a template or an "AI scraper" plugin default rather than chosen. Removing those lines is the whole fix. Note that ChatGPT-User is the fetcher used when a person asks ChatGPT to open your link, so blocking it breaks the case where somebody is actively trying to read you.

A markdown file at the root of your domain. Start with an H1 naming the site, a blockquote saying what it is in a sentence, then a list of links to the pages worth reading, each with a short note on what it contains. Keep it under a page. The most common mistake is not writing one badly but serving your app shell at that URL: a single-page app with a catch-all route answers 200 for /llms.txt with a page of HTML, so it looks present and contains nothing.

A title that names the thing and what it does, and a description of at least a full sentence. This is not a ranking trick: it is the text an assistant quotes when it names you, so a description that reads "Home | My Site" gives it nothing to say on your behalf. Write the sentence you would want repeated back to somebody who asked about you.

It is the smallest of the six and still worth doing. Structured data states what the page is about rather than leaving it to be inferred from prose — an Organization, a Product, a WebPage. One script tag in the head, and it removes an ambiguity rather than adding a signal.

Yes, and more than it used to. A sitemap is how a crawler that has never seen a link to you finds every page in one request — which is exactly the position a new site is in. Serve /sitemap.xml and add a Sitemap: line to robots.txt; the line is the part people forget, and it is the only directive in that file that does something a page cannot do for itself.

The six checks are not worth the same, so four of six passed is not the same number as the percentage. Failing "an assistant can read the page" is not the same size of problem as failing "there is structured data", and averaging them would read more easily and be less true.

• 30 points — An assistant can read the page. The largest, because nothing else matters if there are no words to read.
• 25 points — AI crawlers are allowed in. Next largest: a page that cannot be fetched is a page that was never read.
• 15 points — There is an llms.txt. Advisory and young. No assistant requires it, and a site without one is usually fine.
• 10 points — A sitemap exists and is findable. How a crawler with no inbound link finds the rest of you.
• 10 points — The page says what it is. The sentence an assistant quotes when it names you.
• 10 points — There is structured data. Removes an ambiguity rather than adding a signal.

Whether an assistant mentions you by name today. That means asking the models directly, which costs a request per check — Ahrefs gives that away free and it is worth running too.