Does llms.txt do anything?

It is the first item on almost every AI-visibility checklist, it takes fifteen minutes, and it is the easiest thing in GEO to sell. It is also, as far as anyone has been able to measure, read by nobody.

The idea behind /llms.txt is reasonable. Put a plain Markdown file at the root of your domain that says what your site is and links to the pages that matter. AI systems read it, orient themselves, and stop having to guess. The proposal came from Jeremy Howard in late 2024. It is not an official standard and never was — it is a suggestion that the SEO industry adopted with more enthusiasm than the AI industry did.

Since then it has spread fast. Adoption in the top 10,000 websites went from 1.04% in July 2025 to 5.61% in June 2026, roughly a fivefold increase in twelve months (Casey Burridge, HTTP Archive data). Among the Tranco top 1,000, 8.7% publish one (Rankability). A large chunk of that growth is not a decision anyone made: Shopify quietly pushed the file to every store on the platform in spring 2026, which is why Shopify sites sit at 78.1% adoption.

So the file is everywhere. The obvious next question is whether anything opens it.

The measurement

Ahrefs took the server logs of 137,210 domains and counted every request to /llms.txt during May 2026. Of those domains, 28% publish the file — around 38,000 sites. Here is what happened to them.

The 3% that did get traffic were not visited by who you would expect. Of every request that reached an llms.txt file, named AI bots accounted for 19.5% — and that number is generous, because it lumps in training crawlers and coding agents. The bots people actually publish the file for, the retrieval bots that fetch pages to answer a live question, made up 1.1%. Slackbot, the thing that generates link previews in chat, fetched llms.txt files more often than PerplexityBot did (Ahrefs).

A separate 90-day experiment points the same way. Otterly.ai put an llms.txt on a live site, linked it from the homepage, and logged AI bot traffic for three months. Out of 62,100 AI bot visits, 84 went to the file — about 0.1%. The site's average content page got roughly 265 visits in the same window, so the dedicated AI entry point performed about three times worse than an ordinary page (Otterly.ai).

And the finding that settles it: Ahrefs also looked at requests for llms.txt files that do not exist, and found the AI bot share of those 404s was zero. No AI system goes knocking. If you have not published one, nothing is standing at your door being turned away.

Why it was never going to work

John Mueller of Google compared llms.txt to the keywords meta tag, which search engines stopped trusting more than a decade ago. The reasoning is the same in both cases: a signal the subject writes about itself is worth very little to the party doing the judging. Why read your description of your site when the site is right there? Google's own guidance on generative AI features has a section titled "mythbusting" that tells site owners they do not need machine-readable files to appear in AI results. Mueller's later framing was that llms.txt is a "temporary crutch, perhaps to save some tokens" for AI coding tools reading developer documentation — not something a normal business site needs to think about (both quoted in the Ahrefs study).

The adoption pattern backs that reading. Not one of the major consumer AI platforms — chatgpt.com, claude.ai, gemini.google.com, openai.com, anthropic.com — publishes an llms.txt on its own domain. But the developer documentation sites of Anthropic, xAI, Perplexity, Mistral and Cohere all do (Burridge). The labs put the file exactly where it earns its keep: on technical docs that a coding agent is constantly asked to read. Not on the marketing site.

What to do with the fifteen minutes

Publish it if your CMS does it for you — Shopify and Wix already do, the cost is zero, and if agents ever do mediate AI search the file is already there. Do not pay anyone to write one. Do not let it sit at the top of a checklist above things that are measurably load-bearing.

Spend the time on the ordinary web instead, because that is where the retrieval bots actually go. In the audits we run, the most common blocker is not a missing file at the domain root. It is a page that answers a customer's question well and cannot be reached — absent from the sitemap, linked from nowhere, invisible to a crawler that only follows links. Fixing that takes an afternoon and changes what gets quoted.

The test for any GEO recommendation is the same one llms.txt fails: can you show me, in a log file, that a machine came and read it? If the answer is no, it is a hypothesis, and you should be told it is a hypothesis before you pay for it.