The llms.txt Myth: What the Evidence Actually Shows
llms.txt was pitched as the file that tells AI what to cite. A year on, the data says that promise hasn't held up. Here's what the file is, what the evidence shows, and where it still has a place.
- 01llms.txt is a proposed Markdown file that gives AI systems a curated guide to your site. It isn't an official standard enforced by any major search engine.
- 02A large analysis of about 300,000 domains found no measurable link between having llms.txt and being cited by AI.
- 03Google has said it doesn't use llms.txt, and its AI search guidance says it isn't needed for generative features.
- 04It's cheap and harmless to publish, but it's not a citation lever, and the time is usually better spent elsewhere.
Few GEO tactics have spread as fast as llms.txt. It's easy to add, easy to sell, and easy to believe. It's also one of the clearest cases in AI optimization of enthusiasm running ahead of evidence.
TL;DR: llms.txt is a proposed file, introduced in 2024, that offers AI systems a curated Markdown summary of your site. Adoption is low, and a Search Engine Journal report on an SE Ranking study of roughly 300,000 domains found it had no measurable relationship with AI citations. Google has said it doesn't use the file and that it isn't needed for its generative search features, and OpenAI points site owners to robots.txt. It does have a real use in developer documentation and agent tooling. For marketing visibility, treat it as optional, never as a promise. We've covered it as an emerging signal in earlier posts, and this is the fuller picture the evidence supports so far.
What llms.txt is
The proposal comes from Jeremy Howard of Answer.AI. You place a Markdown file at /llms.txt that lists your most important pages with short descriptions, so a language model can quickly understand your site. A companion llms-full.txt can include the full text of those pages. The spec is described at llmstxt.org. Crucially, it's a proposal. It grants nothing, blocks nothing, and has no enforcement mechanism.
What the evidence says
- Large-scale correlation study. SE Ranking found the file on roughly one in ten domains and reported no measurable link between having it and citation frequency. Removing the variable from their model actually improved accuracy. See the Search Engine Journal write-up.
- Platform statements. Google has said it doesn't support llms.txt, and its generative AI search guidance indicates the file isn't needed. OpenAI's crawler guidance centers on robots.txt, not llms.txt.
- Mixed signals elsewhere. Some crawlers have been seen fetching the file, and Perplexity has said it may use it to prioritize pages, but fetching is not the same as influencing what gets cited.
- Where citations come from. The sites that win AI citations mostly don't bother with the file. Authority, clarity, and third-party mentions seem to matter far more.
Why it probably doesn't work as advertised
Most AI answer systems retrieve through search indexes. A curated file on your own domain asks a model to take a detour most retrieval pipelines weren't built to make. It also can't help a page that isn't indexed, or one that isn't the best answer.
Where it genuinely helps
- Developer documentation. IDE assistants and agent tools can use it to find relevant docs quickly, which is why many technical companies publish it.
- Agent-readability experiments. If you expect AI agents to navigate your site, a clean map is harmless.
- Internal discipline. The exercise of listing your key pages can clarify your own site structure.
What to do about it
- Don't pay extra for it. If an agency sells llms.txt as a headline GEO deliverable, ask for evidence.
- Publish one if it's cheap. It costs almost nothing and does no harm, as long as you don't promise results.
- Control access in the right place. Crawler permissions belong in robots.txt.
- Spend your effort on what correlates with citations: clear, structured answers, original information, crawlability, and reputation off your own site. Our guides on how AI search engines choose sources and GEO for marketers cover these.
- Beware manipulation. Google has said its spam policies apply to AI answers, so don't try to game them with hidden instructions or doorway pages. See our take on GEO and SEO.
FAQ
Will llms.txt hurt me? No. It's harmless, just not proven to help.
Do Claude or ChatGPT read llms.txt? Some tooling does in specific contexts, but no major provider has confirmed that it influences citations in consumer answers.
Should I delete mine? There's no need to. Just stop expecting it to move visibility.
Will this change? Possibly. Standards evolve, and a major platform could adopt it. Until one does, treat claims of impact as unproven.
Last updated: October 19, 2026. Platform support for llms.txt can change; check each provider's current documentation before relying on it.
