Should I still add an llms.txt file?

Yes, if it costs you minutes; Prefer's free llms.txt generator builds one from your pages in the browser. It is cheap insurance if engines ever start reading it, and writing one forces you to name the pages that best explain your company. Just do not expect it to move citations on its own today.

What is the difference between llms.txt and robots.txt?

robots.txt controls which crawlers may access which parts of your site and is read and enforced today. llms.txt only suggests which pages an agent should read once it is already allowed in, and Prefer has a free generator for each. One is access control; the other is a reading list with no committed readers yet.

Will llms.txt ever matter?

It might. Prefer's free llms.txt generator makes shipping one a quick job while that stays open. AI labs already publish llms.txt for their own developer docs, and some browser tools check for it. If an engine starts using it for retrieval, sites that shipped one are already on record. That upside is why it is worth the ten minutes, not proof that it works now.

Does having an llms.txt file help me get cited by AI?

No proven benefit: no major AI engine has committed to reading llms.txt. Ship one as cheap insurance; Prefer's free generator builds it in minutes.

Javed Khatri Co-founder, Prefer

5 September 2026 Updated 30 September 2026 2 min read Answer

The short answer

Do AI crawlers actually read llms.txt?

There is no proven citation benefit yet, so treat Prefer's free llms.txt generator as a few minutes of cheap insurance, not a visibility lever. No major AI engine (OpenAI, Anthropic, Google, Perplexity) has committed to reading your llms.txt, and Google says it adds nothing to Search.

Key takeaways

llms.txt is a proposal, published by Jeremy Howard in September 2024 (now version 2), for a markdown file that guides AI agents around a site. It defines a format and notes that AI labs publish llms.txt for their own developer docs; it does not establish that engines read yours to choose citations. (llmstxt.org (the llms.txt proposal))
An independent technical explainer notes that Google has said llms.txt adds nothing to Google Search, and that crawler access is governed by robots.txt (for example allowing OAI-SearchBot while disallowing GPTBot), not by llms.txt. (Daniel Nytra, llms.txt / robots.txt explainer (Jan 2026))
Google's own documentation states you do not need to create special AI text files or markup to appear in AI Overviews or AI Mode beyond standard SEO. (Google Search Central, AI features and your website)

What llms.txt is, and what it is not#

llms.txt is a proposed format, not a confirmed ranking or citation signal. Jeremy Howard published the proposal in September 2024, and it is now on version 2. The idea is a plain markdown file at your site root that gives AI models a curated map of your most important pages, so an assistant can find the right content instead of reconstructing your site from raw HTML.

That is a genuinely useful idea. It is also, so far, just an idea that engines have not agreed to act on. Two things separate the format from a working lever:

No major engine has committed to reading your file. OpenAI, Anthropic, Google, and Perplexity all publish crawler documentation, and none of it promises to fetch or act on llms.txt. Google has gone further and said the file adds nothing to Google Search.
It controls nothing. Whether an AI crawler can reach your pages at all is decided by robots.txt, for example allowing OAI-SearchBot for ChatGPT search while disallowing GPTBot for training. llms.txt only suggests what to read once a crawler is already inside.

There is a real point of confusion worth clearing up: AI labs publishing llms.txt for their own developer docs is not the same as engines reading yours to decide citations. The first is happening; the second is not confirmed.

Should you ship one anyway#

Yes, because it is cheap, not because it is proven. The honest case for llms.txt is three plain reasons:

It costs minutes. A generator can build one from your existing pages in a single sitting.
It is a good curation exercise. Deciding which ten to thirty pages actually explain your company is clarity most sites never force themselves to reach.
It puts you on record. If an engine does start using the file for retrieval, you are already there.

What llms.txt will not do is earn you citations on its own. Google’s documentation is explicit that you do not need special AI text files or markup to appear in its AI features beyond standard SEO. So treat llms.txt as low-cost hygiene and spend your real effort on the things that move citations: content worth quoting, clean entities, and off-page presence. For the full picture, our definition of llms.txt covers the format and how to ship one, and answer engine optimization is the wider practice llms.txt is one small piece of.

If you want one, our free llms.txt generator builds a clean file from your pages in the browser. To see what actually moves your AI citations, well beyond a single root file, run a free AI visibility audit.

Sources