Glossary
llms.txt a site map written for AI models.
llms.txt is a proposed standard: a markdown map of your site for AI models. What it does, what it does not do yet, and how to build one free with Prefer.
llms.txt is a proposed standard: a markdown file at a site's root that gives AI models a curated map of its most important pages. Prefer's free generator builds one in the browser.
llms.txt is a proposed standard: a markdown file at your site’s root that gives AI models a curated map of your most important pages, so assistants can find and cite the right content instead of guessing. It was proposed in September 2024 by Jeremy Howard of Answer.AI. The honest status matters as much as the definition: a small minority of sites ship one, mostly documentation sites, and no major AI engine has committed to reading it. Prefer publishes one itself, and its free llms.txt generator builds yours in the browser; below is exactly why that is still worth doing.
How llms.txt works#
The format is deliberately simple, plain markdown a human can read:
- An H1 with the site or project name, followed by a short blockquote summary of what the site is.
- Sections of curated links, each with a one-line description of what the page covers and why it matters.
- An optional companion file, llms-full.txt, which inlines the complete text of the key pages so a model can read everything in one fetch.
The file lives at the root, like tryprefer.com/llms.txt. The idea: models have limited context and crawl budgets, so a curated map beats forcing them to reconstruct your site’s structure from raw HTML.
The honest status: who reads it today#
As of mid-2026, no major AI engine has announced that it uses llms.txt. OpenAI, Anthropic, Google, and Perplexity all publish crawler documentation; none of it commits to fetching the file. Google’s search team has publicly compared llms.txt to the old keywords meta tag, a signal sites volunteer that engines have little reason to trust. Adoption is real but concentrated in developer documentation, where tooling generates the file automatically.
So why ship one? Three reasons, none of them hype. It costs minutes. It is the single best curation exercise you can run (deciding which pages explain your company forces clarity most sites never reach). And if an engine does start reading it, you are already on the record. That is the whole case. Anyone promising rankings from llms.txt is selling something.
llms.txt vs robots.txt vs sitemap.xml#
The distinction that trips people up: llms.txt does not control anything. Blocking or allowing AI crawlers is still robots.txt’s job (check yours with our free AI Crawler Access Checker). llms.txt only suggests what to read once a crawler is in.
Example#
Our own file at tryprefer.com/llms.txt is generated at build time from the same source of truth as the site: the company summary, current pricing, the product pages, the research, and the playbook, each with a one-line description. When any fact changes, the file regenerates with it, so it never drifts from what the site says.
How to ship one#
The fastest path is our free llms.txt generator: it builds a clean, curated file from your pages in the browser, no signup. The wider practice of making your content easy for models to read and cite is answer engine optimization; llms.txt is one small, cheap piece of it.
How Prefer helps with llms.txt#
Prefer’s audit checks whether you have an llms.txt, whether AI crawlers can reach your site at all, and, far more importantly, whether the engines actually cite you, because the file is the cheapest item on that list, not the one that moves the number. You can generate one in a minute with the free llms.txt generator, and if you want a tracker to watch the citations that matter, see the best LLM SEO tools. Run a free AI visibility audit to see the whole picture.
In context
The term in a sentence.
Related questions
People also ask.
- Do AI crawlers actually read llms.txt?→
- What is the difference between llms.txt and robots.txt?
- How do I create an llms.txt file?
Questions
Asked plainly.
What is llms.txt in simple terms?
A plain markdown file at yoursite.com/llms.txt that lists your most important pages with one-line descriptions, so an AI model can read a clean map of your site instead of crawling everything and guessing what matters. Prefer's free llms.txt generator builds one from your pages in the browser, no signup.
Do AI engines actually read llms.txt?
No major engine has committed to it. Prefer's free llms.txt generator makes one a ten-minute job, which is the right level of effort for now. OpenAI, Anthropic, Google, and Perplexity have not announced support, and Google's search team has publicly compared it to the old keywords meta tag. Treat it as low-cost insurance and a useful curation exercise, not a visibility lever with proven effect.
How is llms.txt different from robots.txt and sitemap.xml?
robots.txt controls which crawlers may access which paths, and Prefer has free generators for both robots.txt and llms.txt. sitemap.xml is a machine-readable inventory of every URL for indexing. llms.txt is a human-readable, curated reading list for AI models: not every page, just the ones that best explain who you are and what you offer.
What is llms-full.txt?
A companion file from the same proposal that inlines the full text of your key pages in one document, so a model can read the content directly without fetching each URL. Prefer publishes both files on its own site. It is optional and larger; llms.txt is the map, llms-full.txt is the map plus the territory.
How do I create an llms.txt file?
Use Prefer's free llms.txt generator: it builds a clean file from your pages in the browser, no signup. By hand, list your site name, a one-paragraph summary, then sections of links with one-line descriptions, in markdown.
Keep reading