Key Takeaways
- Treat llms.txt as a proposal, not a standard. It is a suggested file format for pointing a language model at your best pages. It is well specified and widely discussed, and it is not something any assistant has committed to reading.
- Notice the difference between publishing one and reading one. Several AI companies publish llms.txt files for their own documentation. That is not the same as their assistants reading the file on your site, and the two get conflated constantly.
- Fix the files that are documented first. robots.txt decides whether an assistant crawler may read you at all. A sitemap tells a crawler what exists. Both have published specifications and both are easy to get wrong.
- Add one if it costs you an hour, not if it costs you a project. A small handwritten file with links to your best pages is cheap and harmless. Rebuilding your site around it, or paying someone to, is not proportionate to the evidence.
- Write the pages before you write the index. A file pointing at thin pages just makes thin pages easier to find. Nothing in this format creates substance that the pages themselves do not already have.
Somebody in a coaching group posts that you need an llms.txt file or AI will ignore your website. A week later somebody else posts that it does nothing. Both sound confident.
The honest position sits between them, and it is knowable rather than a matter of opinion. There is a written specification, there are companies publishing the file, and there is a documentation gap where the evidence of it being read would be.
Here is what the file is, what it contains, who actually reads one, and whether it deserves an hour of your week.
What is llms.txt, and should your coaching site have one?
It is a proposed plain text file at the root of your site that lists the pages a language model should read first. Adding one is cheap and harmless, and it should sit near the bottom of your list.
The proposal comes from Jeremy Howard, who published the specification at llmstxt.org in September 2024. That site is the primary source for the format and the only place the specification lives, which is why it is cited here directly rather than through somebody summarizing it.
What it is not is a standard that assistants have agreed to follow. That distinction is the whole post, and it is the part that gets lost when the file gets sold to you as a service.
What is actually in the file
Very little, by design. The specification requires exactly one thing: a heading with the name of the site or project. Everything after that is optional.
The optional parts are a short summary of what the site is, and lists of links grouped under headings, each link with a sentence saying what it contains. It is written in the same lightweight markup people use for readme files, so it reads fine to a person and parses cleanly for a machine.
For a coaching site, a complete one would be a paragraph naming your practice, then perhaps six links: what you do, who you work with, how you price, your two best explainer pages, and your contact page. It takes about twenty minutes to write and never needs to be long.
Who reads it today, and who only publishes one
Several AI companies publish an llms.txt for their own developer documentation. OpenAI's crawler documentation, for example, links to an llms.txt as the index of its own docs. That is a real fact and it is routinely reported as if it settled the question.
It does not. Publishing a file that helps models read your documentation is a different act from your assistant reading that file on someone else's site. What OpenAI documents for site owners is robots.txt control over its named crawlers, and llms.txt appears nowhere in that role.
We have looked for a published statement from any major assistant that its crawler reads llms.txt on third-party sites, and we have not found one. That is a documentation gap, not proof of nothing happening. It is also exactly the state in which a service should not be sold to you on a promise.
The plates are the files. The bars underneath are what is documented about who reads them, drawn solid where a named company has published it and dashed where it rests on a proposal. Two of the three are settled. The third is the one being sold hardest.
How it differs from robots.txt and a sitemap
The three files answer different questions. robots.txt says what a crawler may fetch. A sitemap says what exists. llms.txt proposes what is worth reading first.
Only the first two rest on published specifications with named crawlers behind them. Google's introduction to robots.txt describes exactly what the file controls and what it does not, and it is worth reading once, because a robots.txt that blocks an assistant crawler is a self-inflicted invisibility that no amount of writing fixes.
That is the practical hierarchy. Unblock, then list what exists, then optionally suggest a reading order. Most sites we look at have a problem at the first step and are being sold a solution at the third.
| File | What it says | Documented as read | Worth to a coaching site |
|---|---|---|---|
| robots.txt | Which crawlers may fetch which paths | Yes, by Google and by OpenAI, with named agents | High, and mostly as a thing to stop getting wrong |
| sitemap.xml | Every page worth finding, with a changed date | Yes, by Google, as a crawl input | Moderate, and usually generated for you already |
| llms.txt | The pages a model should read first, with a note on each | No published statement from any assistant | Low, but the cost is twenty minutes |
| The pages themselves | The actual answers a buyer question lands on | Yes, this is what gets quoted | Highest, and the only one nobody can shortcut |
Whether a coaching site should bother
Add one if it takes an afternoon and nothing else is waiting. Do not restructure your site for it, do not pay a monthly fee for it, and do not let it displace the work of writing pages that answer real questions.
The reasoning is proportionality. The upside is plausible and unmeasured. The cost of a small file is near zero, so a plausible upside is worth having. The cost of a project is not, because a project has to displace something with evidence behind it.
What has evidence behind it is duller. Pages that state your facts plainly, marked up so a machine does not have to infer them, in language a buyer actually uses. The markup half of that is explained in what schema markup is, the broader picture in this plain-English guide, and the single highest-value page to write first in the one page AI quotes about your niche.
If the file becomes documented tomorrow, the twenty minutes you spent will look prescient and the pages you wrote will still be the reason you get named. That asymmetry is the whole argument for doing the boring half first.
If you want to know where you stand before deciding what to build, the free check runs 25 real buyer questions from your niche through ChatGPT at $0 and emails a plain-English readout within the hour. It runs as soon as you ask for it.
Questions we hear the most
What is llms.txt?
A proposed plain text file placed at the root of a website that lists, in a fixed format, the pages a language model should read first. It was proposed by Jeremy Howard and published at llmstxt.org in September 2024.
Where does the file go?
At the root of the site, as /llms.txt. The proposal also allows one under a subpath, such as /docs/llms.txt, and says that where more than one applies, the most specific file should be used.
Do ChatGPT or Perplexity read llms.txt?
Not that either of them documents. Some AI companies publish an llms.txt for their own developer documentation, but we can find no published statement that their assistants read the file on other people sites.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may fetch and is a long-standing documented control. llms.txt is a proposed reading list, telling a model which pages are worth its attention. One restricts, the other suggests.
Will adding one hurt my site?
No. It is a small text file that no browser renders and no search engine penalizes. The realistic risk is cost, not harm: time and money spent on it instead of on the pages that carry your actual answers.
What should I do instead if I only have one hour?
Check that your robots.txt is not blocking assistant crawlers, then write one page that answers your buyers most common question properly. Both have documented effects, which the file does not yet.
Does AnswerHalo add an llms.txt for clients?
Only when the pages behind it already earn it, and never as a headline deliverable. We would rather spend the hour on the page a buyer question actually lands on, because that is the part with evidence behind it.
Can I run the free check today?
Yes. It runs as soon as you ask for it, and the readout lands within the hour between 7am and 5pm Central. Opening your own robots.txt in a browser and reading what it blocks takes about two minutes and can happen now.
Your buyers are already asking. Find out what AI tells them.
The free check asks ChatGPT 25 real buyer questions about your niche and sends you the report within the hour, at $0. If it shows you are already getting named everywhere, we will say so in plain words.
Get my free check →