llms.txt Best Practices, Straight From the Spec
- The spec asks for one H1, an optional blockquote summary, and H2 sections holding lists of links.
- Give every link a short description after a colon. A bare URL list parses but tells a reader nothing.
- Use absolute URLs so the file resolves outside your own site's context.
- The section titled Optional is reserved by the spec. Keep the name, keep it last.
- None of this makes a model read the file. It makes the file correct. Those are different things.
The only llms.txt best practices worth following are the ones written into the spec at llmstxt.org, plus a couple of plain readability choices. Everything else circulating on the topic is unverifiable advice about rankings. This post sticks to what the spec states and to what a person reading your file can actually confirm.
The file is small and the rules are few
llms.txt is a markdown file served at /llms.txt. It has four parts. A single H1. An optional blockquote summary. One or more H2 sections, each holding a markdown list of links. And a reserved section called Optional, if you want one.
That is the whole spec. No schema, no version field, no metadata block. Any tool asking you to add YAML front matter is inventing requirements you now have to maintain.
Use one H1, with your name on it
One H1 at the top. The spec calls it the name of the project or site. Not a tagline, not a keyword string, not a sentence. A second H1 makes the file ambiguous about what it describes, both for a parser and for a person reading from the top.
# Acme Analytics
That is the whole line. Resist adding a subtitle. The blockquote exists for that.
The blockquote is your one-line summary
Directly under the H1, a blockquote holds a short summary of the project.
> A hosted analytics tool that counts human visitors and AI crawler visits in one dashboard.
One or two sentences. The spec places it before the detailed information. It is the only spot in the file where prose belongs. Skip it and the file still parses, but a reader gets a title and a link list with no framing around it.
Every link line gets its own description
A link line is a markdown link, a colon, then a short description.
- [Docs](https://acme.com/docs): Setup guides, API reference, the event model.
The description does real work. It is the only signal a reader gets about whether a link is worth following. "Blog" says nothing. "Notes on failed pricing experiments, updated monthly" says something. This is the biggest quality gap between files that get used and files that get skimmed, and it has nothing to do with rankings.
| Spec element | Write this | Not this |
|---|---|---|
| H1 | One line, the site or project name | A tagline, a keyword, or two H1s |
| Blockquote | One or two sentences of summary | Marketing copy |
| Link line | [Title](URL): description | A bare list of URLs |
| URLs | Absolute, with https | Relative paths |
| Optional section | Last, with that exact title | "Extras", "Misc", "Other" |
Absolute URLs, for the same reason sitemaps use them
Write https://acme.com/pricing, not /pricing. A relative link only resolves when the reader knows your base URL. The file gets read out of context, pasted into a chat window, parsed by a tool that has no idea which site it came from. Absolute URLs remove the guess.
The Optional section is reserved and it goes last
The spec reserves one section name: Optional. Links there are ones a reader can skip. It sits at the end. The name is not yours to replace. Calling it "Extras" or "More" breaks the one convention the spec actually defines.
Use it for things that are real but not central. An archive. A changelog. A second product. The point is to keep the main sections usable without deleting anything.
The spec says nothing about rankings, because it cannot
Here is the honest limit. Following every rule above makes your file correct. It guarantees nothing about whether a model reads it. Adoption is partial. No major AI operator has publicly committed to honouring llms.txt, and neither OpenAI, Anthropic nor Google documents it as a signal their crawlers follow. Some documentation platforms and developer tools do read it. That is the current state.
Treat it as a cheap bet. Writing the file costs a few minutes. Do it. Do not confuse publishing with being read. Publishing is you putting a file on a server. Being read is a crawler requesting it, and only your server logs see the second event.
What to measure instead of guessing
Server logs are the only ground truth. A request line for /llms.txt means something fetched it. No request line means nothing did, however well written the file is.
The agents that show up are not all one shape. The AI crawler directory splits them into live fetchers, indexers and training crawlers, because a live fetch for one person's question and a bulk training crawl mean very different things for your bandwidth and your content planning. A live fetcher like ChatGPT-User (OpenAI's live fetcher) requests a page because someone asked for it right now. A training crawler like Bytespider (ByteDance) does not, and it sends no traffic back.
A crawler hit is not a visit. What matters is whether any of it turns into a person landing on your site. AI referrals arrive as ordinary sessions, so ordinary analytics can read them: which page, which country, how long the person stayed. Before you compare anything, know what normal looks like on your pages. Our note on Bounce Rate for Small Business Sites: What's Normal is a fine starting point. For tagging any campaigns you run alongside this work, UTM Parameters: The 10-Minute Guide covers the naming rules without ceremony.
Comparing tools gets messy when one counts visitors with cookies and another counts them some other way. Kymo counts with a salted server-side hash that rotates at UTC midnight, which the docs on Cookieless identity explain in full.
For the wider picture of how AI answers change what you publish in the first place, The Small Business Guide to AEO is the right next read. The crawler side of that picture gets fresh coverage over in the AI crawlers section of the blog.
See which crawlers actually show up
Kymo puts the crawler side and the human side in one dashboard: who fetched, which pages they took, and what traffic came back. Start tracking it and you will know within a week whether llms.txt is even being read. If you want to understand how each agent is classified first, read the documentation.
Start free → 14-day free trial. No card required.