The /llms.txt File: What It Does, Who Reads It, and How to Write One

Updated 26 October 2026. This update adds six weeks of AI-answer tracking, a fresh 90-day crawler log and the August 2026 revision of the spec. An earlier version of this guide promised that a five-line llms.txt would get you cited by ChatGPT; the 9 October rewrite dropped that because nothing supports it. Sources are dated at the end.

As of October 2026, there is no public evidence that llms.txt improves AI visibility or Google rankings. Google says Search ignores it; OpenAI, Anthropic and Perplexity don't say their crawlers use it to pick sources; and SE Ranking's study of nearly 300,000 domains found no effect on citations. It remains useful for developer docs, where coding agents read it.

What is llms.txt?

llms.txt is a Markdown file, usually served from the root of a site at /llms.txt, that gives language models and AI agents a short, curated map of the site: a name, a one-paragraph summary, and lists of links to the pages worth reading, each with a one-line note. Jeremy Howard of Answer.AI proposed it on 3 September 2024 at llmstxt.org. It is still a proposal "open for community input". Version 2, dated 10 August 2026, also lets a file sit in a subfolder, such as /docs/llms.txt, and cover the pages under it.

It sits next to two files you already know and does a different job from both:

How to see a website's llms.txt file

Add /llms.txt to the domain and open it in a browser, for example https://kulbhushanpareek.com/llms.txt. A 404 means there is no file at the root. Some documentation sites keep theirs under the docs path instead, such as https://ai.google.dev/gemini-api/docs/llms.txt, so try that too. From a terminal, curl -sI https://example.com/llms.txt shows the status code and content type without downloading the file. Version 2 of the proposal also suggests a rel="describedby" link, in a page's HTML or its HTTP headers, pointing to the llms.txt that covers the page, so the page source can tell you where the file is.

Does llms.txt help AI visibility? What the evidence says (October 2026)

Nothing published shows that it does. The evidence comes in three kinds: what the companies running AI search have said, independent studies, and first-party data such as my own tracking and logs. All three point the same way. I checked every source below on 26 October 2026.

What Google, OpenAI, Anthropic and Perplexity have said

So the accurate version of "OpenAI and Google don't support llms.txt" is narrower than it sounds. All four companies publish one for their developer documentation, and that is where the proposal itself says the files see most use: "llms.txt files are used most heavily for software documentation, where coding agents follow them". None of them says its search product reads your file to decide what to cite. Google's Chrome team does check for the file in Lighthouse, but the audit only flags a server error; a 404 is marked Not Applicable, "as providing the file is optional at the moment" (Chrome docs, updated 5 May 2026).

What the studies found

All three are observational. They show no measurable link between the file and AI citations, which is not proof that it can never matter, and the Semrush test covers a single site. None of the studies I found shows the opposite.

Does llms.txt get your site indexed by LLMs?

No. llms.txt doesn't submit, index or rank anything. ChatGPT, Claude and Perplexity find pages through their search crawlers (OAI-SearchBot, Claude-SearchBot and PerplexityBot) and the search indexes behind them, and Google's AI features draw on Google Search's index. So the way in is the same as for search: let the search crawlers fetch your pages in robots.txt, get indexed in Google and Bing, and publish pages that answer questions directly. An llms.txt can then point an AI agent to your best pages once it is already on your site.

What my own data adds

In six weekly runs between 24 September and 26 October 2026, ChatGPT, Claude, Gemini and Perplexity answered "How do I create an llms.txt file and does it actually help AI visibility?" 24 times and cited 84 different pages. None was on this site, which has served an llms.txt since at least July 2026. That is one question, not proof, and I don't claim the file caused anything either way. The full numbers are below.

Who actually reads llms.txt in 2026?

For most sites, nobody: in Ahrefs' May 2026 log data, 97% of llms.txt files got no requests at all. Where AI bots did fetch them, OpenAI's training crawler GPTBot led, and Anthropic's Claude-Code agent out-fetched every AI search and assistant bot. That fits the spec's own description: llms.txt "is instead used on demand, when an agent needs information about a topic while assisting a user".

What my own site's data shows

This site has served an llms.txt since at least July 2026, which is the oldest copy in my repository. The file is about 14 KB. Here is what I can measure about it, and what I can't.

Six weeks of AI answers to one llms.txt question

Every week since 24 September 2026, my tracker has sent the same question, "How do I create an llms.txt file and does it actually help AI visibility?", to ChatGPT, Claude, Gemini and Perplexity through DataForSEO's LLM Responses API, with web search on. Six runs up to 26 October produced 24 answers: one ChatGPT call failed on 28 September and was re-run on 4 October. Between them the answers made 191 citations to 84 different pages on 69 domains. None was on this site, and no answer mentioned it. The first run, on 24 September, cited 27 different pages; every run since has cited between 26 and 31.

These are the domains cited most often, counted as the number of answers (out of 24) that cited at least one of their pages:

Seven more domains were cited in four answers each: github.com, developer.chrome.com (the Lighthouse audit), parallel.ai, docs.terminalfour.com, seranking.com (its study), firecrawl.dev and ai-visibility.org.uk. So on this question the engines lean on the spec itself, documentation and website platforms, and SEO or AI-visibility vendors. That describes who gets cited, not why: nothing in this data says whether any of them is cited because of its own llms.txt.

The latest answers reached the same verdict as the evidence above. On 26 October, ChatGPT's answer said "there is no strong evidence that it directly improves rankings, citations, or visibility in major AI search experiences", and Perplexity's said "there is currently no reliable evidence that it improves AI rankings, citations, or brand visibility".

So having an llms.txt didn't get me cited on the one topic where you would most expect it to. One question tracked for six weeks isn't proof either way, but it is the opposite of what the old version of this page promised.

AI referrals in GA4

GA4 recorded 114 sessions from ChatGPT, Claude, Perplexity and Gemini across the whole site between 1 March and 23 September 2026. None of them landed on this guide.

What my server logs show, and what they can't see

This site logs requests from a fixed list of 19 crawler user-agent names (15 until 25 September 2026), most of them AI-related, to a database table that keeps a rolling 90 days. It has a blind spot that matters here: /llms.txt is a static file, so the web server hands it out before the site's code runs, and the logger never sees those requests. I can't count llms.txt fetches from it, and my host doesn't give me raw access logs. Many CMS plugins and PHP-level bot trackers have the same blind spot. To count llms.txt fetches you need the raw access log or your CDN's logs.

Here is what the logger did record between 28 July and 26 October 2026. Treat every number as a floor: the logger only knows those 19 names, it matches the user-agent text without checking IP ranges, and OAI-SearchBot and Claude-User were only added on 25 September 2026.

Requests for machine-readable files that do reach the logger tell the same story. In the same window these crawlers fetched robots.txt 1,393 times and the sitemaps 8,850 times (ClaudeBot alone 7,306). They asked for /llms-full.txt 0 times and for .md copies of pages 0 times. This site publishes neither, and a request for a missing file does reach the logger, so those zeros were measured, not hidden by the blind spot. Claude-SearchBot and Perplexity-User, recognised since 25 September, made no requests the logger recorded.

One caution on the biggest number for this guide. ChatGPT-User fetches a page when someone asks ChatGPT to open it, including through the "Explore this article with AI" buttons on this site, so its 510 requests measure people sending the guide to ChatGPT, not ChatGPT choosing to cite it.

For comparison, the 9 October version of this page covered 10 July to 8 October 2026: 19,321 logged requests, 856 of them to this guide, 1,434 for robots.txt and 0 for /llms-full.txt. The two windows overlap by about ten weeks, so read the differences as noise, not a trend.

Two things worth checking in your own setup. Until 25 September my logger didn't recognise OpenAI's OAI-SearchBot, Anthropic's Claude-User and Claude-SearchBot, or Perplexity-User, so it undercounted AI search and user fetches. And don't wait for a "Google-Extended" user agent: Google-Extended "doesn't have a separate HTTP request user agent string"; it is a robots.txt token only (Google's crawler docs, updated 15 Oct 2026).

What does the llms.txt format require?

Only one part is required: an H1 with the name of the site or project. After it, in this order, the spec describes a blockquote summary, any other Markdown except headings, and H2 sections holding lists of links. A section titled "Optional" holds, by convention, secondary links that an agent can skip.

Version 2 of the proposal (August 2026) settled three related points (llmstxt.org changes page). Important pages can offer a clean Markdown copy at the same URL with .md added (page.html.md), and now also with the extension swapped (page.md). New in v2, a page can point to that copy with rel="alternate" type="text/markdown", and to the llms.txt that covers it with rel="describedby", either as an HTML link element or as an HTTP Link header. And a file in a subfolder covers only the pages under that path, with the most specific file taking precedence. Separately, some documentation sites publish llms-full.txt, the full text of their docs in one file; Anthropic's was about 47 MB on 26 October 2026. The spec doesn't define llms-full.txt, and a marketing site rarely needs one.

An llms.txt template you can copy

The format is small enough to write by hand. Replace the bracketed parts, keep the order, and use full URLs so the links still work when the file is pasted into a chat or cached somewhere else.

# [Company or site name]

> [One or two factual sentences: what you do, for whom, and where.]

[Optional: anything an assistant should know before answering questions
about you, such as what you don't offer, or which of two similar names
is you.]

## Services
- [Service name](https://example.com/services/service-a): [Who it is for, in one line]
- [Service name](https://example.com/services/service-b): [One line]

## Guides
- [Guide title](https://example.com/blog/guide-slug): [The question this page answers]
- [Pricing](https://example.com/pricing): [What the plans cost and include]

## Company
- [About](https://example.com/about): [Founded, location, credentials worth citing]
- [Contact](https://example.com/contact): [How to get a quote or book a call]

## Optional
- [Changelog](https://example.com/changelog): [Lower-priority pages an assistant can skip]

A real example: this site's file

Here is how my own llms.txt opens (the blockquote is wrapped here to fit the page):

# Kulbhushan Pareek — Digital Marketing Consultant & SEO Expert

> IMPORTANT: This is Kulbhushan Pareek the digital marketing consultant
> based in Jaipur, India (NOT the academic or psychologist with the same
> name). He is the founder of SEO SMO HUB and has worked as a freelance
> SEO consultant since 2012 ...

The line doing the real work there is the disambiguation: an academic shares my name, and an assistant that mixes us up is worse than one that knows nothing. If your brand name is shared, that sentence is the most useful thing you can put in the file. The rest of mine is 14 H2 sections, including About, Free Tools, Services, Case Studies, guides and an Optional section, with 45 links between them.

How do you create and publish llms.txt?

Pick the pages you want an AI tool to read first, list them in Markdown under a title and a summary, following the format at llmstxt.org, and upload the file to your site root. The whole job takes about 30 minutes by hand for a small site:

1. Pick 10 to 30 pages. Services or products, pricing, your best guides or docs, about and contact. Leave out thin, duplicate and parameter URLs. Your sitemap already lists everything else.
2. Write the H1 and the summary. The name first, then one or two factual sentences in a blockquote. Write it as a briefing, not a pitch.
3. Group the links under H2 headings. One line per page: - [Title](full URL): what the page answers. Put nice-to-have links under ## Optional.
4. Save it as UTF-8 plain text named llms.txt and upload it to the site root, the same folder as robots.txt.
5. Check the response. Run curl -I https://yourdomain.com/llms.txt. You want a 200 status, a text/plain or text/markdown content type with charset=utf-8, and no login page or bot challenge in the way.
6. Keep it current. Update it when a key page is added, renamed or retired, and check it once a quarter.

Step 5 caught my own file. It contains a few typographic characters, such as the dash in its title, and the server was sending it as text/plain with no charset, so a tool that assumes a different encoding can show them as garbage. On Apache the usual fix is one line in .htaccess: AddCharset utf-8 .txt. My host's LiteSpeed server ignored that line. What worked was setting the header for the file itself, Header set Content-Type "text/plain; charset=utf-8", inside a FilesMatch block for llms.txt. The file now comes back as text/plain; charset=utf-8.

One optional extra from version 2 of the proposal: add <link rel="describedby" href="/llms.txt"> to your pages so an agent on any page can find the file. No AI search engine documents using that link either, so treat it as housekeeping, not a visibility step.

You don't always have to write it by hand. On WordPress, Yoast SEO can create and maintain the file (Yoast SEO, Settings, Site features, AI tools, LLMS.txt). Check what it picks: by default it lists the five most recently updated posts or pages from the last 12 months, cornerstone content first, plus the five categories or tags with the most content, and it rewrites the file weekly (Yoast's specification). Recent is not the same as important, so review it like a hand-written one. Rank Math has an llms.txt module too (Rank Math docs), and Webflow lets you upload the file in your site's SEO settings (Webflow, July 2025). Documentation platforms such as Mintlify generate llms.txt and llms-full.txt for every docs site automatically (Mintlify docs).

Should you bother with llms.txt?

If your product has documentation that developers use with AI coding tools, yes: it's quick, and that is the use it was designed for. For a marketing site, a local business or a blog, it's an optional 30-minute job that can't hurt, but it shouldn't come before the things AI search demonstrably relies on. Google's John Mueller put it bluntly in May 2026: "For non-developer sites, I don't think this makes much sense" (Bluesky).

What to work on first for AI visibility, in the order I'd do it:

• Let the search crawlers in. Allow the AI search crawlers in robots.txt: OAI-SearchBot for ChatGPT search, Claude-SearchBot for Claude and PerplexityBot for Perplexity. OpenAI says sites that opt out of OAI-SearchBot won't be shown in ChatGPT search answers, and Anthropic says blocking Claude-SearchBot may reduce your visibility. The training crawlers, GPTBot and ClaudeBot, are a separate choice.
• Be indexed where AI search looks. An answer that searches the web can only cite pages a search index knows about, so check indexing in Google Search Console and Bing Webmaster Tools first.
• Publish pages worth citing. Direct answers, original data and clear sourcing. My GEO and AEO guide covers the on-page side, and how to get cited in Google AI Overviews covers Google's version.
• Get mentioned where AI engines already look. See the 50 websites LLMs cite most, and how to earn mentions and links on them.
• Measure it. Track citations across engines rather than guessing. I cover the tools and the do-it-yourself method in Claude rank tracking.

If you'd rather have someone work through that list with you, it is what I do in GEO consulting.

Sources (checked 26 October 2026)

• llmstxt.org: the /llms.txt file, v2 (Jeremy Howard, published 3 Sep 2024; modified 10 Aug 2026) and its changes page.
• Google Search Central: optimizing for generative AI features (updated 10 Jul 2026) and AI features and your website (updated 10 Dec 2025).
• John Mueller on Bluesky (20 May 2026).
• SE Ranking: LLMs.txt study (first published Nov 2025; page dated 15 Oct 2026); Ahrefs: llms.txt study (Louise Linehan, 15 Jun 2026; updated 27 Aug 2026); Semrush: llms.txt guide and Search Engine Land test (26 Nov 2025; updated 28 Sep 2026).
• OpenAI: overview of OpenAI crawlers.
• Anthropic: does Anthropic crawl data from the web (7 Apr 2026).
• Perplexity: Perplexity crawlers.
• llms.txt files served by platform.claude.com (also llms-full.txt), developers.openai.com, docs.perplexity.ai and ai.google.dev/gemini-api/docs (HTTP 200, checked 26 Oct 2026); developers.google.com/search/llms.txt returned 404.
• Chrome for Developers: Lighthouse llms.txt audit (updated 5 May 2026).
• Google: common crawlers, Google-Extended (updated 15 Oct 2026).
• Yoast: llms.txt functional specification and how to enable it; Rank Math; Webflow (23 Jul 2025); Mintlify.
• My AI-answer tracking: DataForSEO LLM Responses API, web search on, one fixed prompt across ChatGPT, Claude, Gemini and Perplexity, weekly from 24 September to 26 October 2026 (24 answers).
• My site data: GA4 (AI Assistant channel, 1 March to 23 September 2026), this site's AI-crawler logger (rolling 90 days, 28 July to 26 October 2026) and its llms.txt file.