You have probably seen llms.txt mentioned in an SEO newsletter and wondered if you are behind. A file that gets your site into ChatGPT? Sounds essential. Here is the honest position. In May 2026, Ahrefs looked at 137,000 domains and found 97 percent of llms.txt files received zero requests, with only 28 percent of sites publishing one. So before you add anything, it helps to know what llms.txt actually does, what it does not, and whether it earns a slot on your to-do list.
This guide keeps it plain. No hype, no jargon dressed up as strategy.
Key Takeaways
- llms.txt is a proposed standard, not an official one. No search engine or AI company is required to read it, and most do not.
- It is not a ranking factor. Google has confirmed on the record that it does not use llms.txt for Search or its AI features.
- It is not robots.txt. One curates content for AI to read; the other controls what crawlers can access. They are different jobs.
- A few systems do read it. Perplexity, Claude and coding agents like Cursor and GitHub Copilot retrieve llms.txt today.
- The cost to add one is low. Roughly half a day, no ongoing upkeep, no penalty risk.
- Verdict: a cheap bet, not a priority. Add it if you value being agent-ready. Do not expect traffic or rankings from it.
What is llms.txt in plain English?
llms.txt is a plain text file placed at the root of your website that gives AI models a clean, curated map of your most important content. Think of it as a hand-picked menu for AI, stripped of ads, navigation and clutter, pointing to the pages you most want understood. It was proposed by Australian technologist Jeremy Howard in 2024.
The file usually lists your key pages with short descriptions, written in simple Markdown. The idea is sound: large language models work with limited context and cannot ingest your whole site in one go. A tidy index could help them find and represent your business accurately.
That is the theory. Whether anyone reads it is a separate question, and we will get to it.
llms.txt vs robots.txt: what is the difference?
llms.txt and robots.txt do opposite jobs. robots.txt tells crawlers which parts of your site they may or may not access. llms.txt does not block anything; it points AI models toward your best content and offers a simplified version of it. One is a locked door with rules. The other is a welcome mat with directions.
The confusion is understandable. Both are plain text files, both live at your site root, and the names rhyme. But they are not interchangeable.
Here is the practical distinction.
| Feature | robots.txt | llms.txt |
|---|---|---|
| Primary job | Controls crawler access | Curates content for AI to read |
| Status | Official, widely respected | Proposed, patchy adoption |
| Blocks bots? | Yes | No |
| Affects Google ranking? | No (access only) | No |
| Read by major AI systems? | Yes | Rarely |
| Format | Directives | Markdown links and notes |
One point worth stressing: llms.txt does not stop AI from training on your content. If a page is public, models can still reach it during inference. Content protection is a robots.txt and opt-out job, not an llms.txt one.
Does Google use llms.txt?
No. Google does not use llms.txt for Search or its AI features, and it has said so directly. Google’s AI optimisation guidance, updated on 15 June 2026, states that you do not need to create machine-readable files, AI text files or Markdown to appear in Google Search, including its generative capabilities, because Search does not use them.
Google’s John Mueller has compared llms.txt to the old keywords meta tag: a self-declared signal that site owners could game, which search engines eventually ignored. He also pointed out that when llms.txt appeared on some Google-owned properties, it was because an internal content system added it, not because Search consumes it.
So if your only reason for adding the file is to rank better on Google, save yourself the effort. It will not move the needle.
Will llms.txt get my site into ChatGPT?
Not on its own. There is no evidence that adding llms.txt increases how often AI assistants cite your site. SE Ranking tested this and found no correlation between llms.txt and AI citations across their sampled data. The bots that actually drive AI visibility, such as GPTBot, rarely request the file at all.
The nuance is that some systems do read it. Reporting indicates Perplexity and Anthropic’s Claude retrieve llms.txt, and coding agents like Cursor and GitHub Copilot rely on it to navigate documentation. Companies including Stripe and Cloudflare ship one.
But there is a gap between “some agents read it” and “it will win you customers.” Right now, for a typical UK SME, the traffic case is thin. If you want to understand how AI search visibility actually works, our guide to generative engine optimisation covers the levers that do move.
How do I add llms.txt in WordPress?
Adding llms.txt in WordPress takes about five minutes. The simplest route is a plugin: Yoast SEO can generate the file automatically with one click, and previews it before publishing. If you prefer manual control, you can create the file yourself and upload it to your site root.
Here is the checklist.
- Option A, plugin route. Install or open Yoast SEO. Find the llms.txt feature. Click generate, review the pages it selected, and publish.
- Option B, manual route. Create a plain text file named
llms.txt. - List your priority pages in Markdown, each with a short description. Homepage, services, key blog posts.
- Upload it to your root so it sits at
yourdomain.co.uk/llms.txt. - Check it loads in a browser at that address.
- Leave it. There is no ongoing maintenance unless your key pages change.
That is genuinely it. The work is picking which pages matter, not the technical setup.
So does your website need one in 2026?
Probably not urgently, but it will not hurt. llms.txt is emerging convention, not confirmed benefit. It is not a ranking factor, most AI systems ignore it, and no study shows it lifts traffic. What it offers is cheap agent-readiness: if AI agents grow in importance and start relying on the file, you are already set up.
Frame it as insurance, not strategy. If you have a spare half-day and you like being early, add one. If your time is tight, your effort is far better spent on content quality, clear structure and the fundamentals that AI search actually rewards. Our guide to ranking in Google AI Overviews is a more productive place to start.
The honest summary: llms.txt is a low-cost, low-return bet. Add it if you want. Do not lose sleep if you do not.
Frequently Asked Questions
Will llms.txt get me into ChatGPT?
Not by itself. There is no evidence that llms.txt increases AI citations, and the crawlers that drive AI visibility rarely request the file. Some systems, including Perplexity and Claude, do read it, but adding the file will not reliably get your business mentioned in AI answers. Strong, well-structured content matters far more.
Does Google use llms.txt?
No. Google confirmed in its June 2026 guidance that Search, including AI Overviews and AI Mode, does not use llms.txt. Google staff have compared it to the discredited keywords meta tag. If your goal is Google ranking, the file will do nothing for you. It is neither a help nor a penalty.
Is llms.txt the same as robots.txt?
No. robots.txt controls which crawlers can access parts of your site. llms.txt does not block anything; it curates and points AI models to your best content. They live in the same place and look similar, but they do opposite jobs. You may want both, for different reasons.
Can I set up llms.txt in five minutes?
Yes. In WordPress, Yoast SEO generates the file in one click. Manually, you create a plain text file, list your key pages in Markdown, and upload it to your site root. The technical part is quick. The only real thinking is choosing which pages deserve a spot on the list.
Does llms.txt stop AI from training on my content?
No. llms.txt is not a blocking tool. If a page is public, AI models can still access it during inference. Controlling training use is a job for robots.txt and specific opt-out signals, not llms.txt. Do not rely on the file to protect proprietary content.
How many sites actually use llms.txt?
Adoption is low. In May 2026, Ahrefs found only 28 percent of the domains it analysed published an llms.txt file, and 97 percent of those files received zero requests. That is a long way from the near-universal use of robots.txt and sitemaps, which tells you how experimental the standard still is.
Not sure which AI search tactics are worth your time?
The AI search space is full of shiny distractions. We cut through it. Lobo Digital blends human strategy with AI execution to get UK businesses found and recommended across Google and AI search, not just chasing every new file format that trends for a fortnight.
Book your Free Marketing Review and we will tell you, honestly, what will actually move your leads and revenue.
