llms.txt has moved from a speculative SEO talking point to something slightly more interesting.
It is now used across thousands of sites, documentation platforms generate it automatically, and companies including OpenAI and Anthropic publish llms.txt files for their own developer documentation. The proposal itself reached version 2 in August 2026. llms-txt
But that does not prove the claim many marketers actually care about:
Add an llms.txt file and AI assistants will cite, rank or recommend your business more often.
There is still no published evidence supporting that as a general GEO ranking tactic.
Google has gone further. In June 2026 it clarified that Google Search, including its generative AI features, does not use llms.txt, and that maintaining one has neither a positive nor negative effect on Google Search visibility or rankings. Google for Developers
So is llms.txt useless?
No.
Is it a proven AI-search optimisation lever?
Also no.
The honest answer sits between the two.
Table of Contents
- What is llms.txt?
- What changed in 2026?
- Who is actually using it?
- Does Google use llms.txt?
- Does ChatGPT read llms.txt?
- What about Claude, Gemini and Perplexity?
- What llms.txt can genuinely do
- What it cannot currently prove
- Is llms.txt the same as robots.txt?
- How to create one in ten minutes
- Where I would spend effort first
- Frequently Asked Questions
What is llms.txt?
llms.txt is a proposed convention for giving AI agents a cleaner guide to a website.
The idea was originally published by Jeremy Howard in September 2024.
Rather than asking an agent to interpret an entire website full of:
- navigation
- scripts
- adverts
- interface elements
- repeated templates
the site can expose a relatively small Markdown-formatted file that provides a summary and links to useful content.
The official proposal describes it as an /llms.txt file placed at a site’s root, or potentially within a subpath, to guide agents towards useful LLM-friendly resources. llms-txt
A very simple example might look like:
# Example Facilities
> UK commercial facilities management company serving multi-site businesses.
## Services
– [Planned Maintenance](https://example.co.uk/planned-maintenance/): Planned maintenance services for commercial estates.
– [Reactive Maintenance](https://example.co.uk/reactive-maintenance/): Emergency and reactive maintenance support.
## Company
– [About Us](https://example.co.uk/about/): Company background and capabilities.
– [Case Studies](https://example.co.uk/case-studies/): Examples of completed client work.
It is essentially a curated machine-readable orientation layer.
But that description is important:
orientation layer
not:
AI ranking file
What changed in 2026?
Quite a lot.
The original proposal was updated to version 2 on 10 August 2026.
The revised proposal says adoption has grown to thousands of sites and that documentation platforms now generate llms.txt automatically. It also describes the file being used particularly heavily for software documentation and agent workflows. llms-txt
Version 2 also introduced clearer ways for agents to discover the files.
For example, a page can advertise:
<link rel=”describedby” href=”/llms.txt”>
and a Markdown version of a page can be identified using:
<link
rel=”alternate”
type=”text/markdown”
href=”/example-page.md”
>
These changes are useful because one weakness in the original idea was obvious:
How does an agent know the file exists unless it guesses /llms.txt?
The v2 proposal attempts to solve that with standard link relationships. llms-txt
That makes the proposal more mature.
It still does not make it an SEO ranking factor.
Who is actually using it?
This is where the story becomes more nuanced than it was in 2025.
OpenAI publishes llms.txt
OpenAI currently publishes a developer-level llms.txt index covering documentation for its API, Codex, plugins, developer resources and other products. OpenAI Developers
That is meaningful adoption of the format.
But it does not establish that ChatGPT Search automatically checks third-party /llms.txt files before answering commercial queries.
Those are two separate claims.
Anthropic publishes llms.txt
Anthropic also exposes an llms.txt resource for its developer documentation, containing links across its Claude developer platform. Claude Platform Docs
Again, this demonstrates that the format itself has practical value for machine-friendly documentation.
It does not prove a Claude search-ranking advantage for websites that publish one.
Google Gemini tooling can use one
Google provides an even more concrete example in its Gemini developer documentation.
Its coding-assistant guidance says that if the Gemini documentation MCP service is unavailable, the agent can fetch the Gemini API documentation’s llms.txt file as a fallback. Google AI
That is actual documented consumption of an llms.txt resource in a specific agent workflow.
But note the scope:
Gemini developer documentation retrieval
not:
Google Search ranking
Those are very different things.
Microsoft documents the same pattern
Microsoft’s Teams SDK documentation provides llms.txt resources specifically so developers can point coding assistants towards relevant technical documentation. Microsoft Learn
That further supports the idea that llms.txt has found a real use case.
That use case is strongest when an agent needs to navigate documentation or a defined information set.
Does Google use llms.txt?
For Google Search, we now have a particularly clear answer.
No.
Google’s current generative AI optimisation guidance specifically lists llms.txt among things website owners do not need to create for Google Search.
Google says its Search systems, including generative AI features, do not use these files in a special way. Google for Developers
In June 2026 Google made the position even more explicit.
It clarified that llms.txt files are not needed for Google Search and will not positively or negatively affect visibility or rankings. Google for Developers
That means you should not sell llms.txt as:
A way to rank better in Google’s AI Overviews.
Google has directly contradicted that claim.
Does ChatGPT read llms.txt?
There are two different questions hidden inside this.
Can OpenAI systems access an llms.txt file?
Clearly the format is not foreign to OpenAI. OpenAI itself publishes these files for its developer documentation. OpenAI Developers
Has OpenAI said ChatGPT Search relies on third-party llms.txt files?
I could not find first-party OpenAI guidance establishing that.
OpenAI’s current publisher guidance instead tells website owners who want to appear in ChatGPT Search to make sure OAI-SearchBot is not blocked.
Its crawler documentation separates OAI-SearchBot, which is used for Search, from GPTBot, which relates to potential training use. OpenAI Help Center
That is the control OpenAI actually documents for website discovery.
So the defensible statement is:
OpenAI publishes llms.txt for its own documentation, but there is no published evidence that adding one improves your visibility in ChatGPT Search.
For crawler controls, internally link to our AI crawlers and robots.txt guide.
What about Claude, Gemini and Perplexity?
The same caution applies.
Anthropic publishes an llms.txt for its own developer documentation. That shows support for machine-readable documentation, but does not establish a general Claude recommendation benefit for third-party websites. Claude Platform Docs
Google’s Gemini developer workflow provides a documented use case for consuming an llms.txt resource, but Google Search explicitly does not use it for rankings or visibility. Google AI
For Perplexity, I did not find equivalent first-party documentation confirming that publishing llms.txt improves visibility in Perplexity Search.
So do not turn absence of documentation into either:
Perplexity definitely ignores it.
or:
Perplexity definitely uses it.
The answer is simply:
not confirmed from the available first-party guidance.
What llms.txt can genuinely do
There is still a useful idea underneath the hype.
An llms.txt file can provide a concise, structured directory to content you consider important.
For an agent explicitly designed to look for or consume it, the file can help answer:
- What is this website?
- Which documentation matters?
- Where are the key resources?
- Which page should I fetch next?
- Is there a cleaner Markdown version?
That makes particular sense for:
Technical documentation
This is where current adoption appears strongest.
Large knowledge bases
A curated index may be easier for compatible agents to navigate than hundreds of competing pages.
Developer resources
OpenAI, Anthropic, Google and Microsoft examples all point towards this use case.
Sites where generation is automatic
If your CMS or plugin can maintain an accurate file at almost no cost, the risk is relatively small.
That is different from spending a week hand-building one because somebody called it “the new sitemap for AI”.
Is llms.txt the same as robots.txt?
No.
They solve completely different problems.
robots.txt
Tells automated crawlers which parts of a site they are permitted or not permitted to crawl.
For example:
User-agent: OAI-SearchBot
Allow: /
llms.txt
Provides a curated guide and links intended to help compatible AI agents navigate useful information.
The current proposal explicitly positions llms.txt as something that can coexist with robots.txt, rather than replace it. llms-txt
So:
robots.txt = access instructions
while:
llms.txt = optional information guide
Never use llms.txt when what you actually need is crawler access control.
How to create llms.txt in ten minutes
If you want one, keep it simple.
Step 1: Create the file
Create:
llms.txt
at the root of the website so it resolves at:
/llms.txt
Step 2: Add the site name
The current proposal requires an H1.
# Example Ltd
Step 3: Add a concise summary
For example:
> UK B2B facilities management provider serving multi-site commercial organisations.
Step 4: Add your most useful sections
## Services
– [Facilities Management](https://example.co.uk/facilities-management/): Integrated facilities management services.
– [Reactive Maintenance](https://example.co.uk/reactive-maintenance/): Reactive maintenance support.
## Company
– [About](https://example.co.uk/about/): Company background and capabilities.
– [Case Studies](https://example.co.uk/case-studies/): Examples of client work.
The proposal supports an H1, optional summary/context and H2 sections containing linked resources. llms-txt
Do not turn it into a keyword dump.
Write for clarity.
WordPress users may not need to create it manually
If the site runs Yoast SEO, this may take less than ten minutes.
Yoast currently offers an llms.txt generator and lets users enable and customise the file from its settings. Its documentation says the generated file lives at the site root and is refreshed automatically. Yoast
That changes the cost-benefit calculation considerably.
If it takes:
two minutes to enable
and:
five minutes to verify
I have little objection to testing it.
If an agency proposes:
a £2,000 bespoke “LLM file strategy”
for a 20-page brochure website, I would have questions.
Where I would spend effort first
For a normal UK service business, llms.txt would not be my first GEO task.
I would check:
1. Crawler access
Can the systems you actually want to reach your site access it?
For ChatGPT Search, OpenAI specifically documents OAI-SearchBot. OpenAI Developers
2. Important content in the HTML
Do not hide the strongest service information behind:
- PDFs
- client-side interfaces
- inaccessible widgets
- images containing text
3. Clear service positioning
Can a machine and a buyer understand:
- what you do
- who it is for
- where you operate
- what makes the offer relevant
without piecing together six pages?
4. Entity clarity
Keep:
- business name
- legal identity
- address
- profiles
- service descriptions
consistent.
5. Evidence
Build:
- useful case studies
- genuine reviews where relevant
- accreditations
- external profiles
- earned coverage
6. Then add llms.txt
Particularly if your CMS can create and maintain it automatically.
This is the broader principle behind Generative Engine Optimisation
Prioritise confirmed technical and content fundamentals before speculative additions.
Frequently Asked Questions About llms.txt
Is llms.txt a ranking factor?
There is no evidence that llms.txt is a general ranking factor.
For Google specifically, the answer is even clearer: Google says llms.txt has no positive or negative effect on Google Search
visibility or rankings.
Does ChatGPT read llms.txt?
OpenAI publishes llms.txt resources for its own developer
documentation, but OpenAI has not documented llms.txt as a
requirement or ranking mechanism for ChatGPT Search.
Its publisher guidance instead tells website owners to allow OAI-SearchBot if they want eligible content to be surfaced in ChatGPT Search.
OpenAI: Developer llms.txtIs llms.txt the same as robots.txt?
No.
robots.txt communicates crawler access rules.
llms.txt is a proposed guide containing site context and links
intended for compatible agents.
One does not replace the other.
llms.txt proposalIs llms.txt an official web standard?
Not in the same sense as a W3C or IETF standard.
It remains a community proposal, although adoption has expanded significantly and version 2 was released in August 2026.
llms.txt specificationShould I bother adding one?
If your CMS generates an accurate file automatically or you can create one in ten minutes, it is reasonable to experiment.
If implementing it requires substantial budget or development work, I would prioritise confirmed crawlability, content, technical SEO and entity issues first.
Can it hurt SEO?
Google says maintaining an llms.txt file does not positively
or negatively affect visibility or ranking in Google Search.
As with any public file, make sure the information is accurate and do not publish private or sensitive information inside it.
Does Google Gemini use llms.txt?
This needs a precise answer.
Google Search does not use llms.txt for visibility or rankings.
Separately, Google’s Gemini API developer documentation describes an agent
workflow that can fetch its own documentation’s llms.txt file
as a fallback.
Those are different products and different use cases.
So, does llms.txt actually do anything?
In 2026:
Yes, in some contexts.
It is being used as a machine-friendly documentation and navigation format.
Major technology companies publish these files.
Some developer and coding-agent workflows explicitly consume them.
The specification has matured and adoption is growing. llms-txt
But:
No, we still do not have evidence that simply adding /llms.txt makes a normal commercial website rank, get cited or get recommended more often by major AI search products.
Google has explicitly said it will not help Google Search.
OpenAI’s documented website discovery control remains OAI-SearchBot rather than an llms.txt requirement.
For the other major answer engines, avoid filling the evidence gap with assumptions.
That leaves a fairly simple recommendation:
If it is quick and inexpensive, add a clean one and keep it accurate. Do not mistake it for an AI visibility strategy.
Your real priorities remain:
- accessible content
- crawler access
- strong service information
- entity clarity
- useful evidence
- technically sound pages
- third-party authority
Then experiment with emerging formats such as llms.txt without pretending we know more than we do.
Get your free AI visibility audit
Generative Engine Optimisation
Learn more about Which AI Crawlers Should You Allow in robots.txt?
