The growth of ChatGPT, Google AI Overviews, AI Mode, Gemini, Claude, Perplexity, Copilot and AI agents has encouraged website owners to look for new ways to make their content easier for artificial intelligence systems to discover and understand.
One proposed solution is a file called llms.txt.
Supporters describe llms.txt as a machine-readable guide that can help large language models find a website’s most important content. Some WordPress plugins now generate it automatically, major developer documentation websites publish their own versions, and AI SEO discussions increasingly present it as a new website requirement.
However, the most important question is not whether llms.txt exists.
The real questions are:
- Does Google use
llms.txt? - Does it improve Google AI Overviews or AI Mode visibility?
- Does ChatGPT Search use it for citations?
- Is it a ranking factor?
- Can it replace an XML sitemap, structured data or robots.txt?
- Should every WordPress website create one?
- Does it offer measurable AI SEO value?
- Is it a genuine technical standard or still an experimental proposal?
The accurate answer is more nuanced than either “every website needs it” or “it is completely useless.”
llms.txt may be a useful optional navigation file for documentation websites, software products, knowledge bases and AI-agent workflows. However, it is not a confirmed Google ranking factor, it does not replace traditional SEO infrastructure, and Google has stated that it does not use llms.txt for Google Search or its generative AI features.
This guide explains what llms.txt is, what it can and cannot do, what major AI and search platforms officially say, how to implement it in WordPress, and whether it deserves a place in your AI SEO strategy.
The Quick Verdict on llms.txt
The most accurate way to describe llms.txt is as an optional, experimental machine-readable content index rather than an established SEO requirement.
The file can be useful when a specific AI tool, agent, documentation platform or retrieval system has been programmed to look for it. It may also provide a clean, human-curated list of a website’s most important resources.
However:
- Google Search does not use it.
- Google says it does not positively or negatively affect rankings.
- It is not required for Google AI Overviews or AI Mode.
- It is not a replacement for robots.txt.
- It is not a replacement for XML sitemaps.
- It is not a replacement for schema markup.
- It does not control AI crawler access.
- It does not guarantee citations in ChatGPT, Claude or Perplexity.
- It does not solve weak content, poor crawlability or low brand authority.
For most WordPress websites, llms.txt should be considered a low-risk optional enhancement after the important SEO work has already been completed.
Your priority should still be strong AI SEO, reliable Technical SEO, clear entities, quality content, internal linking, structured data, brand authority and relevant external mentions.
What Is llms.txt?
llms.txt is a proposed text file placed in the root directory of a website to provide large language models with a concise, structured overview of the website and links to its most important content.
The standard location is:
https://example.com/llms.txt
The original proposal describes it as a way to help language models use a website at inference time. Inference time refers to the stage when an AI model is processing a user’s request and generating an answer, rather than the earlier stage when the underlying model is being trained.
The proposal can be reviewed at:
A typical file may contain:
- The website or project name
- A short description
- Key documentation sections
- Important pages
- Product or service resources
- Guides and tutorials
- API references
- Optional secondary links
- Brief descriptions of each resource
The file normally uses Markdown-style formatting rather than XML, JSON or HTML.
Where Did the llms.txt Proposal Come From?
The llms.txt proposal was published by Jeremy Howard in September 2024 as an attempt to standardise how websites can present important information to large language models.
The proposal was motivated by a practical limitation: modern websites often contain navigation menus, scripts, advertising, interactive elements and complex HTML structures that can make it difficult for an LLM with a limited context window to identify the most useful information.
The proposed solution was a concise Markdown file containing:
- A required H1 heading
- An optional blockquote summary
- Optional explanatory text
- H2 sections
- Markdown lists containing links and descriptions
The proposal remains a community-driven standard rather than a universally adopted web protocol.
This distinction matters because a proposed convention does not automatically become a ranking signal or an officially supported feature across Google, OpenAI, Anthropic, Microsoft and Perplexity.
What Is the Intended Purpose of llms.txt?
The intended purpose of llms.txt is to help an LLM or AI application quickly identify the most useful parts of a website without processing every page and navigation element.
For example, a developer may ask an AI coding assistant:
How do I authenticate with this software platform’s API?
If the software documentation website provides a well-maintained llms.txt file, an AI tool designed to read that file could locate links to:
- Authentication documentation
- API references
- SDK instructions
- Error-code guides
- Code examples
- Security requirements
The file therefore acts more like a curated content directory than a traditional search engine ranking signal.
Its potential value depends on whether the AI system actually checks for the file and how it processes the listed resources.
What Does an llms.txt File Look Like?
A standard llms.txt file uses readable Markdown formatting.
A simple example could look like this:
# Example Company
> Example Company provides WordPress development, Technical SEO and AI search optimization services.
This file provides a curated list of the most useful public resources on this website.
## Main Resources
- [About Example Company](https://example.com/about/): Company background, experience and expertise.
- [Services](https://example.com/services/): Overview of professional services.
- [Technical SEO Guide](https://example.com/technical-seo/): Complete Technical SEO guidance.
- [AI SEO Guide](https://example.com/ai-seo/): Guidance for AI Overviews and answer engines.
## Case Studies
- [Local SEO Case Study](https://example.com/local-seo-case-study/): Local search growth campaign.
- [Website Speed Case Study](https://example.com/speed-case-study/): WordPress performance improvements.
## Optional
- [Contact](https://example.com/contact/): Contact and consultation information.
- [Privacy Policy](https://example.com/privacy-policy/): Website privacy information.
The file should be concise enough to act as an index rather than becoming a duplicate copy of the entire website.
What Is llms-full.txt?
llms-full.txt is a related concept intended to provide a more comprehensive Markdown version of a website’s documentation or content.
While llms.txt acts as a curated index, llms-full.txt may contain a combined version of many documentation pages in one large text file.
For example, OpenAI publishes curated llms.txt indexes and more extensive llms-full.txt resources for parts of its developer documentation.
OpenAI’s documentation index can be viewed here:
https://developers.openai.com/api/docs/llms.txt
A typical relationship is:
/llms.txt
This provides a concise list of resources.
/llms-full.txt
This provides a large combined body of documentation.
A full-file approach may be useful for:
- API documentation
- Software libraries
- Developer frameworks
- Technical standards
- Product manuals
- Knowledge bases
- Open-source projects
It is usually unnecessary for a normal local business, service website, portfolio or small blog.
Is llms.txt an Official Web Standard?
llms.txt is not currently equivalent to established web standards such as robots.txt, XML sitemaps, canonical tags or Schema.org structured data.
The file is a proposal with growing adoption across some documentation ecosystems, tools and WordPress plugins. It is not part of the Robots Exclusion Protocol, and there is no universal requirement that AI systems must retrieve or respect it.
This means individual platforms can:
- Support it
- Ignore it
- Use only selected parts
- Interpret it differently
- Stop supporting it
- Create their own alternative formats
Website owners should therefore avoid treating the file as a guaranteed cross-platform AI SEO mechanism.
llms.txt vs robots.txt
llms.txt and robots.txt have completely different purposes.
A robots.txt file communicates crawler-access instructions to compliant bots. An llms.txt file provides a suggested content index for AI applications.
The difference can be summarised as follows:
| File | Primary Purpose | Controls Crawling? | Supports Google Search? | Provides Content Links? |
|---|---|---|---|---|
robots.txt |
Tell crawlers which URLs they may request | Yes, as an advisory protocol | Yes | Not as its main purpose |
llms.txt |
Provide a curated guide to important content | No | Google says it does not use it | Yes |
| XML sitemap | List canonical URLs for search discovery | No direct access control | Yes | Yes |
| Schema markup | Describe page entities and meaning | No | Yes | Not as a website index |
Google explains that robots.txt is primarily used to tell search engine crawlers which URLs they can access. It is not a reliable privacy mechanism or a guaranteed method for removing a page from Google.
Official Google robots.txt guidance:
https://developers.google.com/search/docs/crawling-indexing/robots/intro
An llms.txt file cannot tell GPTBot, OAI-SearchBot, ClaudeBot or PerplexityBot to stop crawling your website.
Crawler permissions must still be managed through robots.txt, firewall rules, authentication and other access controls.
llms.txt vs XML Sitemaps
An XML sitemap helps search engines discover canonical website URLs and understand information such as modification dates, language versions and media relationships where supported.
An llms.txt file serves a different purpose.
An XML sitemap may contain hundreds or thousands of URLs, while an effective llms.txt file should usually contain a carefully selected collection of high-value resources.
For example:
XML sitemap:
2,500 post, page, product and taxonomy URLs
llms.txt:
15–30 strategically selected resources
WordPress includes core XML sitemap functionality, and major SEO plugins can provide additional sitemap control. Your XML sitemap remains an established part of Technical SEO.
An llms.txt file should not replace or duplicate it.
llms.txt vs Schema Markup
Schema markup provides structured information about entities and page content.
For example, schema can identify:
- An organisation
- A person
- An article
- A product
- A service
- A local business
- A review
- A breadcrumb trail
- A job posting
- A video
- An event
Google officially uses supported structured data to understand content and enable certain search-result features.
Official Google structured data documentation:
https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data
An llms.txt file does not provide the same entity-level relationships or page-specific structured information.
It cannot replace:
OrganizationschemaPersonschemaArticleschemaProductschemaServiceschemaLocalBusinessschemaBreadcrumbListschema
If you must choose between fixing inaccurate schema and adding llms.txt, fix the structured data first.
llms.txt vs Markdown Versions of Pages
Some documentation websites provide Markdown versions of their HTML pages.
For example:
https://example.com/documentation/authentication/
May also be available as:
https://example.com/documentation/authentication.md
Markdown can provide a cleaner textual version for AI applications, but Google states that website owners do not need to create special Markdown files to appear in Google Search or Google’s generative AI features.
Google’s AI optimisation guidance is available here:
https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
Creating Markdown duplicates of every WordPress page may also introduce:
- Maintenance work
- Conflicting updates
- Duplicate URLs
- Incorrect canonicalisation
- Outdated content
- Additional security concerns
- Unnecessary technical complexity
A normal WordPress website does not need Markdown copies of all its content merely for AI SEO.
Does Google Use llms.txt?
Google has stated that Google Search does not use llms.txt.
Google’s generative AI optimisation guidance says website owners do not need to create special machine-readable files, AI text files, Markdown versions or special AI markup to appear in Google Search, including its generative AI capabilities.
Google’s guidance can be reviewed here:
https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
Google has also clarified that maintaining an llms.txt file:
- Is not necessary for Google Search
- Does not positively affect visibility
- Does not negatively affect visibility
- Does not influence rankings
- May still be maintained for other services that use it
Google Search documentation updates are available here:
https://developers.google.com/search/updates
Google’s clarification removes much of the uncertainty around its SEO value.
Does llms.txt Help Google AI Overviews?
llms.txt does not provide a special route into Google AI Overviews.
Google handles AI Overviews and AI Mode as features within Google Search. A page generally needs to be:
- Crawlable by Googlebot
- Indexed
- Eligible to appear in Search
- Eligible to display a search snippet
- Relevant to the query
- Helpful and trustworthy
Google’s official AI feature guidance continues to emphasise established SEO practices rather than llms.txt.
Official guidance:
https://developers.google.com/search/docs/appearance/ai-features
The more important priorities include:
- Useful original content
- Correct robots.txt access
- Internal linking
- Indexability
- Page experience
- Accurate structured data
- High-quality text and visual content
- Strong entity and brand signals
For a broader AI search strategy, read my guide to AI SEO for Google AI Overviews, ChatGPT, Gemini and Perplexity.
Does llms.txt Improve Google Rankings?
Google says llms.txt does not positively or negatively affect visibility or rankings.
It should therefore not be presented as:
- A Google ranking factor
- A Technical SEO ranking requirement
- A replacement for crawlability
- A new type of Google schema
- A Google AI Overview optimisation file
- A method for recovering rankings
- A method for increasing PageRank
- A shortcut to topical authority
If a WordPress plugin says it can generate llms.txt, that feature may still be convenient. However, enabling the feature does not by itself improve Google rankings.
For ranking-drop diagnosis, use a structured process such as my Google Core Update Recovery Checklist rather than depending on experimental files.
Does ChatGPT Use llms.txt?
OpenAI publishes llms.txt and llms-full.txt resources for parts of its own developer documentation, demonstrating that the format can be useful for organising technical resources.
OpenAI’s documentation index:
https://developers.openai.com/api/docs/llms.txt
However, OpenAI’s official crawler documentation identifies robots.txt controls for:
OAI-SearchBotGPTBotChatGPT-User- Other documented OpenAI user agents
OpenAI crawler documentation:
https://developers.openai.com/api/docs/bots
The crawler documentation does not state that publishing an llms.txt file will improve a website’s ChatGPT Search ranking, citation frequency or recommendation visibility.
Therefore, the careful conclusion is:
- OpenAI uses
llms.txton its own documentation. - Some OpenAI-related tools may be capable of reading the format.
- There is no official guarantee that ChatGPT Search uses every website’s
llms.txt. - There is no official evidence that the file acts as a ChatGPT ranking factor.
- OAI-SearchBot access remains more important for ChatGPT Search discovery.
Allowing the correct crawler, publishing source-worthy content and building a trustworthy entity are more important than simply creating the file.
Does Perplexity Use llms.txt?
Perplexity publishes an llms.txt documentation index to help AI agents and developers navigate its technical documentation.
Its public crawler guidance, however, focuses on robots.txt controls for PerplexityBot and Perplexity-User.
Official Perplexity crawler guidance:
https://docs.perplexity.ai/docs/resources/perplexity-crawlers
The existence of Perplexity’s own documentation file does not automatically prove that an external website’s llms.txt will improve citations or ranking within Perplexity Search.
A realistic conclusion is that the file may be useful to Perplexity-related tools or agents capable of reading it, but it should not replace:
- Crawlability
- Accurate content
- Strong sourcing
- Clear authorship
- Relevant backlinks
- Entity signals
- Page-level optimisation
Does Claude Use llms.txt?
Anthropic’s published crawler guidance focuses on robots.txt instructions for ClaudeBot, Claude-SearchBot and Claude-User.
Anthropic states that website owners should use the relevant robots.txt directives to manage its documented bots.
Official Anthropic crawler guidance:
At present, website owners should not assume that creating llms.txt will automatically increase visibility in Claude Search.
For Claude-related discovery, the more established priorities are:
- Allowing Claude-SearchBot when visibility is desired
- Providing accessible public content
- Maintaining clear page structures
- Publishing authoritative information
- Protecting private content through authentication
Why Do OpenAI and Perplexity Publish llms.txt If It Is Not an SEO Standard?
The format is particularly useful for developer documentation.
A developer platform may contain hundreds of pages covering:
- Authentication
- API endpoints
- SDKs
- Error handling
- Rate limits
- Data formats
- Security
- Model references
- Integration guides
- Examples
A curated Markdown index can help coding assistants, agents and retrieval systems locate specific documentation more efficiently.
This does not mean that every local plumber, roofing contractor, SEO consultant or small eCommerce shop needs the same implementation.
The suitability of the file depends on the website’s content, audience and technical use cases.
Is llms.txt a Ranking Factor?
There is no confirmed evidence that llms.txt is a ranking factor in Google, ChatGPT Search, Perplexity, Claude or other major search and answer platforms.
Google has explicitly ruled out any positive or negative ranking effect within Google Search.
For other platforms, a lack of official confirmation means website owners should avoid making unsupported claims.
It is reasonable to say:
An AI tool designed to read llms.txt may use the file as a navigation aid.
It is not reasonable to say:
Adding llms.txt will make your website rank in ChatGPT.
The first statement describes a possible technical use. The second promises an unverified ranking outcome.
Why Has llms.txt Become Popular in AI SEO?
The format has become popular because it offers a simple response to a complicated problem.
Website owners want a direct way to tell AI systems:
- This is who we are.
- These are our most important pages.
- These are our best guides.
- These are our official products.
- These are our preferred sources.
- These are the pages you should cite.
The idea is attractive because it resembles familiar SEO files such as robots.txt and XML sitemaps.
However, modern search visibility depends on far more than submitting a list of preferred links.
AI systems and search engines may evaluate:
- Content relevance
- Source authority
- External corroboration
- Entity relationships
- Freshness
- Originality
- User intent
- Technical accessibility
- Reviews
- Brand mentions
- Links
- Structured data
- Page experience
A self-created list cannot independently prove that the listed pages are authoritative.
When llms.txt May Be Useful
llms.txt is most useful when a website has a clear reason to provide a curated machine-readable content directory.
Potential use cases include the following.
Developer Documentation Websites
Developer documentation is one of the strongest use cases because content is usually technical, structured and frequently accessed by AI coding assistants.
The file can link directly to:
- Installation guides
- API authentication
- Endpoint documentation
- SDK references
- Code examples
- Error messages
- Changelogs
- Migration guides
This helps reduce the chance that an AI tool retrieves an irrelevant marketing page when the user needs technical documentation.
Software and SaaS Products
Software companies may use llms.txt to direct agents toward official product information.
Useful links may include:
- Product documentation
- Security documentation
- Pricing
- Feature descriptions
- Terms of service
- Integration guides
- API references
- Support resources
The file may help distinguish official documentation from outdated forum discussions or third-party tutorials.
Large Knowledge Bases
A complex knowledge base may contain thousands of help articles.
A curated file can identify:
- Getting-started content
- Frequently used tutorials
- Troubleshooting guides
- Billing information
- Account settings
- Security resources
- Important policies
The file should not list every article. Its value comes from selection and organisation.
Agentic Websites and AI Applications
Websites designed for AI agents may provide llms.txt as one component of a broader machine-readable architecture.
However, agent readiness also depends on:
- Semantic HTML
- Accessible controls
- Clear forms
- Stable URLs
- Structured data
- Reliable APIs
- Authentication
- Safe action confirmation
- Well-defined permissions
Read my guide to Agentic SEO and preparing websites for AI agents for the wider technical framework.
Professional Portfolios With Deep Content Libraries
A professional portfolio containing detailed articles, services, case studies and specialist resources may use a concise file to present its strongest content.
For example, an SEO consultant could highlight:
- AI SEO guides
- Technical SEO resources
- Local SEO articles
- Case studies
- Service pages
- Professional background
- Contact information
The file may support AI tools that use the proposal, but it should still be treated as an optional index.
When llms.txt Is Probably Unnecessary
Many websites will receive little practical benefit from creating the file.
It may be unnecessary for:
- A five-page brochure website
- A local business with limited content
- A temporary landing page
- A website with no useful public resources
- A website that is not technically crawlable
- A website containing mostly duplicate content
- A website with unclear services
- A private membership portal
- A staging website
- A thin affiliate website
If a site has only Home, About, Services and Contact pages, an AI crawler can usually find them through normal navigation, internal links and the XML sitemap.
Creating another file may add maintenance without adding meaningful clarity.
Should Local Businesses Create llms.txt?
A local business can create the file, but it is unlikely to be a primary visibility factor.
Local businesses should first focus on:
- Google Business Profile optimisation
- Accurate name, address and phone information
- Local service pages
- Reviews
- Location relevance
- Local citations
- Service-area content
- Local schema
- Case studies
- Fast mobile performance
- Strong conversion paths
These elements are more important for Local SEO and AI recommendations than an experimental root-level file.
My Local SEO strategy for small businesses and guide to reviews and AI search visibility explain the higher-priority local signals.
Should eCommerce Websites Create llms.txt?
An eCommerce business may create a curated file, but it should not attempt to list every product.
A more useful eCommerce version could link to:
- Main product categories
- Best-selling collections
- Shipping policy
- Return policy
- Size guide
- Product documentation
- Brand pages
- Buying guides
- Customer support
- Business information
Product discovery still depends more heavily on:
- Crawlable category and product pages
- Product structured data
- Accurate pricing
- Availability
- Merchant feeds
- Reviews
- Canonicalisation
- Image quality
- Internal linking
- Page performance
An llms.txt file should not become another duplicate product feed.
Potential Benefits of llms.txt
The file may provide several practical benefits when it is implemented with a clear purpose.
Possible benefits include:
- A clean summary of the website
- A curated list of authoritative pages
- Easier navigation for compatible AI tools
- Reduced ambiguity around preferred resources
- A useful index for documentation agents
- A simplified reference for custom RAG systems
- Better organisation of technical resources
- A human-readable content map
- A central location for important URLs
- Low technical implementation cost
These benefits are operational rather than guaranteed ranking benefits.
Limitations of llms.txt
The file has significant limitations that should be understood before implementation.
Important limitations include:
- No universal platform support
- No confirmed ranking benefit
- No guaranteed AI citations
- No crawler-access control
- No privacy protection
- No replacement for structured data
- No replacement for internal links
- No replacement for XML sitemaps
- No standard analytics reporting
- No guarantee that listed content will be trusted
- No guarantee that AI systems will prefer listed pages
- No mechanism for validating claims
- Ongoing maintenance requirements
A file can describe a page as authoritative, but it cannot prove authority.
Risks of Poor llms.txt Implementation
A badly managed file can create confusion rather than value.
Potential risks include:
- Linking to outdated pages
- Listing redirected URLs
- Including 404 pages
- Exposing private resources
- Promoting low-quality content
- Creating inconsistent brand descriptions
- Duplicating an entire XML sitemap
- Publishing incorrect service information
- Linking to staging domains
- Including tracking parameters
- Forgetting to update changed URLs
- Generating bloated files
- Relying on automatic plugin selection without review
The file should never contain:
- Passwords
- API keys
- Private client documents
- Internal dashboards
- Customer data
- Unprotected reports
- Private downloads
- Confidential URLs
Anything listed in the file should be considered publicly discoverable.
Can llms.txt Control AI Training?
llms.txt is not an AI training opt-out mechanism.
To communicate crawler preferences, website owners must use the documented robots.txt controls provided by individual platforms.
Examples include:
GPTBotfor OpenAI model-development crawlingOAI-SearchBotfor ChatGPT Search discoveryClaudeBotfor Anthropic model-development crawlingClaude-SearchBotfor Claude searchPerplexityBotfor Perplexity searchGoogle-Extendedfor certain Gemini training and grounding uses
OpenAI states that OAI-SearchBot and GPTBot are independently controlled through robots.txt.
Do not use llms.txt to write instructions such as:
Do not train on this content.
There is no universal requirement that a crawler must treat that sentence as a valid opt-out.
For crawler configuration guidance, read AI Crawler Control for WordPress.
Can llms.txt Protect Private Content?
llms.txt cannot protect private content.
In fact, listing a private URL may make it easier to discover.
Private resources should be protected with:
- Authentication
- Password protection
- Role-based permissions
- Private storage
- Signed URLs
- Server controls
- Application-level authorisation
Robots.txt is also not a complete security tool. Google explains that robots.txt is designed primarily to control crawler access, not to keep sensitive pages private.
Does llms.txt Replace Internal Linking?
llms.txt does not replace internal linking.
Internal links help users and crawlers:
- Discover pages
- Understand topic relationships
- Navigate between resources
- Find supporting evidence
- Move from informational content to services
- Understand website hierarchy
- Distribute internal authority
An important article should still link naturally to related content, relevant services, case studies, author information and contact options.
For example, this article connects naturally with:
- AI SEO
- Entity SEO
- GEO vs AEO vs SEO
- Technical SEO for WordPress
- Brand authority for AI search
- Agentic SEO
A root-level file cannot provide the same page-level context.
Does llms.txt Replace Brand Authority?
A website cannot become a trusted AI source simply by describing itself as trusted inside its own file.
Brand authority is built through:
- Expert content
- Clear authorship
- Professional experience
- Case studies
- Reviews
- Backlinks
- Third-party mentions
- Consistent profiles
- Accurate entity information
- Social proof
- Original research
- Strong services
- Proven results
Read my guide to building brand authority for AI search for a detailed strategy.
Does llms.txt Fix Generic AI Content?
llms.txt cannot make weak content authoritative.
If the listed articles contain generic, repetitive or unverified AI-generated text, providing an index does not improve their actual quality.
Content should include:
- First-hand experience
- Original analysis
- Useful examples
- Specific recommendations
- Evidence
- Accurate references
- Transparent authorship
- Real business context
- Updated information
My guide explaining why generic AI content does not rank anymore covers the quality standards that matter more than experimental technical files.
How to Create llms.txt in WordPress
WordPress does not currently provide a core llms.txt feature equivalent to its built-in virtual robots.txt or XML sitemap functionality.
Website owners can create the file through several methods:
- A physical text file
- A supported SEO plugin
- A dedicated
llms.txtplugin - A custom WordPress endpoint
- A server or deployment process
Only one method should manage the final output.
Method 1: Create a Physical llms.txt File
A physical file is the simplest and most reliable implementation.
The process is:
- Create a plain-text file named
llms.txt. - Add the required Markdown content.
- Connect to the website through File Manager, FTP, SFTP or SSH.
- Open the WordPress root directory.
- Upload the file beside
wp-config.php. - Confirm that it loads at
/llms.txt. - Test every listed URL.
- Update the file when important content changes.
The final location should be:
https://yourdomain.com/llms.txt
The file should normally return:
HTTP 200 OK
Content-Type: text/plain
A physical file is suitable when:
- The website has a stable resource list
- Updates are infrequent
- The owner can access hosting files
- No plugin-generated file is active
- Version control or deployment manages the website
Method 2: Generate llms.txt With Yoast SEO
Yoast SEO includes an llms.txt implementation that can create the file in the website root.
Yoast’s developer documentation states that its feature can:
- Create an
llms.txtfile - Place it in the root directory
- Update it through scheduled processing
- Select important pages
- Generate descriptions automatically
Yoast also advises removing or disabling conflicting physical files when its generated file is being used.
Official Yoast documentation:
https://developer.yoast.com/features/llms-txt/functional-specification/
After enabling any plugin feature:
- Open
/llms.txt. - Review every selected page.
- Remove low-value URLs.
- Check descriptions.
- Confirm that no private content appears.
- Test again after plugin updates.
Do not assume automatic selection is strategically correct.
Method 3: Use a Dedicated WordPress Plugin
Some WordPress plugins are designed specifically to generate llms.txt or AI-readable documentation.
Before installing one, evaluate:
- Update history
- Active support
- Code quality
- Security
- File ownership
- Caching compatibility
- URL selection control
- Multisite support
- Multilingual support
- Ability to disable automatic generation
- Uninstallation behaviour
Avoid installing a large plugin only to create a small text file.
A physical file or lightweight custom implementation may be more appropriate.
Method 4: Create a Dynamic WordPress Endpoint
A custom WordPress endpoint can serve llms.txt dynamically without requiring a physical file.
This is useful when:
- Content links need central management
- The site uses automated deployment
- The file should be generated from WordPress
- The owner wants to avoid another plugin
- The website has no conflicting physical file
The following snippet creates a custom /llms.txt response.
/**
* MHR Custom llms.txt Endpoint for WordPress
*
* Creates:
* https://yourdomain.com/llms.txt
*
* Add through:
* - A custom functionality plugin
* - A trusted code-snippet plugin
* - A child theme's functions.php file
*
* Do not use this snippet when:
* - A physical llms.txt file already exists
* - Yoast or another plugin is already generating llms.txt
*/
add_action( 'template_redirect', 'mhr_output_custom_llms_txt', 0 );
function mhr_output_custom_llms_txt() {
if ( empty( $_SERVER['REQUEST_URI'] ) ) {
return;
}
$request_uri = wp_unslash( $_SERVER['REQUEST_URI'] );
$request_path = trim(
(string) wp_parse_url( $request_uri, PHP_URL_PATH ),
'/'
);
$home_path = trim(
(string) wp_parse_url( home_url( '/' ), PHP_URL_PATH ),
'/'
);
$expected_path = trim(
$home_path . '/llms.txt',
'/'
);
if ( $request_path !== $expected_path ) {
return;
}
/*
* Do not publish the custom file when WordPress is configured
* to discourage search engines.
*/
if ( '1' !== get_option( 'blog_public' ) ) {
status_header( 404 );
exit;
}
status_header( 200 );
header(
'Content-Type: text/plain; charset=' .
get_option( 'blog_charset' )
);
/*
* Optional:
* Prevent the text file itself from appearing as a Google result
* without blocking crawlers from requesting it.
*/
header( 'X-Robots-Tag: noindex, follow', true );
$lines = array(
'# MD Harunur Rashid',
'',
'> SEO, AI SEO & Digital Growth Consultant with 17+ years of experience helping businesses improve organic visibility, AI search readiness, WordPress performance and digital growth.',
'',
'This file provides a curated list of the most important public resources on mhrmasum.info.',
'',
'## Professional Information',
'',
'- [About MD Harunur Rashid](https://mhrmasum.info/about/): Professional background, experience and areas of expertise.',
'- [SEO and Digital Growth Services](https://mhrmasum.info/services/): SEO, AI SEO, Local SEO, WordPress and digital growth services.',
'- [Portfolio](https://mhrmasum.info/portfolio/): Selected SEO, Local SEO, Technical SEO and website optimisation work.',
'',
'## AI SEO Resources',
'',
'- [AI SEO Guide](https://mhrmasum.info/ai-seo-rank-google-ai-overviews-chatgpt-gemini-perplexity/): How to improve visibility in Google AI Overviews, ChatGPT, Gemini and Perplexity.',
'- [Agentic SEO](https://mhrmasum.info/agentic-seo-preparing-websites-for-ai-agents/): How to prepare websites for AI agents.',
'- [Entity SEO](https://mhrmasum.info/entity-seo-the-missing-part-of-modern-seo/): How search engines and AI systems understand entities.',
'- [GEO vs AEO vs SEO](https://mhrmasum.info/geo-vs-aeo-vs-seo/): Differences between modern search optimisation disciplines.',
'- [Brand Authority for AI Search](https://mhrmasum.info/how-to-build-brand-authority-for-ai-search/): How to build trust and recognition across AI platforms.',
'',
'## SEO and WordPress Resources',
'',
'- [Technical SEO Checklist for WordPress](https://mhrmasum.info/technical-seo-checklist-for-wordpress-websites/): Crawlability, indexing, speed, schema and WordPress SEO guidance.',
'- [Google Core Update Recovery Checklist](https://mhrmasum.info/google-core-update-recovery-checklist/): How to diagnose and recover from ranking losses.',
'- [Local SEO Strategy](https://mhrmasum.info/local-seo-in-2026-complete-guide-for-small-businesses/): Complete Local SEO guidance for small businesses.',
'- [Why Generic AI Content Does Not Rank](https://mhrmasum.info/why-generic-ai-content-does-not-rank-anymore/): How to improve originality, usefulness and content quality.',
'- [Reviews and AI Search Visibility](https://mhrmasum.info/how-reviews-help-local-seo-and-ai-search-visibility/): How reviews support Local SEO, entities and AI discovery.',
'',
'## Optional',
'',
'- [Technical SEO and Keyword Research Portfolio](https://mhrmasum.info/portfolio/technical-seo-and-keyword-research/): Technical SEO and keyword research project information.',
'- [Website Speed Optimisation Portfolio](https://mhrmasum.info/portfolio/website-speed-optimization/): WordPress performance and speed optimisation work.',
'- [Local SEO Portfolio](https://mhrmasum.info/portfolio/local-seo/): Local SEO project examples.',
'- [Contact MD Harunur Rashid](https://mhrmasum.info/contact/): Consultation and project enquiry information.',
'',
);
echo implode( "\n", $lines ); // phpcs:ignore WordPress.Security.EscapeOutput.OutputNotEscaped
exit;
}
The snippet uses WordPress’s template_redirect hook, which runs before WordPress selects the page template.
Official WordPress hook documentation:
https://developer.wordpress.org/reference/hooks/template_redirect/
Google supports the X-Robots-Tag HTTP response header for non-HTML resources, so the optional header can prevent the file itself from appearing in Google results without blocking access to it.
Official Google X-Robots-Tag documentation:
https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag
Important Instructions for the WordPress Snippet
The dynamic endpoint must be tested carefully after activation.
Complete these checks:
- Open
https://yourdomain.com/llms.txt. - Confirm that plain text appears.
- Confirm that no theme header or footer appears.
- Confirm that the response is not a 404.
- Confirm that the website does not redirect it to the homepage.
- Clear WordPress and CDN caches.
- Test every listed link.
- Check that another plugin is not also generating the file.
- Verify the output after changing themes.
- Keep the snippet documented.
Do not run several llms.txt generators simultaneously.
Recommended llms.txt for mhrmasum.info
For mhrmasum.info, a concise, curated file can be justified because the website contains a growing collection of detailed AI SEO, Technical SEO, Local SEO, Entity SEO and digital growth resources.
The file should highlight the strongest subject clusters rather than every post.
A recommended manual version is:
# MD Harunur Rashid
> SEO, AI SEO & Digital Growth Consultant with 17+ years of experience helping local businesses, eCommerce brands, agencies and website owners improve organic visibility and digital growth.
This file provides a curated index of the most important public resources on mhrmasum.info.
## Professional Information
- [About MD Harunur Rashid](https://mhrmasum.info/about/): Professional experience, expertise and consultant profile.
- [SEO and Digital Growth Services](https://mhrmasum.info/services/): SEO, AI SEO, Local SEO, WordPress optimisation and digital growth services.
- [Portfolio](https://mhrmasum.info/portfolio/): Selected SEO, WordPress, Local SEO and Technical SEO projects.
## AI SEO and Modern Search
- [AI SEO Guide](https://mhrmasum.info/ai-seo-rank-google-ai-overviews-chatgpt-gemini-perplexity/): Guide to Google AI Overviews, ChatGPT, Gemini and Perplexity visibility.
- [Agentic SEO](https://mhrmasum.info/agentic-seo-preparing-websites-for-ai-agents/): Preparing website architecture, content and interactions for AI agents.
- [Entity SEO](https://mhrmasum.info/entity-seo-the-missing-part-of-modern-seo/): Building clear entity relationships for search and AI systems.
- [GEO vs AEO vs SEO](https://mhrmasum.info/geo-vs-aeo-vs-seo/): Differences between Generative Engine Optimization, Answer Engine Optimization and SEO.
- [Brand Authority for AI Search](https://mhrmasum.info/how-to-build-brand-authority-for-ai-search/): Building trust, mentions and authority for AI discovery.
## Technical SEO and Content Quality
- [WordPress Technical SEO Checklist](https://mhrmasum.info/technical-seo-checklist-for-wordpress-websites/): Technical SEO, crawlability, indexing, performance and schema guidance.
- [Google Core Update Recovery Checklist](https://mhrmasum.info/google-core-update-recovery-checklist/): Diagnosing and recovering from ranking losses.
- [Why Generic AI Content Does Not Rank](https://mhrmasum.info/why-generic-ai-content-does-not-rank-anymore/): Improving originality, usefulness, expertise and content differentiation.
- [Local SEO Strategy](https://mhrmasum.info/local-seo-in-2026-complete-guide-for-small-businesses/): Local search strategy for small businesses.
- [Reviews and AI Search Visibility](https://mhrmasum.info/how-reviews-help-local-seo-and-ai-search-visibility/): How reputation signals support local and AI discovery.
## Case Studies and Portfolio
- [Technical SEO and Keyword Research](https://mhrmasum.info/portfolio/technical-seo-and-keyword-research/): Technical SEO and keyword research project work.
- [Website Speed Optimisation](https://mhrmasum.info/portfolio/website-speed-optimization/): WordPress website speed and performance optimisation.
- [Local SEO Portfolio](https://mhrmasum.info/portfolio/local-seo/): Local search optimisation examples.
- [Roofing Local SEO Case Study](https://mhrmasum.info/portfolio/roofing-brisbane-local-seo/): Local SEO work for a roofing business.
- [Pest Control GMB Case Study](https://mhrmasum.info/portfolio/pest-control-melbourne-gmb/): Google Business Profile optimisation case study.
## Optional
- [Contact MD Harunur Rashid](https://mhrmasum.info/contact/): Contact and consultation information.
This version is focused enough to be useful without becoming another full sitemap.
How Many URLs Should llms.txt Contain?
There is no universally enforced maximum, but the file should remain selective.
A practical starting point is:
- Small business website: 5–15 links
- Professional portfolio: 10–25 links
- SaaS website: 15–40 links
- Documentation website: 20–100 carefully grouped links
- Large knowledge base: High-level sections rather than every article
The correct number depends on whether every link helps an AI system understand the website’s purpose and strongest resources.
Do not include 5,000 URLs simply because they exist.
What Pages Should Be Included?
Include pages that clearly represent the website’s expertise, products, services or official documentation.
Strong candidates include:
- About page
- Main services
- Core product documentation
- Authoritative guides
- Key category hubs
- Original research
- Important policies
- Case studies
- Contact information
- Technical references
- Support resources
Each link should include a useful description.
A weak entry is:
- [SEO](https://example.com/seo/)
A stronger entry is:
- [Technical SEO Guide](https://example.com/technical-seo/): A complete guide to crawlability, indexing, structured data, website speed and technical optimisation.
Descriptions help a compatible system understand why the link matters.
What Pages Should Be Excluded?
Exclude pages that do not help explain the website or provide useful information.
Pages to exclude may include:
- WordPress login URLs
- Admin pages
- Cart pages
- Checkout pages
- Internal search results
- Tag archives
- Thin author archives
- Duplicate pagination
- Tracking URLs
- Redirected URLs
- Staging URLs
- Test pages
- Private reports
- Customer accounts
- Expired offers
- Low-value attachment pages
Only canonical public URLs should be included.
Should llms.txt Be Added to the XML Sitemap?
There is usually no need to add llms.txt to the XML sitemap.
An XML sitemap is designed primarily to help search engines discover URLs that you want them to crawl and potentially index. Google has stated that llms.txt is not needed for Search and has no positive or negative ranking effect.
Adding it to the XML sitemap may also encourage search engines to treat it as another indexable URL rather than a specialised resource.
Keep the two systems separate.
Should llms.txt Be Linked in the Website Footer?
A footer link is not required.
Compatible tools should know to check the conventional root location:
/llms.txt
A footer link may be appropriate for:
- Developer documentation
- Open-source projects
- AI-agent documentation
- Transparency pages
For a normal service website or portfolio, the file can remain accessible at the root without being added to the main navigation or footer.
Should llms.txt Be Noindexed?
Noindexing the file is optional.
Google may discover and index text files, but Google has said that the existence of llms.txt has no special Search value.
If you do not want the file itself appearing in search results, return:
X-Robots-Tag: noindex
Google supports X-Robots-Tag for non-HTML resources.
Do not block the file in robots.txt if your objective is to make it retrievable by compatible AI tools that respect robots.txt.
How Often Should llms.txt Be Updated?
The file should be updated whenever its important information changes.
Review it after:
- Publishing a major guide
- Changing a service URL
- Rebranding
- Changing the company description
- Adding a core product
- Removing documentation
- Migrating the website
- Redirecting listed pages
- Updating the site structure
- Launching a new content cluster
A monthly or quarterly review is usually sufficient for a portfolio or business website.
A software documentation site may require automated weekly or release-based updates.
How to Test llms.txt
Testing should confirm that the file is technically accessible and strategically accurate.
Open:
https://yourdomain.com/llms.txt
Then check:
- The response returns
200 OK - The content type is plain text
- The file is readable
- The H1 title is correct
- The description is current
- Every URL uses HTTPS
- Every link returns successfully
- No links redirect unnecessarily
- No staging domains appear
- No private URLs appear
- Markdown formatting is valid
- The file is not duplicated by another plugin
- The content is not excessively large
You can also inspect the HTTP response with:
curl -I https://example.com/llms.txt
A suitable response may look like:
HTTP/2 200
content-type: text/plain; charset=UTF-8
x-robots-tag: noindex, follow
How to Check Whether AI Bots Request llms.txt
Server logs or Cloudflare logs can show whether automated systems request the file.
For an Apache or Nginx access log, use:
grep -i 'llms.txt' access.log
To count requests:
grep -i 'llms.txt' access.log | wc -l
To inspect user agents requesting it:
grep -i 'llms.txt' access.log | awk -F'"' '{print $6}' | sort | uniq -c | sort -nr
A request proves that a bot or user retrieved the file.
It does not prove that:
- The content influenced a generated answer
- Your website received a citation
- Rankings improved
- The AI system trusted the file
- The listed pages were processed
- The file created a conversion
Log analysis should therefore be interpreted carefully.
Can GA4 Track llms.txt Requests?
GA4 will not normally track direct requests to a plain-text llms.txt file because the file does not contain the Google Analytics JavaScript tag.
Use:
- Server logs
- Cloudflare logs
- CDN analytics
- Web Application Firewall logs
- Hosting analytics
GA4 can still track users who later click through to normal website pages, provided referral and campaign information is available.
For AI referral measurement, read How to Track ChatGPT Search Traffic in GA4.
How to Measure the Value of llms.txt
Measuring direct value is difficult because major platforms do not provide a standard llms.txt performance report.
A practical evaluation may include:
- Number of file requests
- User agents requesting the file
- Changes in AI referral traffic
- Changes in AI citations
- Landing pages receiving AI traffic
- Brand mentions in AI answers
- Lead or revenue growth from AI referrals
- Technical support usage
- Agent retrieval success
However, correlation does not prove causation.
If ChatGPT referrals increase after publishing llms.txt, the increase may also result from:
- New content
- Stronger brand authority
- More backlinks
- Increased search demand
- OAI-SearchBot crawling
- Improved internal linking
- New platform features
- Better content freshness
Do not attribute every AI visibility change to one file.
Common llms.txt Myths
Several unsupported claims have emerged around the file.
Understanding these myths can prevent wasted effort and misleading SEO promises.
Myth 1: Every Website Needs llms.txt
Not every website needs it.
A small website with a clear structure may gain little from an additional machine-readable index.
Myth 2: llms.txt Is the AI Version of robots.txt
The files have different purposes.
Robots.txt manages crawler-access preferences. llms.txt provides a proposed content guide.
Myth 3: llms.txt Improves Google AI Overview Rankings
Google states that it does not use the file and that it has no positive or negative effect on visibility or rankings.
Myth 4: llms.txt Guarantees ChatGPT Citations
OpenAI provides no guarantee that publishing the file will produce ChatGPT Search citations.
Crawler access, content quality, relevance and authority remain more important.
Myth 5: llms.txt Replaces XML Sitemaps
XML sitemaps remain an established discovery mechanism for search engines.
The files serve different purposes.
Myth 6: llms.txt Replaces Schema
Structured data provides page- and entity-level meaning that a simple Markdown link list cannot replace.
Myth 7: llms.txt Can Block AI Training
Training crawler controls must be implemented through documented robots.txt directives or enforceable access controls.
Myth 8: More URLs Make the File Better
The file should be curated.
Adding every URL removes the main advantage of having a concise guide.
Myth 9: Automatically Generated Files Need No Review
Automatic selection can include low-value, redirected or irrelevant pages.
Human review remains necessary.
Myth 10: llms.txt Can Fix Weak Content
A directory cannot improve the expertise, originality or usefulness of the pages it lists.
Higher-Priority AI SEO Work Than llms.txt
Before creating llms.txt, make sure the website has completed the more important work.
Priority areas include:
- Crawlability
- Indexability
- Mobile usability
- Website speed
- Helpful original content
- Strong internal linking
- Accurate schema
- Entity clarity
- Brand authority
- Reviews and reputation
- Author information
- Case studies
- External mentions
- AI crawler access
- Conversion tracking
These improvements benefit users, Google Search and AI platforms whether or not they support llms.txt.
Recommended Strategy for mhrmasum.info
For mhrmasum.info, creating a small manually reviewed llms.txt file is reasonable because the site has a growing specialist content library and clear professional expertise.
However, it should be treated as a supporting experiment rather than a major ranking initiative.
The recommended approach is:
- Publish a concise file with 15–25 important links.
- Focus on AI SEO, Agentic SEO, Entity SEO, Technical SEO and case studies.
- Include clear descriptions for every link.
- Keep OAI-SearchBot, Claude-SearchBot and PerplexityBot accessible when AI discovery is desired.
- Review the file monthly.
- Check server logs for requests.
- Monitor AI referral traffic.
- Do not promise ranking improvements.
- Continue strengthening content and brand authority.
- Keep the XML sitemap, schema and internal linking unchanged.
This balanced approach allows experimentation without presenting the file as a proven ranking factor.
A 30-Day llms.txt Implementation Plan
A structured implementation process can prevent duplication, broken links and unnecessary plugin conflicts.
Days 1–7: Audit the Website
During the first week, identify which pages genuinely represent the website’s strongest resources.
Complete these tasks:
- Review the post sitemap
- Review the page sitemap
- Identify core topic clusters
- Check canonical URLs
- Remove outdated pages
- Review redirects
- Review private content
- Check robots.txt
- Review AI crawler access
- Identify the active SEO plugin
Days 8–14: Build the File
During the second week, create a concise and strategically organised draft.
Complete these tasks:
- Write the H1 title
- Add a short brand summary
- Create logical H2 sections
- Select important pages
- Write clear descriptions
- Remove tracking parameters
- Exclude low-value archives
- Validate every URL
- Choose physical, plugin or custom delivery
Days 15–21: Publish and Test
During the third week, publish the file and check the technical response.
Complete these tasks:
- Upload or activate the file
- Open
/llms.txt - Confirm
200 OK - Confirm plain-text output
- Test on mobile and desktop
- Clear caches
- Check CDN behaviour
- Confirm no duplicate generator exists
- Test all links
- Optionally add an X-Robots-Tag header
Days 22–30: Monitor and Document
During the final week, begin collecting evidence without assuming that the file has created a ranking benefit.
Complete these tasks:
- Check server logs
- Record user agents
- Monitor AI referral traffic
- Test important brand prompts
- Check broken links
- Document the implementation date
- Record the original file
- Schedule a monthly review
- Compare results after 90 days
Monthly llms.txt Audit Checklist
The file should remain accurate if you choose to maintain it.
Use this checklist:
- The file returns
200 OK - The file uses plain-text output
- The website name is correct
- The summary is accurate
- All links use HTTPS
- All links are canonical
- No URL returns 404
- No unnecessary redirect exists
- No staging URL appears
- No private URL appears
- Descriptions remain accurate
- New priority resources are considered
- Outdated resources are removed
- The file remains concise
- Only one system generates the file
- Server-log requests are reviewed
- AI referral traffic is monitored
- No ranking claims are made without evidence
Frequently Asked Questions About llms.txt
The following questions address the most common concerns about llms.txt, WordPress implementation and AI SEO.
What Is llms.txt?
llms.txt is a proposed root-level Markdown file that provides a concise description of a website and a curated list of important resources for compatible large language models and AI applications.
Where Should llms.txt Be Located?
The conventional location is:
https://example.com/llms.txt
It should be placed at the root of the relevant domain or subdomain.
Does Google Use llms.txt?
No. Google states that Google Search does not use llms.txt and that website owners do not need it for conventional or generative Google Search features.
Does llms.txt Affect Google Rankings?
No. Google has clarified that the file does not positively or negatively affect search visibility or rankings.
Does llms.txt Help Google AI Overviews?
Google does not use it as a special AI Overview optimisation mechanism.
AI Overview eligibility continues to depend on established Search requirements such as crawlability, indexability, snippet eligibility, relevance and content quality.
Does ChatGPT Use llms.txt?
OpenAI publishes llms.txt resources for its own developer documentation, but its official crawler guidance does not guarantee that publishing the file improves ChatGPT Search rankings or citations.
Does Perplexity Use llms.txt?
Perplexity publishes an llms.txt index for its own developer documentation. However, its crawler-management guidance for external websites focuses on PerplexityBot and Perplexity-User robots.txt controls.
Is llms.txt the Same as robots.txt?
No.
Robots.txt communicates crawler-access preferences. llms.txt provides a proposed list of important content.
Can llms.txt Block GPTBot?
No. GPTBot must be controlled through robots.txt or an enforceable firewall rule.
Can llms.txt Replace an XML Sitemap?
No. XML sitemaps remain an established search-engine discovery mechanism.
Can llms.txt Replace Schema Markup?
No. Schema provides structured page and entity information that llms.txt cannot replace.
Does WordPress Create llms.txt Automatically?
WordPress core does not currently provide a standard built-in llms.txt feature.
Some SEO plugins and dedicated plugins can create it.
Can Yoast SEO Create llms.txt?
Yes. Yoast documents a feature that creates the file in the website root and updates it through scheduled processing.
Should Every WordPress Website Enable It?
No. The decision should depend on the website’s content, technical use case and maintenance capacity.
A small website may receive little practical value.
Should I Include Every WordPress Post?
No. Include only the strongest and most useful resources.
The file should not become a duplicate XML sitemap.
Should I Create llms-full.txt?
Most business and portfolio websites do not need it.
A combined full-text file is more appropriate for extensive developer documentation, technical manuals and software knowledge bases.
Should llms.txt Be Added to robots.txt?
There is no established requirement to declare llms.txt inside robots.txt.
The conventional root location should be sufficient for systems designed to request it.
Should I Noindex llms.txt?
Noindexing is optional.
Use an X-Robots-Tag: noindex response header when you want the file to remain retrievable but do not want it appearing as a Google search result.
How Often Should I Update It?
Review it monthly or quarterly and whenever important URLs, services, documentation or brand information change.
Can llms.txt Improve AI Referral Traffic?
It may help compatible systems navigate important resources, but there is no guarantee that it will generate additional referrals.
Monitor server logs, AI citations, landing pages and conversions before drawing conclusions.
Is llms.txt an AI SEO Myth?
It is a myth when it is promoted as a universal ranking file, guaranteed ChatGPT citation method or replacement for SEO.
It can be useful when it is presented accurately as an optional curated index for compatible AI applications.
Final llms.txt Decision Framework
Use the following framework before deciding whether to create the file.
Create It When
Creating the file may be reasonable when:
- Your website has extensive documentation
- You manage a large knowledge base
- You operate a software or SaaS platform
- You want to support AI-agent workflows
- You have a strong curated content library
- You can maintain the file
- You understand that results are not guaranteed
Delay It When
Delay the file when:
- Important pages are not indexed
- The website has broken links
- Technical SEO is weak
- Content quality is poor
- Schema is inaccurate
- The site structure is confusing
- AI crawlers are accidentally blocked
- The site has no clear brand entity
- Conversion tracking is missing
Skip It When
Skipping the file may be sensible when:
- The website has only a few simple pages
- There is no documentation or knowledge base
- Nobody can maintain it
- It would contain mostly duplicate links
- The website is private
- The proposed content contains sensitive information
- It would distract from higher-priority SEO work
Final Verdict: Useful File or AI SEO Myth?
llms.txt is neither a revolutionary AI SEO ranking method nor a completely meaningless file.
It is a useful proposal for specific situations, particularly:
- Developer documentation
- SaaS platforms
- Knowledge bases
- Technical products
- AI-agent workflows
- Large specialist content libraries
However, it becomes an AI SEO myth when people claim that it:
- Improves Google rankings
- Is required for AI Overviews
- Guarantees ChatGPT citations
- Replaces robots.txt
- Replaces XML sitemaps
- Replaces schema
- Blocks model training
- Fixes generic content
- Creates authority automatically
Google has clearly stated that it does not use llms.txt for Google Search and that the file has no positive or negative ranking impact.
The best strategy is to treat it as an optional experiment.
Create it when your website has enough high-quality resources to justify a curated index. Keep it concise, accurate and public. Monitor requests and AI referrals. Do not present it as a guaranteed ranking factor.
Most importantly, do not allow llms.txt to distract you from the work that actually builds sustainable search and AI visibility:
- Helpful original content
- Technical accessibility
- Strong entities
- Clear authorship
- Brand authority
- Accurate schema
- Internal linking
- Reviews
- Case studies
- External mentions
- Fast WordPress performance
- Proper AI crawler configuration
For support with AI SEO, WordPress Technical SEO, AI crawler control, structured data, content systems and digital growth strategy, explore my SEO and digital growth services or contact MD Harunur Rashid for a detailed consultation.