What Is llms.txt? The Spec Explained (2026)

Part of our llms.txt guide.

llms.txt is a plain-text Markdown file placed at your site root that proposes a curated, LLM-friendly index of your most important pages. Jeremy Howard of Answer.AI proposed it in September 2024. It is a voluntary convention, not a standard, and no major AI engine has confirmed using it as a ranking or citation signal.

Where llms.txt came from

Jeremy Howard, co-founder of Answer.AI (and previously fast.ai), published the proposal on 3 September 2024. The problem he wanted to solve is real: language models have limited context windows, and a typical web page is a mess of navigation, ads, cookie banners, and JavaScript. Converting that HTML into clean text an LLM can reason over is lossy and expensive.

His answer was a file at https://example.com/llms.txt that hands the model a clean, curated map of your site in Markdown, so it does not have to reverse-engineer your HTML. The spec lives at llmstxt.org.

The format

The spec is deliberately tiny. A valid llms.txt has:

• A single H1 with the site or project name (the only required element)

• A blockquote with a short summary of what the site is

• Zero or more free-text paragraphs or lists giving context

• Zero or more H2 sections, each grouping curated links with a one-line description each

• An optional ## Optional section for links that can be skipped if the model is short on context

Here is a minimal example:

Acme is a payments API for marketplaces. These docs cover setup,

the REST reference, and webhooks.

Docs

• Quickstart: Get a live charge in 5 minutes

• API reference: Every endpoint and field

• Webhooks: Event types and retry logic

Optional

• Changelog: Release history

The convention encourages linking to clean .md versions of each page where possible, so a model that follows a link gets Markdown rather than HTML.

What llms.txt is not

It helps to be precise, because a lot of marketing content overstates this file.

That last row matters most. Because the site owner writes the file, engines treat its claims with suspicion. This is the same reason nobody trusts the keywords meta tag.

The honest adoption reality

Two things are true at once: some well-known companies publish an llms.txt, and no major engine has confirmed it changes anything.

On the publishing side, early adopters include Anthropic, Stripe, Cursor, Zapier, and documentation platforms like Mintlify and GitBook that generate the file automatically. A SE Ranking study of 300,000 domains found roughly a 10% adoption rate, so about one site in ten.

On the consumption side, the evidence is thin to negative:

• In July 2025, Google's Gary Illyes said Google does not support llms.txt and has no plans to. Google's John Mueller compared it to the keywords meta tag, noting that engines prefer to check the live site directly rather than trust what an owner claims.

• No major provider (OpenAI, Anthropic, Google, Meta, Mistral) has publicly committed to using llms.txt as a signal in production answers.

• A study across ~300,000 domains found no clear effect of having llms.txt on how often a domain is cited by LLMs.

• Crawler log analyses find that GPTBot, ClaudeBot, PerplexityBot and similar bots overwhelmingly fetch HTML directly and almost never request the llms.txt file.

So the file is cheap to create and low-risk, but treat any promise of a citation or ranking lift with skepticism. There is currently no engine confirmation and no measured effect.

Should you bother?

If you run documentation and already generate clean Markdown, the cost is near zero and the file is arguably good hygiene. If someone is selling it to you as an AI-visibility growth lever, that claim is not supported by evidence today. The things that reliably help AI systems cite you are the same fundamentals that help search: crawlable HTML, clear on-page answers, structured data, and content that genuinely answers the query.

FAQ