llms.txt in 2026: Hype vs. Reality (and Why You Should Still Ship One)

Let's get the awkward part out of the way: HejGeo offers a free llms.txt generator. We'd love to tell you this file is the secret lever of AI visibility. The data says it isn't. We're going to show you that data anyway, because the case for shipping an llms.txt file survives honesty — it just doesn't survive hype.

Here is the short version. As of June 2026, no major AI vendor has committed to consuming llms.txt. Crawler logs show AI bots almost never request it. And yet it costs roughly nothing to ship, carries zero risk, and buys you optionality in a market that reorganizes itself every quarter. That's the whole argument. The rest of this post is the evidence.

What llms.txt actually is

llms.txt is a proposed standard, published in September 2024 by Jeremy Howard, co-founder of Answer.AI. The idea: place a Markdown file at the root of your domain (https://example.com/llms.txt) that gives language models a curated, clutter-free map of your site — what it is, what matters, and where to find the canonical version of each thing.

The format is deliberately simple:

• An H1 with the site or project name
• A blockquote with a one-paragraph summary
• H2 sections containing lists of links, each with a short description
• An optional Optional section for content that can be skipped when context is tight

There's also a heavier sibling, llms-full.txt, which inlines the actual content of key pages into a single Markdown file instead of linking out to them.

The reasoning is sound. HTML pages are full of navigation, cookie banners, JavaScript, and ads. Context windows are finite. A file that says "here's the clean, structured essence of this site" solves a real problem — if anyone on the consuming side agrees to read it.

Who adopted it

Adoption on the publishing side was real and fast, at least in one niche: developer documentation. The inflection point came in November 2024, when Mintlify rolled out llms.txt across every docs site it hosts. Overnight, thousands of documentation sites — including Anthropic's and Cursor's — were serving the file. Cloudflare, Vercel, and much of the modern dev-tools ecosystem followed; Stripe and Zapier are reportedly among the adopters as well.

Outside of docs, adoption is broader than skeptics admit but thinner than advocates claim. SE Ranking found that 10.13% of a roughly 300,000-domain sample had an llms.txt file — a sample figure, not a global adoption rate. One in ten domains in that sample is not nothing. It's also not a standard.

Notice what every name on the adopter list has in common, though: they're all publishers of llms.txt. The list of confirmed consumers — AI systems that officially fetch and use the file — is empty. Even Anthropic, which publishes an llms.txt for its own docs, has never committed to reading anyone else's.

The honest data

This is where most llms.txt content gets vague. Let's not be vague.

Limy.ai's 90-day monitoring study tracked around 500 million AI-bot visits across the sites it observes. Requests for llms.txt: 408. Not 408 thousand. Four hundred and eight, total — a rounding error of a rounding error.

Otterly.AI's log analysis found the same pattern at smaller scale: out of 62,100 AI bot requests, exactly 84 touched llms.txt. That's 0.1%.

Google said the quiet part out loud. John Mueller's verdict: "none of the AI services have said they're using llms.txt — and you can tell when you look at your server logs that they don't even check for it." He compared the file to the keywords meta tag — a self-declared claim about your own content, which is precisely the kind of signal search engines learned to ignore twenty years ago. Gary Illyes has confirmed that no Google AI system uses the file.

No vendor commitment exists. OpenAI, Anthropic, Google, Meta, Perplexity — none has announced support. As of June 2026, that hasn't changed.

Why no major vendor consumes it

The non-adoption isn't laziness. There are three structural reasons.

The trust problem. llms.txt is what a site owner claims their site contains. Unverified self-declaration is a spam vector: nothing stops you from stuffing the file with content that doesn't match your actual pages. Any AI system using it would need to cross-check the file against the real HTML — at which point it has already crawled the real HTML and the file is redundant. That's Mueller's core critique, and nobody has a good rebuttal.

The redundancy problem. Modern AI crawlers parse rendered pages, structured data, and sitemaps well. The hard part of search was never "find the content" — it's ranking and trusting it. llms.txt helps with the solved problem, not the unsolved one.

The incentive problem. A standard works when consumers commit first (robots.txt worked because crawlers honored it before most sites had one). Here, publishers adopted en masse while consumers stayed silent. A million publishers can't will a standard into existence if nobody on the other side reads the file.

So why ship one anyway?

Because the cost-benefit math is lopsided in the boring direction.

Cost is effectively zero. Generating an llms.txt takes minutes if your sitemap is in order. There is no downside risk: it can't hurt rankings, can't confuse crawlers, can't slow your site. The worst case is that it sits there unread — which, statistically, it will.

Optionality is cheap insurance. The AI search landscape reshuffles quarterly. If any major vendor flips llms.txt support on — even partially, even just for agentic workflows — the sites that already serve a clean file get whatever benefit exists from day one. You're buying a lottery ticket that's free and never expires.

Agentic fetches are the one real, current use case. When a user points an AI agent, a coding assistant, or a custom GPT at your site, those tools can and sometimes do pull llms.txt to orient themselves — that's exactly why the docs ecosystem adopted it. The traffic is marginal, but it's the only place the file demonstrably does work today. If you sell to developers, this alone justifies the file.

It forces a useful exercise. Writing a good llms.txt means deciding what your most important pages are and summarizing your site in one paragraph. If you can't do that, you have a content-architecture problem that no file format will fix — and now you know about it.

What llms.txt does not do: it doesn't influence whether ChatGPT, Claude, Perplexity, or Gemini cite you. That's driven by the things the Princeton GEO research and citation studies actually measure — crawlable content, statistics, quotable claims, source citations, freshness, and your standing in the indexes these engines search. If you have one hour for AI visibility, spend it on your robots.txt AI-bot policy and your content, not on llms.txt.

How to generate one properly

If you're going to ship it, ship it right:

1. Put it at the root: /llms.txt, served as plain text or Markdown, no redirect chains.
2. Follow the spec: one H1 (site name), one blockquote summary, H2 sections with [link](url): description lists. Keep the summary factual — this is not ad copy.
3. Curate ruthlessly. Ten to fifty of your genuinely important URLs beat a sitemap dump. Lead with the pages that answer the questions your customers ask AI engines.
4. Keep it in sync. A stale llms.txt pointing at 404s is worse than none. Regenerate it whenever your key pages change — automate it from your sitemap if you can.
5. Skip llms-full.txt unless you run docs. For documentation sites it's useful for agent workflows; for a marketing site it's maintenance burden with no consumer.
6. Don't block it in robots.txt (yes, we've seen this), and don't put anything in it you wouldn't publish on the page itself.

What not to expect

No ranking improvement. No citation lift in AI answers. No measurable traffic, except occasional agentic fetches. No substitute for fixing your actual content, schema, and crawl policy. Anyone selling llms.txt as a "GEO ranking factor" is selling you the keywords meta tag of 2026.

Ship it in ten minutes, then move on to the work that moves numbers.

If you want to know what actually moves those numbers for your site: HejGeo tracks how ChatGPT, Claude, Perplexity, and Gemini mention and cite your brand, runs a full technical SEO audit alongside it, and gives you a prioritized fix list — including, yes, a free llms.txt generator, sold with exactly the caveats you just read. The free plan covers one project, no credit card. Check your AI visibility →