# llms.txt: what it is, what it is not, and how to write one that earns its place > A practical guide to the llms.txt format — the correct structure, the mistakes that make it useless, and an honest account of what adoption actually looks like today. - Source: https://shiftrank.ai/blog/llms-txt-guide - Published: 2026-06-24 - Updated: 2026-08-07 - Author: Shiftrank - Category: AEO - Tags: llms.txt, aeo, standards --- `llms.txt` is a markdown file at your domain root that tells an AI client what your site is and which pages matter. Jeremy Howard proposed it in 2024, and it has spread quickly through developer-tools companies — Anthropic, Stripe, and Cloudflare all publish one. Two things are true at once, and skipping either produces bad advice. The format is genuinely useful and cheap to adopt. And no major AI provider has committed to reading it as a retrieval input. This guide treats it accordingly. ## What the format is `llms.txt` lives at `https://example.com/llms.txt`. It is markdown, and the structure is fixed enough to be parsed but loose enough to write by hand: ```markdown # Acme > Acme is usage-based billing infrastructure for SaaS companies. > Meter events, price them, and invoice — without building a billing system. Acme handles metering, rating, invoicing, and revenue recognition. Most customers integrate in a day using the HTTP API or one of six SDKs. ## Docs - [Quickstart](https://acme.com/docs/quickstart): Send your first metered event in ten minutes - [Pricing models](https://acme.com/docs/pricing-models): Tiered, volume, package, and hybrid schemes - [API reference](https://acme.com/docs/api): Full HTTP surface with request and response schemas ## Product - [Pricing](https://acme.com/pricing): Plans, limits, and overage rates - [Security](https://acme.com/security): SOC 2 Type II, data residency, retention ## Optional - [Changelog](https://acme.com/changelog): Weekly release notes ``` The parts, in order: 1. **An H1** with the site or organisation name. Exactly one. 2. **A blockquote** with a one- or two-sentence summary. This is the highest-leverage text in the file — it is the sentence most likely to be lifted verbatim when something describes you. 3. **Optional prose** giving context that does not fit the blockquote. No headings here. 4. **H2 sections** grouping links. Each link is `[Title](absolute-url): one-line summary`. 5. **An `## Optional` section** at the end for material a client can skip when it needs to be brief. Some sites also publish `llms-full.txt`, which inlines the full text of the linked documents into a single file. It is a different artefact with a different use — a compact corpus to paste into a context window, rather than a map — and it is only worth maintaining if your docs are stable enough that it will not go stale. ## The mistakes that make it useless **Dumping your sitemap into it.** The value is curation. Two hundred links with no summaries is worse than eight links with good ones, because it hands the consumer the same "figure out what matters" problem it already had. **Marketing voice in the blockquote.** "Acme empowers modern teams to unlock the future of revenue" tells a model nothing it can use to decide whether your page answers a question. Say what the product does, for whom, in concrete nouns. The test: could a competitor's blockquote be swapped in without anyone noticing? If yes, rewrite it. **Relative URLs.** The file may be read entirely out of context, with no base URL. Every link must be absolute. **Letting it rot.** A file listing three deprecated endpoints is actively harmful — it is a confident, machine-readable statement of things that are no longer true. Either wire it into your release process or keep it small enough that reviewing it is trivial. **Serving it as HTML.** It should return `text/plain` or `text/markdown`. A framework that wraps unknown routes in your app shell will silently break it. Check with `curl -sI https://example.com/llms.txt`. ## What it is not It is not `robots.txt`. `robots.txt` is access control that crawlers broadly honour; `llms.txt` is editorial guidance that consumers may ignore. Publishing `llms.txt` grants no permissions and revokes none. If you want to control what AI crawlers may fetch, that is a `robots.txt` and edge-rules problem — see [robots.txt for AI crawlers](/blog/robots-txt-for-ai-crawlers). It is not a ranking signal. Nobody has published evidence that `llms.txt` affects retrieval or citation frequency, and you should be suspicious of anyone selling it as one. It is not a substitute for the pages themselves. A model that follows your links still has to parse what it finds. If the destination pages extract badly, a tidy map just gets it to bad content faster. ## Writing yours Start from the questions people actually ask about you, not from your site structure. If prospects ask "does it support usage-based pricing", "how long does integration take", and "what does it cost", then the linked pages should answer those three questions and the summaries should say so. Keep it to fifteen or twenty links. Write summaries that state what the reader will learn, not what the page is called. Then set a calendar reminder to reread it in a quarter. You can see [ours](/llms.txt) — it is short on purpose. ## Knowing whether anyone reads it This is the part that turns `llms.txt` from an act of faith into a measurable one. Requests to `/llms.txt` are ordinary HTTP requests, with a user agent attached. If you log them, you know precisely which operators fetch the file, how often, and whether they come back after you change it. Since client-side analytics never see them, that measurement has to happen server-side or at whatever sits in front of your origin. Shiftrank serves the file at the edge and records every fetch — which is how we can say with confidence that ours gets read. ## FAQ ### What is llms.txt? llms.txt is a markdown file served at the root of a domain (https://example.com/llms.txt) that gives large language models a curated map of the site: a one-line description of what the organisation does, and a short set of links to the pages that matter most, each with a brief summary. It was proposed by Jeremy Howard in 2024 as a way to give models a compact, high-signal entry point instead of forcing them to infer structure from navigation markup. ### Is llms.txt the same as robots.txt? No, and they solve opposite problems. robots.txt is an access control file — it tells crawlers what they may not fetch. llms.txt is an editorial file — it tells models what is worth reading and how to describe you. robots.txt is widely honoured; llms.txt is a convention that consumers may or may not read. ### Do AI companies actually read llms.txt? Adoption is uneven and no major provider has committed to reading it as a ranking or retrieval input. Treat it as low-cost infrastructure rather than a growth lever: it takes an afternoon, it doubles as a genuinely useful internal artefact, and the downside of publishing one is close to zero. If you can measure requests to the file, you will know exactly who reads yours.