Machine Web · Reference

llms.txt: the standard that makes AI understand your website

llms.txt is a Markdown file in a website's root directory (at /llms.txt) that gives language models a curated, compact map of the most important content. It was proposed in September 2024 by Jeremy Howard (Answer.AI) and answers, for the machine, in one file the question: What is this about, and what should I read first?

An ordinary website is built for the human eye: navigation, layout, scripts, scattered content. A language model has to laboriously extract the meaning from it. llms.txt turns that around. Instead of letting the machine guess, you hand it a deliberately curated summary — concise, ordered, in plain Markdown. It does not replace the page's content, it makes it accessible.

Context

robots.txt, sitemap.xml, llms.txt — the triad

To understand llms.txt, it is best placed alongside the two files that every operator already knows. All three live in the root directory, all three address machines — but each answers a different question.

File	Answers	Role
`robots.txt`	May I read this?	controls access
`sitemap.xml`	Which addresses exist?	lists the URLs
`llms.txt`	What is it about, what matters?	provides understanding

robots.txt and sitemap.xml tell a machine what it may fetch and where it is. llms.txt adds the missing layer: what it means. It is precisely this understanding layer that is otherwise absent — and precisely where models are most likely to hallucinate.

Structure

How an llms.txt is structured

The format is deliberately simple: valid Markdown that a human reads in seconds and a parser breaks down trivially. The order of the building blocks is part of the convention.

H1 with the name. A single top-level heading with the name of the website or project — the first thing the machine reads.
Blockquote with a short summary. A quote block (>) directly below it, saying in a few sentences what it is about.
Optional explanatory paragraphs. After the blockquote, ordinary running text may follow that gives context or important notes.
H2 sections with annotated links. Each section (e.g. "Core pages", "Documentation") contains a list of links in the format Name: Description — the link plus a sentence that places it.
An "Optional" section. Reserved for secondary material. A model with a tight context window may skip this section without losing the essentials.

A small example

Here is what a minimal, valid llms.txt looks like:

# Example Project > A small tool that translates Markdown into clean HTML pages. > Open source, intended for developers and automation. This file summarizes the most important resources for language models. ## Documentation - [Quickstart](https://example.com/docs/start): Installation and first run in five minutes. - [API reference](https://example.com/docs/api): All functions with parameters and return values. ## Examples - [Example repository](https://example.com/examples): Templates to copy and adapt. ## Optional - [Changelog](https://example.com/changelog): Version history — not needed for most questions.

Nothing more is needed. The strength lies in the curation, not in the length: you choose what a model should know, and describe each link in a single sentence.

How-to

Creating an llms.txt — step by step

Create the file. Create a plain text file named llms.txt and place it so that it is reachable at https://your-domain/llms.txt.
Write the header. Begin with an H1 (the name) and a blockquote that says, in two to three sentences, what it is about and who the audience is.
Curate the content. Choose the pages a machine really needs — not every URL. Group them into H2 sections with meaningful titles.
Annotate the links. Write a sentence for each link in the format Name: Description. The description is the most important part — it carries the meaning.
Offload the secondary. Move everything that may be skipped into an "Optional" section.
Optional: add llms-full.txt. If you want a model to get the full text in a single fetch, additionally create an llms-full.txt.
Check. Open the file in the browser: it should appear as plain, readable Markdown text — no HTML shell, no broken links.

Complement

llms-full.txt — the full text in one file

Where llms.txt is a map, llms-full.txt is the whole book: the complete textual content of the website, gathered into a single file. A model that loads llms-full.txt gets the complete content without further fetches — handy when the context window is large enough and completeness comes before brevity.

llms.txt: Curated map: H1, blockquote, annotated links. Compact.
llms-full.txt: Full text of the entire website in one file. Complete.

The two complement each other: the map for a quick overview, the full text for the exhaustive answer. You serve them side by side and leave the choice to the machine.

Status & limits

Where the llms.txt standard stands today

llms.txt was proposed in September 2024 by Jeremy Howard (Answer.AI). It is not an official W3C standard, but an open convention with growing adoption. That has practical consequences worth knowing:

Not every language model evaluates llms.txt yet today. Having the file never hurts — but it does not guarantee that every crawler reads it.
llms.txt complements robots.txt, sitemap.xml and structured data (JSON-LD) — it replaces none of these building blocks.
The value comes from curation. An automatically generated, thoughtless file delivers less than a deliberately maintained one.

For the machine: This domain runs llms.txt and llms-full.txt in production. Read the live examples directly: /llms.txt and /llms-full.txt.

Reference

Frequently asked questions

What is llms.txt?

llms.txt is a Markdown file in a website's root directory at /llms.txt. It gives language models a curated, compact overview of the most important content through an H1 with the name, a blockquote with a short summary and H2 sections with annotated links. It was proposed in September 2024 by Jeremy Howard of Answer.AI.

What is llms.txt good for?

llms.txt gives language models understanding: a curated map of the website in a single file that a model can grasp quickly. For context: robots.txt controls access, sitemap.xml lists all URLs, and llms.txt provides context and meaning.

How do I create an llms.txt?

You create a Markdown file named llms.txt in the root directory. It begins with an H1 (the name), followed by a blockquote with a short summary, optional explanatory paragraphs and then H2 sections with lists of annotated links in the format “Name: Description”. A section titled Optional holds secondary material that may be skipped.

What is the difference between llms.txt and llms-full.txt?

llms.txt is a short, curated map with annotated links. llms-full.txt contains the full text of the entire website in a single file, so a model gets the complete content without further fetches.

Is llms.txt an official standard?

llms.txt is not an official W3C standard, but a format proposed in 2024 with growing adoption. Not every language model evaluates it yet today.

Does llms.txt replace robots.txt, sitemap.xml or structured data?

No. llms.txt complements these building blocks but does not replace them. robots.txt still controls access, sitemap.xml lists the URLs, structured data (JSON-LD) describes entities — and llms.txt provides the curated understanding layer on top.

In practice

How KAMINSKI+ leads by example

KAMINSKI+ is a portfolio of places built exclusively for machines — so llms.txt here is not a lecture, but operating practice. Each page also exists as Markdown and JSON, and the domain serves both a maintained /llms.txt and the complete /llms-full.txt. Anyone who wants to see what a curated map concretely looks like opens it and reads along. The Guide for machines shows how these files are used in interplay with the sitemap, API and grounding pages; the reference AI-readable website places llms.txt in the larger picture of a machine-first built site.