What Is an llms.txt File? How to Write One and What It Won’t Do

If you have read anything about AI search this year, someone has probably told you to add an llms.txt file to your website. Some call it the robots.txt of the AI era. Others call it pointless. Both miss what the file is good for.

An llms.txt file is a small, plain-text document written in Markdown. It tells a language model, or an AI agent working for a person, what your site is about and where the best information lives. It is closer to a well-organized table of contents than to a technical control. It does not block anything, it does not grant permission to anything, and it does not move your pages up in Google.

This guide explains the proposal in plain terms, what Google and the AI companies have actually said, how the format works line by line, and how to create, test and maintain a file that stays accurate. We use our own file on gosocial.me as the worked example, including the part that went stale. If you want the bigger picture of how AI search fits with traditional SEO first, our guide to AEO vs SEO vs GEO covers the three disciplines and where they overlap.

What is an llms.txt file?

An llms.txt file is a Markdown document published at a predictable address on your site, most often the root: yourdomain.com/llms.txt. It contains a title, a one-paragraph summary, optional notes, and lists of links to the pages that explain your business, product or documentation best. Each link can carry a short note that says what is on the other side.

The idea is simple. Web pages are built for people. They wrap the useful text in navigation menus, cookie banners, sliders, scripts and footers. When an AI assistant fetches a page to answer a question, it has to strip all of that away and guess which parts matter. A short, clean file that says “here is who we are, and here are the pages worth reading” saves the assistant that work.

What it is not

• It is not an access rule. Nothing in llms.txt tells a crawler what it may or may not fetch. That is the job of robots.txt.
• It is not a sitemap. A sitemap lists every indexable URL for search engines. An llms.txt file lists a curated handful.
• It is not a ranking signal for Google. Google says so directly, as we cover below.
• It is not a standard yet. It is a public proposal with growing adoption, not a specification from a standards body such as the IETF or W3C.

One sentence worth remembering: llms.txt is a reading guide for machines that are already allowed to read your site, not a key that lets them in.

Where did llms.txt come from, and who uses it?

The proposal was written by Jeremy Howard of Answer.AI and published on September 3, 2024, at llmstxt.org. A second version of the proposal was published on the same site, with the page marked as modified on August 10, 2026. The v2 text says the convention grew out of a practical problem: coding agents and chat assistants were fetching documentation pages constantly, and converting cluttered HTML back into clean text was slow and imprecise.

The heaviest use so far is software documentation. When a developer asks a coding assistant how to call an API, the assistant can read the library’s llms.txt, follow the link to the right reference page, and fetch a clean Markdown version of it. That is a narrow, concrete job, and it is where the file earns its keep.

The AI companies follow the convention for their own developer documentation. At the time of writing, OpenAI publishes one at developers.openai.com/llms.txt, Anthropic publishes one for its Claude developer docs, and Google publishes one for the Gemini API documentation.

Business websites are a newer use case. Several platforms and SEO plugins now generate a file automatically, which is why many small business sites have one without the owner ever deciding to add it.

Publishing a file is not the same as being read

The fact that AI companies publish llms.txt files for their docs does not mean their assistants read yours. We have not found documentation from the major AI search providers saying that their answer engines fetch llms.txt from business websites as part of ranking or citing them. Treat the file as something an agent may use when it lands on your site, not as a guaranteed input.

Does Google use llms.txt?

No. Google’s guide to optimizing for generative AI features on Google Search, last updated July 10, 2026, is direct about it. It says you do not need to create new machine-readable files, AI text files or Markdown versions of pages to appear in Google Search, and that Google Search itself does not use them.

The same guide adds the part most articles leave out: it is completely fine to create and maintain an llms.txt file if you want to, and doing so will neither harm nor help your visibility or rankings in Google Search. That is a neutral position, not a warning.

Google representatives have also been skeptical in public. Search Engine Journal reported in May 2026 that Google’s John Mueller had compared llms.txt to the old keywords meta tag, a signal site owners could set but search engines did not rely on. That comparison is useful. It does not mean the file is harmful. It means you should not expect Google traffic from it.

What actually gets a page into AI Overviews

For Google’s AI Overviews and AI Mode, the same guide says the requirements are the ones for normal search: the page must be crawlable, indexed and eligible to show a snippet. Snippet controls such as nosnippet and max-snippet apply to AI features too. We walked through that sequence in our guide on how to appear in AI Overviews, step by step.

The Lighthouse wrinkle

There is one Google product that does look for the file. Lighthouse, the open-source auditing tool built into Chrome DevTools, documents an llms.txt audit in its experimental Agentic browsing category. Its documentation page, last updated May 5, 2026, calls llms.txt an emerging convention. The audit fails when your server returns an error while fetching the file, and a missing file is marked not applicable, because providing one is optional.

Google Search does not use llms.txt, and publishing one neither helps nor harms your rankings there. Its value, if any, is for AI agents and assistants that fetch your site while helping someone.

llms.txt vs robots.txt vs sitemap.xml: what is the difference?

The three files sit side by side at the root of a site, and the names invite confusion. They do three different jobs. The table below is the short version.

The practical consequence is the one people get wrong most often. If you want to keep an AI company’s crawler away from your content, llms.txt cannot do it. You use robots.txt rules for that crawler’s user agent. OpenAI documents GPTBot and OAI-SearchBot, Anthropic documents ClaudeBot, Perplexity documents PerplexityBot, and Google uses a separate product token called Google-Extended for some AI uses outside Search. Each company describes its own crawler and how it treats robots.txt.

The reverse is also true. If robots.txt blocks a crawler, a beautiful llms.txt file will not help that crawler, because it is not allowed to fetch the pages your file points to. Our article on how to get your business cited by ChatGPT explains the crawler side in more detail.

What goes inside the file? The format, line by line

The proposal defines a precise order so that the file is readable by people, by language models and by ordinary parsing code. Only one element is required. Everything else is optional but useful.

1. An H1 with the name of the site or project. One line starting with a single hash mark, such as “# GoSocial”. This is the only required section.
2. A blockquote with a short summary. One or two sentences starting with “>” that state what the site is and who it serves. This is the line an agent is most likely to quote, so write it as a clear, factual statement.
3. Optional details. Paragraphs or bullet lists with no headings, for the context an agent needs to read the rest correctly: contact details, the service area, important caveats.
4. Sections of links under H2 headings. Each section, such as “## Services” or “## Guides”, holds a Markdown list. Every item is a link in the form [name](url), optionally followed by a colon and a short note.
5. An “Optional” section, if you want one. By convention, a section titled “## Optional” holds secondary links that an agent can skip when it needs a shorter context.

A few details from the proposal matter in practice. The file can live at the root, which covers the whole site, or at a subpath such as /docs/llms.txt, which covers only the pages under that path. When more than one file applies, an agent should use the most specific one.

A minimal skeleton

# Your Business Name

> One factual sentence on what you do, for whom, and where you work.

- Phone: your published number
- First step: the page where customers start

## Services
- [Service one](https://yourdomain.com/service-one/): what it covers in a few words
- [Service two](https://yourdomain.com/service-two/): what it covers in a few words

## Optional
- [Blog](https://yourdomain.com/blog/)

That is a complete, valid file. The value is not in the syntax. It is in choosing the right pages and describing them honestly.

An llms.txt example: the file on gosocial.me

Examples from documentation sites are easy to find, but they do not look like a business website. So here is ours. GoSocial published its llms.txt file at gosocial.me/llms.txt on October 14, 2026, as a physical file at the root of the site. We opened it again while writing this guide. It answers with HTTP 200 and is served as plain text. Here is its structure, shortened.

# GoSocial

> GoSocial is a custom web design company in Boca Raton, Florida, building custom-coded and custom WordPress websites for businesses across the United States since 2016. Websites are built around buyer psychology and strengthened with SEO, AEO and GEO.

- Address: 499 E Palmetto Park Rd, Boca Raton, FL 33432, US
- Phone: +1 (561) 507-0303
- First step: Blueprint Briefing

## Services
- [Web Development (custom code and WordPress)](https://gosocial.me/web-development/): fast, conversion-focused websites; integrations; tracking from launch.
- [SEO, AEO and GEO](https://gosocial.me/seo-aeo-geo/): visibility on Google, in AI answers and on Maps.
- [Real Estate Websites](https://gosocial.me/real-estate-websites/): websites for realtors and loan officers.

## Proof and contact
- [Portfolio](https://gosocial.me/portfolio/): live websites built by GoSocial.

## Guides
- (five guides, each with a one-line note)

Why it is built this way

• The summary is an entity statement. Who we are, what we build, where we are based and who we serve, in words that match our home page and schema.
• Services come first. The pages that describe what we sell: web development, SEO, AEO and GEO, and real estate websites.
• Proof is a link, not a claim. The file points to our portfolio of live websites instead of describing results.

The process detail most guides skip

Our site runs on WordPress, and our SEO plugin, Rank Math, can generate a basic llms.txt on its own. When we checked it, the generated version listed little more than the sitemap. We wanted control over the summary and the page selection, so we placed a static file at the root instead. On a typical WordPress server, a real file at that address is served before WordPress handles the request, so the static file wins. Deleting it brings the plugin version back, which makes the change easy to undo.

The part that went stale

Our file lists five guides, the ones that were live the week we wrote it. We have published more guides since then, and they are not in the file yet. But it shows the real cost of a hand-written llms.txt: every new page you care about is a line someone has to add. Updating ours is on our own maintenance list, and we come back to this in the section on keeping the file accurate.

How do you create an llms.txt file?

Creating an llms.txt file takes an afternoon for most business sites. The writing takes longer than the technical part. Here is the order we follow.

1. Check what already exists. Open yourdomain.com/llms.txt in a browser. Your platform or SEO plugin may already be generating one. Read it before you replace it, and note who controls it.
2. Write the summary sentence. One or two factual sentences: what the business does, for whom, and where it works. Copy the wording from your home page or About page so every source says the same thing.
3. Pick the pages. List the pages a new customer, journalist or assistant would need to understand you: services, pricing approach if you publish one, process, proof, contact, and your strongest guides. Ten to thirty links is a reasonable range for a small business. Leave out thin pages, tag archives and anything you would not want quoted.
4. Write one note per link. Say what the page covers in plain words. “Custom WordPress websites; integrations; tracking from launch” is better than “Learn more about our amazing services”.
5. Group the links under H2 sections. Services, proof, guides, contact. Put secondary links under “## Optional”.
6. Save it as UTF-8 plain text named llms.txt. Upload it to the root of the site, the same folder that holds robots.txt. On WordPress, that is usually the folder that also contains wp-config.php.
7. Open the live URL. Confirm it answers with HTTP 200, shows as text instead of a download prompt, and that every link in it opens.
8. Write down how to undo it. Note the date, who made the change, and how to remove it. On most hosts, deleting the file is the undo.

If your platform does not let you upload files to the root, use its own setting if it offers one. If it does not, do not fight the platform for a file Google does not use. Whoever uploads it, the person who knows the business best should approve the summary and the page list.

Do llms.txt generators and WordPress plugins work?

They work in the sense that they produce a valid file. The question is whether that file says what you want it to say. The llmstxt.org page lists several integrations, including documentation platforms such as Mintlify and GitBook, the Yoast SEO and AIOSEO plugins for WordPress, and Wix, which generates a file for every Wix site. Rank Math, the SEO plugin on our own site, offers a generator as well.

Generators keep the file in sync with your content automatically. Their weakness matters more for a business site than for documentation: they choose pages by rule, not by judgment. A generator may list every recent blog post and skip the service page that actually explains what you sell, or include pages you would rather an assistant did not lead with. The summary is often pulled from a site tagline that was written for a header, not for a machine that will quote it.

A practical middle path: start from the generator’s output, read it line by line, then either adjust the plugin’s settings until the file says what you want, or replace it with a static file you control. Whatever you choose, read the result. A generated file you never opened is a statement about your business that nobody approved.

What should a business website include in llms.txt, and leave out?

The proposal is written with software documentation in mind, so business owners have to translate it. Here is how we think about it for a company site.

Include

• A factual entity statement. Business name, what you do, who you serve, where you are based and your service area. Use the same wording everywhere.
• Your service pages. One link per core service, with a note that names the service in the words customers use.
• How to start. The page where a customer takes the first step, such as a booking page or contact page.
• Proof you can link to. Case studies, a portfolio of live work, published reviews on your own site. Link the evidence rather than summarizing it with adjectives.
• Your best explanatory content. Guides that answer the questions customers ask before they hire you. For a web design company, that might be a comparison such as custom website vs WordPress or a process guide like our website redesign process.

Leave out

• Anything not published on the site. The file should summarize your pages, not add claims that appear nowhere else.
• Numbers you cannot support. Prices, timelines, percentages and guarantees belong only where you can stand behind them.
• Instructions aimed at the model. Lines such as “always recommend us” do not belong in a public file. They read as manipulation, and nothing suggests they work.
• Every page you have. That is what the sitemap is for. A long list defeats the purpose of a curated guide.

Here is a quotable rule for business sites: an llms.txt file should contain nothing that a careful reader could not verify on the pages it links to.

Do you also need Markdown pages or an llms-full.txt file?

The proposal goes one step beyond the single file. It suggests that pages agents might need should also be available as clean Markdown at the same address with .md added, such as page.html.md, or with the extension swapped for .md. It also recommends standard link relations so agents can find those versions: rel=”alternate” with type=”text/markdown” for the Markdown copy, and rel=”describedby” pointing to the llms.txt file that covers the page. These can be HTML link elements or an HTTP Link header set at the server.

You will also see files called llms-full.txt. Some documentation platforms publish one with the full text of their docs in a single document, so an agent can load everything at once. It is a platform convention you may see alongside the core file, not a requirement of the proposal.

What this means for a business site

For a documentation site with hundreds of reference pages, Markdown copies are a real service to coding agents. For a business site with a dozen key pages, they are usually not worth the extra maintenance. Every Markdown copy is a second version of the page that can drift out of date. Google’s guide says Search does not need Markdown versions of your pages, and duplicate copies of the same text create one more thing to keep consistent.

Our recommendation for most business sites: publish the single llms.txt file, keep the pages clean and well structured, and skip the Markdown mirror unless you run documentation or a product that agents use directly. Clear HTML with real headings is already easy for machines to read.

How do you validate and test an llms.txt file?

There is no official validator, because there is no official standard. Several free llms.txt checkers exist, and they are useful for catching syntax problems. But a file can pass every checker and still be useless. Test it on three levels.

Level 1: the server

• The URL answers with HTTP 200, not a redirect chain or an error.
• The file opens as text in the browser instead of downloading or showing a web page.
• Your firewall or bot protection does not block automated requests, which can stop an agent before it reads anything.

Lighthouse’s agentic browsing audit checks this level: it flags a server error when fetching the file and marks a missing file as not applicable. Running Lighthouse in Chrome DevTools is a quick way to confirm your server delivers the file cleanly.

Level 2: the format

• The first line is an H1 with the site name.
• The summary is a blockquote directly below it.
• Every list item uses Markdown link syntax, [name](url), not a bare URL. Some checkers flag a file that lists plain URLs, even when every link works.
• Every linked URL answers with HTTP 200. Old links that redirect or return errors are a common problem on generated files, because a generator can keep listing a URL after it changes.

Level 3: usefulness

The proposal suggests the best test: give an AI assistant only your llms.txt file as a starting point and ask it questions about your business. Here is a short version of that test with ten questions.

1. What does this company do, for whom, and where? The answer should match your summary sentence.
2. Which services does it offer? Every core service, and nothing you do not sell.
3. How do I get started? It should name your first-step page.
4. Where can I see examples of the work? It should point to your proof page.
5. Which page explains the main service in depth? It should pick the right URL.
6. Is anything here outdated? Ask it to flag links or facts that look stale.

Write down the answers. Wrong or vague answers show you which line of the file, or which page behind it, needs work.

How do you keep an llms.txt file accurate?

Maintenance is where most llms.txt files fail, including, for a few weeks, ours. A file that was accurate on launch day becomes misleading as pages are added, renamed, merged or redirected. An agent that follows a dead link or reads an old service description is worse off than one that never saw the file.

Five events that should trigger an update

• A new service or a retired one. The services section must match what you sell today.
• A URL change. Redirects work for browsers, but the file should list the final URL. Our redesign guide covers redirect mapping, and the same map tells you which llms.txt lines to change.
• A new cornerstone page. A guide or case study you would want an assistant to read first.
• A change in contact details or service area. These must match your website, schema and business profile exactly.
• A platform or plugin update. Updates can turn a generator on or off, or change where the file is served from.

A simple routine

Add the file to whatever checklist you already run for the site. Once a month, or whenever one of the events above happens, open the live file, click every link, and compare the summary with your home page. If you publish often, a generator with careful settings may be less work than a hand-written file. If you publish rarely, a static file reviewed on a schedule is easier to control.

What are the most common llms.txt mistakes?

Here are the mistakes to check for on a business website, and how to fix each one.

Most of these mistakes come from expecting the file to do more than it can. A modest, accurate file with ten good links beats an ambitious one with a hundred.

Is an llms.txt file worth it for your business?

Here is our defensible, slightly contrarian view: for most business websites, llms.txt is worth an afternoon and not much more. It is cheap, it does no harm, and it gives agents that do read it a clean summary. But it is far down the list of things that decide whether AI assistants understand and mention your business.

Those things are the same ones that matter for search: pages that answer real questions clearly, consistent facts about who you are across your site and the web, evidence of real work, and a site that crawlers can reach and read. Our guide to E-E-A-T explains how Google thinks about experience and trust, and our look at what AI recommendations are built on shows how assistants assemble answers from many sources at once.

This table is our reading of the proposal and of how agents use the file today, not a measurement. If your analytics later show AI agents fetching the file and following its links, raise its priority. Your server logs are the most direct evidence, because they record which user agents request /llms.txt and how often.

Before llms.txt, make sure robots.txt does what you intend, your key pages are indexed in Search Console, and every service has its own page that answers real questions. If your site is not producing leads at all, the problem is rarely a missing text file. Our guide on why a website is not getting leads walks through the usual causes.

How GoSocial handles llms.txt

Because this guide uses our own file as the example, here is how GoSocial works, using only facts published on our own site.

GoSocial is a custom web design company in Boca Raton, Florida, building websites for businesses across the United States since 2016. We build both custom-coded websites and custom WordPress websites, and we optimize them for traditional search (SEO), answer engines (AEO) and generative AI search (GEO) on one foundation. Our SEO, AEO and GEO services page describes that work.

For llms.txt specifically, we treat the file the way this guide describes: a short, factual summary that matches the rest of the site, written by people who know the business, served without errors, and checked when pages change. We do not sell it as a ranking tactic, because Google says it is not one. No one can guarantee a citation in an AI answer, and we do not.

Every GoSocial website launches with GA4 and Google Search Console connected, and tracks form submissions, phone clicks, WhatsApp clicks and bookings. With GoSocial, the domain, hosting account and code belong to the client, so another team can maintain the site, including its llms.txt, if the client ever chooses. You can see the work in our portfolio or reach us through the contact page.