$ man how-to/create-llms-txt

llms.txt is a machine-readable file that sits at your site root and helps LLMs and AI assistants understand what your site contains and how it is organized. Think of it as robots.txt for AI comprehension. robots.txt tells crawlers what they can access. llms.txt tells AI assistants what is actually there and what it is about. When an AI assistant encounters your domain, llms.txt gives it instant context — site description, content types, feed URLs, key topics. It is a small text file with outsized impact on AI discoverability. The format is simple markdown with a few conventions.

CODE

The file follows a simple markdown structure. Start with a level-one heading containing your site name. Add a blockquote with a one-sentence description. Then use level-two headings to organize sections: About (author, tech stack, network), Content Types (paths and descriptions for each content type), Feeds (RSS and other feed URLs), and Key Topics (bulleted list of what your site covers). Keep it concise. This is not a full sitemap. It is a high-level content map that helps AI assistants decide whether your site is relevant to a query and where to look for specific types of information. Each content type entry should include the URL path and a brief description of what lives there. Feed URLs help AI assistants discover your latest content without crawling every page.

PATTERN

Place llms.txt in your public directory so it is served at yourdomain.com/llms.txt. In a Next.js app, that means the public/ folder at the app root. In ShawnOS, each of the three sites has its own llms.txt tailored to that site's content. shawnos.ai/llms.txt maps the blog, knowledge base, how-to wiki, daily logs, and all wiki sections. thegtmos.ai/llms.txt maps the GTM knowledge base and Clay wiki. thecontentos.ai/llms.txt maps the content wiki. Test it by navigating to the URL in your browser. You should see the raw markdown text. Then test it by asking an AI assistant about your site and checking whether it references the content types and structure described in your llms.txt. The file is not widely standardized yet, but early adoption means you are indexed before the standard becomes crowded.