$ man geo/llms-txt-guide

llms.txt is an emerging web standard - proposed at llmstxt.org - that gives AI assistants and language models a structured, human-readable overview of your website. Think of it as a README for AI. While robots.txt tells crawlers what they can access and sitemaps tell them where pages are, llms.txt tells them what your site is about, what it offers, and where to find the most important content. The file lives at your site root - yoursite.com/llms.txt - and uses Markdown formatting with a clear structure: site name, brief description, main sections with links, and optionally a more detailed version at llms-full.txt. AI assistants that support the standard read llms.txt when they encounter your domain, giving them immediate context about your site without having to crawl and parse every page. This is particularly valuable for AI coding assistants, research tools, and chatbots that need to quickly understand what a site offers.

PATTERN

The format is simple Markdown with a specific structure. Start with a top-level H1 heading with your site or project name. Follow with a blockquote containing a one-line description of what the site does. Then add sections with H2 headings for different areas of your site, with bullet-pointed links using Markdown link syntax. Each link can have a colon-separated description after it. The standard also supports an extended version at llms-full.txt that includes more detail, full content excerpts, or API documentation. For a site like ShawnOS.ai with multiple properties in a monorepo - a main site, a wiki, and a blog - the llms.txt file maps out each property and its purpose, making it trivial for an AI assistant to understand the site structure. Keep the main llms.txt concise - under 100 lines. Use llms-full.txt for comprehensive documentation.

CODE

If you run multiple sites or a monorepo with multiple Next.js apps, each site should have its own llms.txt. In a Next.js app, serve llms.txt as a static file from the public directory or generate it dynamically through an API route. For the ShawnOS.ai monorepo, each app has its own llms.txt that describes that specific site, while the main domain's llms.txt links to the other properties. Dynamic generation is useful when your content changes frequently - you can build the llms.txt at build time from your content data, ensuring it always reflects the current state of your wiki entries, blog posts, and tool pages. The build script reads your content arrays, extracts titles and URLs, formats them as Markdown links with descriptions, and writes the output to public/llms.txt. This way, every deploy automatically updates your llms.txt to match your current content inventory.

PRO TIP

AI assistants may not read your entire llms.txt - they might process just the first section or stop after a token limit. Structure your llms.txt with the most important content first. Lead with your core offering and highest-value pages. Put your wiki, guides, and reference content near the top. Push less critical pages - about, contact, legal - to the bottom. Think of it as an inverted pyramid, just like writing for AI extraction in general. The first 20 lines of your llms.txt should give an AI assistant enough context to understand your site and find your best content. If it reads nothing else, those 20 lines should be sufficient. This is the same front-loading principle that drives answer blocks and content extractability, applied to your site-level metadata.