Should you add llms.txt to your website?

Add llms.txt only if you have a use for a compact Markdown directory of your public pages. It is optional documentation. Google's AI features guidance says that special AI text files are not required for AI Overviews or AI Mode.

The community llms.txt proposal gives systems that choose to read the file a short list of useful pages. It cannot force a citation or replace crawlable content, internal links, sitemaps, or canonical URLs. If nobody needs the file, leaving it absent is reasonable.

Structure of an llms.txt file

A small file can look like this:

# Site title
> Brief site description

## Section Name
- [Page Title](https://example.com/page): One-line summary of what this page covers.
- [Another Page](https://example.com/another): Summary text.

Each section groups related pages. The proposal also allows an ## Optional section for secondary material. For agents that choose to read the file, factual summaries can make it easier to select a relevant destination. Treat them as navigation copy, not as guaranteed instructions to every AI product.

Generating llms.txt in Laravel

A Laravel application with a database-backed content model can generate llms.txt dynamically through a route or controller. This route serves the file at /llms.txt:

// routes/web.php
Route::get('/llms.txt', function () {
    $content = collect([]);

    $content->push("# " . config('app.name'));
    $content->push("> " . config('app.description'));
    $content->push("");

    // Products section
    $products = Product::where('is_active', true)
        ->where('inventory_count', '>', 0)
        ->take(500)
        ->get();

    if ($products->count()) {
        $content->push("## Products");
        foreach ($products as $product) {
            $summary = Str::limit($product->meta_description ?? $product->name, 120);
            $content->push("- [{$product->name}]({$product->url}): {$summary}");
        }
        $content->push("");
    }

    // Documentation section
    $docs = Documentation::published()->take(200)->get();
    if ($docs->count()) {
        $content->push("## Documentation");
        foreach ($docs as $doc) {
            $content->push("- [{$doc->title}]({$doc->url}): {$doc->excerpt}");
        }
    }

    return response($content->implode("\n"))
        ->header('Content-Type', 'text/plain')
        ->header('Cache-Control', 'public, max-age=3600');
});

Cache the output if the queries are expensive, and refresh it when public content changes. Remove deleted, unpublished, and redirected URLs.

Generating llms.txt in Next.js

In the Next.js App Router, a route handler can return plain text. The content helpers in this example must return only published, public pages:

// app/llms.txt/route.ts
import { getProducts, getDocs } from '@/lib/content';

export async function GET() {
  const products = await getProducts({ limit: 500 });
  const docs = await getDocs({ limit: 200 });

  const content = `# Example Site
> Product information and documentation.

## Products
${products.map(p => `- [${p.name}](${p.url}): ${p.excerpt}`).join('\n')}

## Documentation
${docs.map(d => `- [${d.title}](${d.url}): ${d.excerpt}`).join('\n')}
`;

  return new Response(content, {
    headers: {
      'Content-Type': 'text/plain',
      'Cache-Control': 'public, s-maxage=3600, stale-while-revalidate=86400',
    },
  });
}

Static site with a Go generator

For a static site, generate the file from the same public-page list as your sitemap. Sorting keeps the output predictable; returning errors makes failed writes visible:

package main

import (
    "fmt"
    "os"
    "sort"
    "strings"
)

type Page struct {
    Title   string
    URL     string
    Summary string
    Section string
}

func writeLLMSTxt(path string, pages []Page) error {
    sort.Slice(pages, func(i, j int) bool {
        if pages[i].Section == pages[j].Section {
            return pages[i].URL < pages[j].URL
        }
        return pages[i].Section < pages[j].Section
    })

    var output strings.Builder
    output.WriteString("# Example Site\n> A technical knowledge base\n")
    section := ""
    for _, page := range pages {
        if page.Section != section {
            section = page.Section
            fmt.Fprintf(&output, "\n## %s\n", section)
        }
        fmt.Fprintf(&output, "- [%s](%s): %s\n", page.Title, page.URL, page.Summary)
    }
    return os.WriteFile(path, []byte(output.String()), 0o644)
}

The omitted catalog loader should include only public, canonical routes and should reject newlines or unescaped Markdown delimiters in titles, URLs, and summaries. Do not publish draft records merely because a directory walk found them.

WordPress with a plugin approach

A WordPress mu-plugin can generate a basic file from published content. First check that your existing plugins are not already serving the same path:

add_action('template_redirect', function () {
    if (wp_parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH) !== '/llms.txt') {
        return;
    }

    $posts = get_posts([
        'post_type' => ['post', 'page', 'product'],
        'post_status' => 'publish',
        'posts_per_page' => 500,
    ]);

    header('Content-Type: text/plain; charset=utf-8');
    echo "# " . get_bloginfo('name') . "\n";
    echo "> " . get_bloginfo('description') . "\n\n";

    echo "## Articles\n";
    foreach ($posts as $post) {
        $excerpt = wp_trim_words($post->post_excerpt ?: $post->post_title, 20);
        $permalink = get_permalink($post);
        echo "- [{$post->post_title}]({$permalink}): {$excerpt}\n";
    }
    exit;
});

Check the file after publishing

Check that /llms.txt returns 200 text/plain, contains only public canonical URLs, omits drafts and redirects, and changes when the source catalog changes. A server error is a defect if you deliberately publish the file; not publishing the optional file is not.

Keep the file only if you can maintain it alongside the pages it describes. Measure any use through actual requests or a tool integration, rather than assuming it improves search visibility.