Aizii
Aizii Learn · AI Search
← Back to Learn

How to add llms.txt to a self-hosted site

A self-hosted site means you control the server itself: the file system, the web server config, and the deployment. Adding llms.txt is the simplest version of this task: put the file in your web document root (the same directory as index.php or index.html, wherever your vhost points; commonly public_html/ or /var/www/html), and it serves at [yourdomain.com]/llms.txt, replacing yourdomain.com with your actual domain. The work is in confirming the server hands it out correctly.

Why a self-hosted site is different

Nobody stands between you and the root, which means the failure modes are all yours. The classic ones: the file lands in the wrong directory (a staging vhost's docroot, a subdirectory, anywhere but the production web root), the server sends it with the wrong content type, an aggressive cache keeps serving the old version after you update it, or file permissions block the web server from reading it. Every one of these shows up as "the file is there but the check fails," and the fix is a server setting, not a file edit.

The file you are publishing

The full format lives in our llms.txt guide. This is the file itself, from a sample outdoor gear store:

# Cedar and Pine Outfitters
> Cedar and Pine Outfitters sells outdoor apparel and camping gear from its Seattle storefront, shipping across the US. Runs an in-house tent repair service and publishes beginner backpacking guides.
## Shop
- [Tents](https://[yourdomain.com]/collections/tents): Two-person and family tents, described by season rating and packed weight.
- [Sleeping bags](https://[yourdomain.com]/collections/sleeping-bags): Down and synthetic bags, described by temperature rating.
- [Backpacks](https://[yourdomain.com]/collections/backpacks): Day packs and multi-day packs, listed with capacity in liters.
## Guides
- [Backpacking checklist](https://[yourdomain.com]/pages/backpacking-checklist): Complete gear list for a first overnight trip.
- [Tent care and repair](https://[yourdomain.com]/pages/tent-repair): How the in-house repair service works, turnaround and pricing.
## Policies
- [Shipping and returns](https://[yourdomain.com]/pages/shipping-returns): Free shipping over 75 dollars. 60-day returns on unused gear.
## Optional
- [Journal](https://[yourdomain.com]/blogs/journal): Trip reports and gear maintenance notes.

Steps

  1. Check first. Open [yourdomain.com]/llms.txt in a browser. Then check the headers with curl -I https://[yourdomain.com]/llms.txt and look for a 200 status and a text/plain content type.
  1. Upload the file. Use SFTP or your host's file manager to place llms.txt in the document root: the directory that holds your site's index file. Set permissions to 644. If you deploy through git or a pipeline, commit the file into the repo instead so it survives the next deploy.
  1. Fix the content type if it is wrong. Most servers already serve .txt as text/plain, but check the curl output. On Apache add AddType text/plain .txt to your site config if needed; on nginx the default types map usually covers it. If the browser offers the file as a download instead of rendering it, this is the setting to fix.
  1. Clear the cache after updates. If you run a page cache, CDN, or proxy in front of the server, purge or bypass it for /llms.txt when you update the file; otherwise you will stare at the old version and conclude the upload failed.
  1. Use your logs. You own the access logs, which is an advantage no hosted platform gives you: filter them for requests to /llms.txt and you can see which crawlers (look for user agents like GPTBot, ClaudeBot, or PerplexityBot) are actually fetching the file. That is the only real measurement of whether anyone reads it.

The fundamentals

  • •What goes in the file: your name, one-paragraph summary, and curated links with one-line descriptions, see our llms.txt guide
  • •Crawler permissions live in robots.txt, never in llms.txt, see our robots.txt guide
  • •The structured data behind the pages you link, see our product schema guide

A hypothetical example

Say your store runs on a VPS with nginx. You SFTP llms.txt into the docroot, and the browser shows it fine, but your curl check shows the CDN in front of the site still serving a 404 from before the file existed. You purge the CDN cache for that path, and the 200 appears. Later, your logs show a handful of requests to /llms.txt from AI crawler user agents, which tells you the file is at least being fetched. That is the self-hosted loop: upload, verify headers, clear caches, read logs. Nothing about it is hard; it just requires you to remember the layers.

What this guide does not cover

Tuning crawler access rules in your robots.txt or firewall. Log analysis beyond the /llms.txt hits. The transaction layer.

Frequently Asked Questions

Where do I upload llms.txt on an Apache or Nginx server?
Place llms.txt in the document root directory specified in your virtual host configuration, such as /var/www/html.
Do I need special Nginx configuration to serve llms.txt?
Usually no, but ensure your server block does not restrict .txt files and that it returns Content-Type text/plain; charset=utf-8.
How do I check permissions for the file on Linux?
Ensure the file has read permissions for the web server user, typically chmod 644 llms.txt.

Check your own brand now

Run the free scan at aizii.ai/aeo-scan. One click, no signup, and it shows how readable your site is to AI tools today.

Run the free scan