How to Generate an LLMs.txt File (Manually and Using Generators)

At this point you realise a llms.txt file can give your website a big advantage in the AI-powered future of search. The next step is to actually create and implement that file on your site. This post will guide you on how to create a llms.txt file, exploring both manual and automated ways. Whether you’re a DIY type or prefer plug-and-play tools, we have you covered. Let's give your content that extra visibility boost with minimum hassle!

llms.txt Manual Compilation

You can easily create a llms.txt file manually, but you will have to pay close attention to the format. Here's how to do it - step by step:

1. Identify Your Core Content:

First, make a list of the pages on your site that you want an AI to know about. These can be: - FAQ pages answering common user questions Knowledge base article or documentation. - High value blog posts/guides. - Lots of info on product pages (though docs or FAQs are usually more useful to list, rather than all product pages, unless you have a top products list). - “About” or “How it works” pages if they offer important context for your site.

Quality over quantity. Depending on your site size, you may choose between 5 and 50 URLs, but a typical number may be 10–20 core pages. Better to point to a few very relevant pages, rather than bombarding the AI with everything (that’s what your sitemap is for). It’s like giving an intrepid AI explorer a map with Xs on it that says ‘Dig here,’” said SEO expert Carolyn Shelby.[61] Choose your X marks the spot carefully.

2. Collect Titles and Descriptions:

For each page you selected: - Write a short, descriptive title (if the title of the page you selected is long or not self explanatory). This could be just the guide or FAQ name. e.g. "Installation Guide" or "Pricing FAQ". - Optionally, enter a one-line summary or note about what that page is about. Keep it short – a phrase or a sentence that conveys the idea. For example, “How to set up step by step for new users” or “Frequently asked questions about billing and pricing”.

If the title of the page is clear enough, you may just use that for the link text and not have any further description or only a short qualifier.

3. (Optional) Organise into Sections:

If you have different categories of content it’s useful to group them under headings in the llms.txt. For example, if you have docs and blog posts you might want to do the following:

## Documentation

- [Getting Started](...) - översikt

- [User Manual](...) – description

## Blog Highlights

- [Case Study:...](...) – description

- [Industry Report 2025](...) - summary

Sections (H2’s in Markdown, denoted by ##) assist in structuring the file. But if all you have is a few pages and they are all doing something similar, you might as well just list them all without sections. The llmstxt.org proposal states that the only mandatory part is the top H1 title, and you may have zero or more H2 sections for lists of links [40][41]. So sections are optional but recommended if they make sense for you.

4. Summary of your site:

At top you should add a short paragraph about your site or project as a blockquote. This is basically your site’s elevator pitch for an AI. One or two sentences will do. For example,

> AcmeAnalytics is a visualising data platform that enables marketing teams to convert raw data into interactive dashboards in minutes.

This will be the first thing in the file (after the H1 title which is just the name, e.g. # AcmeAnalytics Docs). This summary provides important context [56][57]. Think about what a user would ask an AI about your site – e.g., “What is AcmeAnalytics?” – this summary would allow the AI to answer that directly or at least know the domain of your site.

5. Create the file in Markdown

Now let’s put it all together in a plain text file. - Line 1: # [Your Site/Project Name]

- Then a blockquote with your short site description (start the line with >) - If desired, an introduction or note (e.g., “Below are key resources for understanding and using AcmeAnalytics.”) – this can be regular text or also part of the blockquote. - and then your sections and links as discussed.

For example, a simplified llms.txt might be:

### AcmeAnalytics Docs

> AcmeAnalytics is a tool for visualising marketing data. This file describes some of the most important resources for allowing artificial intelligence systems to quickly find relevant information.

## How To’s

- [Getting Started Guide](https://acmeanalytics.com/docs/getting-started): See how to install AcmeAnalytics and build your first dashboard.

- [User Manual](https://acmeanalytics.com/docs/user-manual): Full documentation of all features.

## Hilfe-Center

- [Dashboard FAQ](https://acmeanalytics.com/help/dashboard-faq): Frequently asked questions about building and customising dashboards.

- [Account & Billing FAQ](https://acmeanalytics.com/help/billing-faq): Common questions about account creation, subscriptions, and billing.

## Optional

- [API Reference](https://acmeanalytics.com/docs/api) – for custom integrations (advanced)

- [Change log](https://acmeanalytics.com/docs/changelog)

Note: - We used a “Optional” section (with H2 title “Optional”) for less critical links [42] . According to the spec , if an AI has a tight context and needs to drop something , it can drop the Optional section . We made API and Changelog optional because not every Ai query will require them. - Links follow a consistent format - [Title](URL): Description. It needs the link part in markdown. You can easily add a description by adding a colon and a space after the link . - We kept descriptions fairly brief.

6. llms.txt Save As:

You can create the file with any plain text editor you like (VSCode, Notepad, Sublime, etc. Save it as exactly llms.txt (all lowercase). Your job is to humanise the following text in English language while retaining its meaning and tone. Do not add or remove any information. Do not add any other text to the output. Encoding is important: if a file is not properly encoded, AI might read it incorrectly. Indeed, one challenge noted is that if the file is not UTF-8, LLMs may interpret characters incorrectly or reject it [62].

7. Upload to your website's root:

Upload the llms.txt file to the root of your domain via FTP, SFTP, your web host's file manager, or your site's deployment process. So if someone visits https://yourwebsite.com/llms.txt they should see the content. This is where your robots.txt is.

8. Give it a try:

Open a browser (or a command-line tool like curl) and grab yourwebsite.com/llms.txt. Make sure it loads and shows the content as you expect (some browsers won't render Markdown fully, but you can at least see the text). Check the file for broken links or typos.

And that’s it for manual creation! If your site content changes infrequently you may be done. But if you are frequently adding important content (like new help articles, new guides), then you will have to remember to update llms.txt accordingly. This is where automation saves the day.

Automating llms.txt creation with tools

For larger, more active sites, manual maintenance of llms.txt might become a chore. Fortunately, there are some tools and plugins emerging that help automate this process:

1. Yoast SEO (WordPress Plugin)

Yoast SEO, a popular SEO plugin, has added native support for llms.txt for WordPress sites. Here’s how it works: - One-

click activation: In the Yoast SEO settings, under “Site Features”, you will see an option for “AI discovery file (llms.txt)”. You simply switch it on[49][63]. - If enabled, Yoast will automatically generate a llms.txt file for you. It will generate a virtual file for you to access at yourdomain.com/llms.txt without you having to upload one. - Weekly updates: Yoast regenerates the file every week using a WP cron job49. This means if you’ve published new posts or pages it finds relevant, they’ll be added. - Smart content selection – Yoast tries to pick up your most important content, likely based on post types or maybe those with good internal linking. It talks about auto-detecting your latest blog posts, product guides, documentation, etc., so the artificial intelligence can get fresh and relevant content [49]. - Preview: You can preview what Yoast is putting in llms.txt before it goes live [64] . This is useful to verify it is listing what you expect.

For WordPress users, Yoast is by far the easiest. It’s basically doing the same thing to llms.txt that it does to sitemap.xml (which Yoast auto generates as well). Given Yoast’s popularity, this could go a long way in getting llms.txt used across the web. Yoast also ensures proper formatting, encoding and automatically excludes noindex pages (for example, pages you’ve marked noindex or blocked in robots.txt won’t be listed by Yoast, to prevent hidden content from being exposed [65]).

How to enable in Yoast (quick steps) [66][63] - Update Yoast SEO to the latest version (llms.txt was added in late 2024 via Yoast’s integration with this standard). - Go to Yoast -> Settings in your WordPress admin. - Look for “Site Features” section or search for “llms.txt” in Yoast settings. - Enable the toggle for AI discovery file (llms.txt). - Save changes. - That's all. From there, you can visit yourdomain.com/llms.txt to see the file (it might take a few minutes, or until the next cron run, for it to show up; Yoast says it uses weekly cron, but it probably generates immediately on toggle, then schedules weekly updates). The file might show things like a list of your recent blog posts, main pages, etc. If you want to refine what’s included, Yoast may not allow manual selection at the moment (it automates it), but you could give them feedback. Also, make sure that your important content is easy to find on the site; Yoast probably assumes that well-linked or high-priority content will be included.

2. Other CMS Plugins or Services:

Yoast is a big name here, but there are others catching up. Check if your CMS or tech stack has similar plugins: - For instance, some SEO SaaS or tools could have a llms.txt generator. The search results include mentions of a guide on Keploy [67] named “LLMs.txt Generator”, which could suggest some interest in the idea. - There are scripts on GitHub that will generate llms.txt based on a list of URLs or your sitemap by selecting the top N links. - If you use a static site generator (Jekyll, Hugo, etc.) you can probably write a template to generate llms.txt during your build, taking in all pages tagged “featured” or something similar.

3. Custom Automation

If it is not in the box for your context then you can script it. - Get all your site’s pages or posts from your site’s API or database. - Filter to ones that match your criteria (e.g. published content, not private, maybe filter by category or tag “important”) - Markdown format with sections you want. - Write it to a file and put it on the root of your site.

That requires developer skills, but it’s doable. It’s pretty much like building a custom sitemap, only with a hand-picked subset and a bit more text.

Be aware of the challenges of manual vs automated: - A fully automated approach may not “know” which content is truly best without guidance. For example, an automated selection of the latest posts might miss an older evergreen FAQ that is super important. Manual curation, however, can grow stale (failing to include a newly important page). A hybrid approach can work: automate the base list, but allow manual tweaks.

One good automated strategy is to use existing signals of importance: - If your site is hierarchical (docs-like), perhaps include all top-level docs pages. - Pages in the top percentiles of engagement or traffic, if you use analytics. - If you’ve got cornerstone content designated (Yoast has a feature for marking “cornerstone content”), use that.

Yoast’s weekly refresh approach is clever, it covers content updates without constant manual involvement [49]. They also mention that Yoast’s generation ensures the file has only indexable pages and uses proper UTF-8 encoding and Markdown formatting[65], taking care of technical pitfalls for you.

4. Online Generators (Soon):

At the time of writing llms.txt is so new that there aren’t many dedicated online generators yet. But we can imagine simple tools where you input your site name, description, and paste a list of important URLs with descriptions, and it outputs the formatted llms.txt content. If you don’t want to code Markdown by hand, you could do it in Excel by making a list and then concatenating it into a text format.

You could also use the very same thing you're using to write this, namely, ask it to propose a llms.txt structure for your site given a list of pages. (But be sure to check for accuracy.)

How to avoid the usual mistakes

· Formatting Counts: As mentioned before, a messy format could confuse AI. Make sure:

# One H1 (single #) at the top.

· Use correct Markdown link syntax [text](URL).

Start each link line with a dash or asterisk to make it a list item.

· Use headings for the links themselves. They should be list items under headings.

· The file must be plain text. Do not use fancy characters (unless part of content). Make sure quotes or dashes are plain text if you copy-paste titles (sometimes word processors insert curly quotes or en dashes that might render oddly).

Validate by opening the file in a Markdown viewer or just visually check that it’s consistent.

Keep it focused: don't list hundreds of urls. This is not a second sitemap. Models have context limits. If you overfill them then you defeat the purpose. The idea is to give them a high-value overview in bite-sized pieces. One guide advises making llms.txt files “concise and expert-level”[34][68].

Frequent Updates vs Static: If your site’s main content changes often, use automation (like Yoast). If it doesn't, a static file created by hand is fine, just don't forget about it. Maybe set a calendar reminder to review it every couple of months or when you launch major content.

Don’t Add Irrelevant Content: Sure, it’s tempting to add promotional pages or any other less-relevant things thinking “maybe AI will send traffic there.” But remember, the goal is not to market to AI, it’s to help AI help users. In contrast, listing irrelevant content might just waste the AI’s context window, or worse, lead it to provide sub-optimal answers. For example, do not include your 'Careers' page or 'Contact us' – these are seldom useful to a user's informational query. Focus on content-rich pages.

Coordinate with Robots/Sitemap: Ensure pages in llms.txt are accessible (not blocked) and are pages you’d want in search results too (you don’t noindex those), as noted. Ensure that if something is truly meant to be confidential or behind a paywall, it’s not in llms.txt (that could lead to AI accessing and potentially exposing that info inadvertently).

How to Check If AI Is Using Your llms.txt

This is tricky because artificial intelligence systems aren't transparent. However: - Check the server logs for hits to /llms.txt. Preferably identify the user agent. If you see something like Mozilla/5.0 (compatible; ChatGPT-User) or GPTBot, it means OpenAI’s bot might be pulling it. If you see Google-Extended or similar, that could be Google’s AI crawler (Google has an AdsBot and a new Google-Extended user agent for Bard/AI stuff)[58]. - If you have a log analyser, filter the requests to llms.txt by user agent to see which bots are hitting it. - Another indicator if you are working with AI tools and they appear to have up-to-date knowledge of your content quoting maybe your site tagline or your site summary. For example, if you ask ChatGPT (with browsing) something about your site and it responds with text from your llms.txt summary, then you know it found and used it. - As AI platforms improve, they could offer interfaces for site owners (e.g. OpenAI or others may have a 'webmaster' portal similar to Google Search Console for AI ingestion - not present at the moment, but we can hope).

Tool Mention: Amrut.AI (Hypothetical)

Since this blog series is sponsored by or meant to promote an AI tool, it’s worth mentioning that there are some emerging services that are trying to make these kinds of tasks easier. For example, Amrut.AI (our company) is developing AI-driven solutions to automatically generate files like llms.txt, sitemap.xml, and robots.txt for your site. It’s about managing all these technical SEO/AI optimisations in one place. Instead of juggling plugins or manual edits, soon enough you could simply provide your site’s URL to a platform like Amrut.AI and it would analyse your content and generate an optimised llms.txt file for you – and keep it updated as your site grows.

At present, Yoast and the like generate llms.txt but a generic tool that is not platform-dependent might help sites on any tech stack. If you’re not using WordPress or want more control over how content is selected, watch for services like this. An AI-driven generator has the advantage that it can potentially find popular or semantically important pages you may have missed, making sure your llms.txt really reflects what people are asking about.

(The above is an instance of how you might integrate with a promotional angle, but practically speaking, watch for SEO tools to add llms.txt support. Bing for example might build it into their webmaster tools – who knows.)

Last Steps

Now that llms.txt is implemented, it is mostly a waiting game to see it bear fruit in AI interactions. You can fetch and index with search engines but with llms.txt, you don’t “submit” it anywhere – you just host it and AI crawlers will find it when they crawl your site (some may find it via your robots.txt or sitemap links, others may just try the URL convention, and certainly the SEO community has been evangelising it so major AI players are likely aware to look for it).

To put it briefly, a llms.txt file can be as easy or difficult as you want it to be. – For control, do it manually and curate your content. - If you’re on a supported platform and want convenience, enable a plugin like Yoast and let it do it for you. - If you are in the middle somewhere, use a generator or script to help you but always verify output to make sure it matches what you think is the best content for your site.

Use this guide and you'll have your llms.txt file up and running, joining early adopters in this brave new SEO frontier. It’s a small task that could pay big dividends in making sure AI models “see” your site the way you want them to. You've set your site up to be part of those answers, with llms.txt acting as your ambassador within algorithms, as more users search for answers via AI assistants. Have fun tuning!