What Is llms.txt?
An llms.txt file is a simple Markdown document that sits in the root of your website, alongside files such as robots.txt and your XML sitemap.
Instead of providing instructions for search engine crawlers, it helps AI systems understand your website. The file gives large language models a concise summary of your business, explains what your website offers, and points them towards your most important pages. This approach reduces the need for AI systems to work through navigation menus, JavaScript and other page elements before understanding your content.
Jeremy Howard of Answer. AI introduced the format in September 2024. You can find the full specification at llmstxt.org if you’d like to explore the technical details.
Does It Improve AI Search?
The honest answer is not yet, at least not in any confirmed way.
No major AI provider has publicly confirmed that it reads llms.txt when generating responses. Google’s John Mueller has also described the file as a “temporary crutch” for AI coding tools rather than something designed to improve search visibility.
In other words, you shouldn’t expect an llms.txt file on its own to make ChatGPT or other AI assistants reference your website more often.
However, the file does provide measurable value today.
Google includes llms.txt as one of the default checks within Lighthouse’s Agentic Browsing category in PageSpeed Insights. According to Google’s documentation, AI agents need to spend more time understanding your website when the file doesn’t exist. Lighthouse highlights that missing optimisation in its report.
Adding the file won’t guarantee more AI visibility, but it takes very little time to implement. It can improve one of your PageSpeed Insights categories and prepares your website if the standard becomes more widely adopted. That’s a small investment with very little risk.
What Does an llms.txt File Contain?
A typical llms.txt file follows a straightforward structure.
It usually includes:
- An H1 heading with your business name.
- A short Markdown blockquote describing your business.
- Links to important pages, such as your services, about page and contact page.
Keep the content factual and concise. Avoid marketing phrases such as “industry-leading” or “best-in-class”. Current guidance encourages plain, verifiable information rather than promotional copy.
Should You Add One?
If you use WordPress with Yoast SEO, recent versions can generate an llms.txt file automatically from your existing content. For many websites, that’s the quickest and easiest solution.
If your website doesn’t generate one automatically, creating the file manually usually takes no more than a few minutes once you understand the format.
I include this type of technical optimisation in every website I build, alongside improvements for the other Lighthouse categories, including performance, accessibility, best practices, and SEO. On its own, llms.txt is a small enhancement. Combined with other technical improvements, it helps create a faster, better-structured website that’s ready for both search engines and AI tools.
If your website has never had a proper technical audit, this is just one of many improvements worth checking.
Want to Know Where Your Website Stands?
If you’re unsure how your website performs from a technical perspective, get in touch. I’ll carry out a detailed review, explain the findings in plain English and recommend practical improvements that make a genuine difference.
