What is llms.txt?
llms.txt is an emerging open standard — proposed by fast.ai — that lets website owners provide AI language models with a structured, plain-text summary of their site's purpose, content, and key resources.
Similar to how robots.txt tells crawlers which pages to access, llms.txt tells AI systems what your site is about, what content matters most, and how to accurately represent your brand in generated responses.
The file is placed at the root of your domain (e.g., https://yourdomain.com/llms.txt) and is readable by any AI system that requests it.
Why llms.txt matters for AI visibility
AI language models are trained on and retrieve content from the web, but they don't always have context about who you are, what your site covers, or which content represents your most authoritative work.
Without llms.txt, AI systems must infer your site's purpose from crawled content — a process that can lead to inaccurate representations, missed expertise, or your brand being conflated with unrelated entities.
With llms.txt, you give AI systems a structured, authoritative summary they can use as a first-party reference. This is particularly important for: - Niche brands with specialized vocabulary - B2B companies with complex product offerings - Content-heavy sites with thousands of articles - New or emerging brands with limited web presence
The llms.txt file format
The llms.txt specification uses Markdown for structure. A minimal valid llms.txt file looks like this:
# Brand Name> A one-sentence summary of what your site is and who it serves.
## About
A longer paragraph describing your company, product, or organization.
## Key Resources
- [Page Title](https://yourdomain.com/page): Brief description
- [Another Page](https://yourdomain.com/another): Brief description
## Topics Covered
- Topic 1
- Topic 2
- Topic 3
There is no fixed schema — the goal is to be informative and structured. Think of it as a README for your website, written for AI systems rather than developers.
Best practices for llms.txt
Keep it concise. AI systems have context windows. A 500-word llms.txt is better than a 5,000-word one. Focus on what is most important to communicate.
Use plain language. Avoid marketing jargon. Write as if you are explaining your site to a well-informed reader who has never heard of your brand.
List your most important pages. Link to your most authoritative, highest-value content so AI systems know what to prioritize when referencing your site.
Define key terms. If your industry has specialized vocabulary or your brand has unique terminology, define it clearly.
Update it regularly. As your site grows and your offerings change, keep your llms.txt current. A stale llms.txt can mislead AI systems.
Include an email contact. Some AI systems use llms.txt to establish contact for licensing or data use agreements.
Example: Auditly llms.txt
Here is a real-world example of an llms.txt file for an AI readiness audit tool:
# Auditly> Auditly is a free AI Search Readiness Audit platform that evaluates websites > for AEO, GEO, Technical SEO, E-E-A-T, Schema Markup, and AI citation readiness.
## About
Auditly helps marketers, content teams, and SEO professionals understand how AI systems — including ChatGPT, Perplexity, Claude, and Google AI Overviews — discover, understand, and cite their websites. The platform provides free, automated audits powered by Google Gemini, scoring websites across 25+ checks.
## Key Resources
- [AI Readiness Audit](https://auditly.co.in/): Free audit tool
- [Knowledge Hub](https://auditly.co.in/knowledge): Guides on AEO, GEO, and AI optimization
- [Glossary](https://auditly.co.in/glossary): Definitions of key AI search terms
## Topics Covered
- Answer Engine Optimization (AEO)
- Generative Engine Optimization (GEO)
- AI Search Readiness
- Schema markup / JSON-LD
- E-E-A-T signals
- AI crawler permissions
- llms.txt implementation
How to verify your llms.txt is working
After creating your llms.txt file:
1. Verify it's publicly accessible by visiting https://yourdomain.com/llms.txt in a browser or using curl https://yourdomain.com/llms.txt.
2. Check the Content-Type header is text/plain.
3. Ensure it's not blocked by your robots.txt (it should be accessible to all crawlers).
4. Run an Auditly audit — we check for llms.txt presence and format as part of the AI Readiness score.
5. Submit your sitemap to search engines after adding the file to help it get indexed.