How to Set Up llms.txt for Your Web3 Project (2026 Guide)

The llms.txt file is a Markdown document placed at your website root that gives AI models a structured, concise overview of your project. Think of it as robots.txt for AI comprehension — instead of telling crawlers what to crawl, it tells AI models what your project is, what it does, and where to find more information.

The specification was proposed by Jeremy Howard (founder of fast.ai) and is documented at llmstxt.org. Major companies including Anthropic, Cloudflare, Stripe, and many others have already deployed llms.txt files. For Web3 projects, it's a critical piece of LLMO (Large Language Model Optimization) infrastructure.

This guide by Astral (astral3.io) walks you through creating and deploying llms.txt for your crypto project — with real examples and Web3-specific recommendations.

Why Does llms.txt Matter for Web3 Projects?

AI models face a fundamental limitation: context windows are too small to process entire websites. Most websites have complex HTML, JavaScript, navigation, and ads that make it hard for LLMs to extract meaningful information.

The llms.txt file solves this by providing:

• Concise, structured information in a format LLMs can easily process
• A single access point for all essential project information
• Human and machine readable Markdown format
• Links to detailed pages for deeper information

For Web3 projects specifically, this is critical because many crypto sites are JavaScript-heavy SPAs that AI crawlers can't render. The llms.txt file gives AI models the information they need regardless of your site's technical stack.

The llms.txt Specification

According to the official spec, an llms.txt file contains these sections in order:

1. H1 heading — The name of your project
2. Blockquote — A brief, one-paragraph description of what the project is
3. H2 sections — Organized information about different aspects of the project
4. Links — Markdown links to detailed documentation or pages

Additionally, the spec proposes:

• /llms.txt — The concise version at the site root
• /llms-full.txt — An expanded version with complete details
• Optional: .md extensions on existing pages for clean Markdown versions

Step 1: Create Your llms.txt (Concise Version)

Here's a template for a Web3 project:

# YourProject

> YourProject is a [category] protocol on [chain] that [what it does in one sentence]. [Key differentiator]. [Key metric if available].

## About

[2-3 sentences expanding on what the project does, who it's for, and why it matters.]

- Website: https://yourproject.com
- Documentation: https://docs.yourproject.com
- Twitter: https://x.com/yourproject
- Discord: https://discord.gg/yourproject

## Key Features

- [Feature 1 with brief explanation]
- [Feature 2 with brief explanation]
- [Feature 3 with brief explanation]

## How It Works

[Brief technical explanation — 2-3 sentences]

## Token

- Token: $TOKEN
- Chain: [Chain name]
- Contract: [Contract address]

## Team

- [Founder name] — [Role]. [LinkedIn or Twitter]
- [Co-founder name] — [Role]. [LinkedIn or Twitter]

## Links

- [Documentation](https://docs.yourproject.com)
- [Audit Report](https://link-to-audit)
- [Whitepaper](https://link-to-whitepaper)
- [CoinGecko](https://coingecko.com/en/coins/yourtoken)
- [DefiLlama](https://defillama.com/protocol/yourproject)

Step 2: Create Your llms-full.txt (Detailed Version)

The full version should include everything from the concise version, plus:

• Detailed problem statement: What problem does the project solve and why does it matter?
• Technical architecture: How the protocol works at a technical level
• Comparison tables: How you compare to alternatives (LLMs love tables)
• FAQ section: Questions and answers matching real user queries
• Use cases: Specific examples of how the project is used
• Metrics: TVL, users, transactions, growth data
• Security: Audit information, security measures
• Roadmap: Key upcoming milestones

Step 3: Deploy to Your Site Root

The files must be served at:

• https://yoursite.com/llms.txt
• https://yoursite.com/llms-full.txt

For static sites / Next.js / Astro

Place the files in your public/ directory. They'll be served as-is.

For React SPAs (Vite, Create React App)

Place files in public/, but make sure your router isn't catching these URLs. If you use a catch-all rewrite (common for SPAs), exclude the static files:

// vercel.json example
{
  "rewrites": [
    { "source": "/llms.txt", "destination": "/llms.txt" },
    { "source": "/llms-full.txt", "destination": "/llms-full.txt" },
    { "source": "/((?!llms\\.txt|llms-full\\.txt).*)", "destination": "/index.html" }
  ]
}

Headers

Serve with these headers for best compatibility:

• Content-Type: text/plain; charset=utf-8
• Cache-Control: public, max-age=86400 (cache for 24 hours)

Step 4: Verify Your Deployment

After deployment, verify with curl:

curl -sI https://yoursite.com/llms.txt

You should see:

• HTTP status: 200
• Content-Type: text/plain; charset=utf-8
• No redirects (especially important if you have www/non-www redirects)

Then verify the content:

curl -s https://yoursite.com/llms.txt | head -5

Who Already Uses llms.txt?

Common Mistakes to Avoid

1. SPA catch-all intercepting the file: Your llms.txt returns your app's HTML instead of the Markdown. Test with curl, not your browser.
2. Wrong Content-Type: Serving as text/html instead of text/plain.
3. Too promotional: AI models prefer factual, balanced information. State what you do clearly, but don't make every sentence a sales pitch.
4. Too vague: "We are a leading blockchain platform" tells AI nothing. Be specific: "YourProject is a lending protocol on Ethereum with $200M TVL specializing in RWA collateral."
5. Forgetting the blockquote: The one-paragraph summary in a blockquote (>) is the most important part — it's often what AI models extract first.
6. No links to sources: Include links to docs, audits, aggregators. AI models may follow these for additional context.

Get Your Free AI Visibility Audit →