# CrawlReadable

> CrawlReadable scores how readable a website is to AI crawlers such as GPTBot, ClaudeBot and PerplexityBot. It fetches a homepage the way most AI crawlers do, as raw HTML with no JavaScript, runs 20 checks worth 100 points, and lists fixes ranked by points.

## Tool

- [Score a site](https://crawlreadable.com/): free, no signup
- [API](https://crawlreadable.com/openapi.json): GET /api/audit?url=<domain> returns the full report as JSON
- [Leaderboard](https://crawlreadable.com/leaderboard): latest score for every site checked
- [State of AI readiness](https://crawlreadable.com/state-of-ai-readiness): live aggregate data

## Guides

- [Methodology](https://crawlreadable.com/methodology): every check, weight and limit
- [AI crawlers and robots.txt: every user agent and what it controls](https://crawlreadable.com/guides/ai-crawlers-robots-txt): The user agents OpenAI, Anthropic, Perplexity, Google and Apple use, what blocking each one does, and copy-paste robots.txt rules.
- [llms.txt: what it is, who reads it, and a template](https://crawlreadable.com/guides/llms-txt): What the llms.txt proposal is, what the 2026 adoption data says about whether AI systems request it, and a template you can copy.
- [JavaScript and AI crawlers: why single-page apps look empty](https://crawlreadable.com/guides/javascript-ai-crawlers): Why most AI crawlers see a blank page on JavaScript-rendered sites, how to tell if yours is affected, and the fixes for React, Vue and other frameworks.