State of AI readiness
Across the 18 sites scored on CrawlReadable so far, the average score is 74/100. 28% block OpenAI's GPTBot in robots.txt, and 6% serve under 400 characters of text to crawlers that don't run JavaScript.
74average score out of 100
28%block GPTBot
28%block ClaudeBot
6%serve too little text without JavaScript
Most common fixes
- Mirror llms.txt at /.well-known/llms.txt: 100% of sites
- Add real copy to the page: 89% of sites
- Offer a machine-readable interface: 83% of sites
- Keep sitemap dates fresh: 56% of sites
- Describe your organisation in JSON-LD: 56% of sites
About this data
This page updates with every audit. It covers the latest score of each site anyone has checked here, so it leans towards sites people chose to test, not a random sample of the web. Sites that answered with an error or bot challenge have no score and aren't counted.