CrawlReadable
Certificate of AI readabilityNo. CR-20261018-example.com

Can AI crawlers read example.com?

AI crawlers miss most of this site.

Site inspected
example.com
Fetched from
https://example.com/
Date
2026-10-18
Checks failed
14 of 20

Score by category

  • Access18/26
  • Content14/40
  • Structure3/20
  • Discovery0/10
  • Interface0/4

What an AI crawler gets

The first 400 characters of text in the HTML, before any JavaScript runs.

Example Domain This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.

171 characters of readable text, under the 400 we count as readable

Which AI crawlers are allowed

Crawlerrobots.txt says
GPTBotAllowed
OAI-SearchBotAllowed
ClaudeBotAllowed
Claude-SearchBotAllowed
PerplexityBotAllowed
Perplexity-UserAllowed
Google-ExtendedAllowed
CCBotAllowed
BytespiderAllowed
AmazonbotAllowed

Findings, biggest first

Each finding shows the points it would add to the score once fixed.

  1. Finding 1

    Put your words in the HTML

    Content, needs a developer

    A crawler that runs no JavaScript found under 400 characters of text. Whatever your page says after scripts run, it doesn't say to them.

    +18points
  2. Finding 2

    Add structured data (JSON-LD)

    Structure, about 1 hour

    Machine-readable facts about your business: name, logo, what you sell. AI systems use it to describe you accurately.

    +6points
  3. Finding 3

    Publish a sitemap.xml

    Discovery, about 1 hour

    A list of every page you want found. Without one, crawlers only find what's linked.

    +6points
  4. Finding 4

    Publish a robots.txt file

    Access, about 5 min

    Crawlers look here first for the rules of your site. With no file, every bot guesses.

    +4points
  5. Finding 5

    Give the page an h1 heading

    Content, about 5 min

    One clear main heading tells a crawler what the page is about.

    +4points
  6. Finding 6

    Use semantic HTML tags

    Content, about 1 hour

    Tags like main, nav, article and footer show which part is content and which is chrome.

    +4points
  7. Finding 7

    Describe your organisation in JSON-LD

    Structure, about 1 hour

    Organization or WebSite objects tie your site to a known entity, so answers name you correctly.

    +4points
  8. Finding 8

    Keep sitemap dates fresh

    Discovery, about 1 hour

    No lastmod date in the last 90 days, so crawlers can't tell what changed and may not come back.

    +4points
  9. Finding 9

    Offer a machine-readable interface

    Interface, needs a developer

    A valid /openapi.json or /llms-full.txt lets AI agents use your product or read your docs in full, not just your homepage.

    +4points
  10. Finding 10

    Add an llms.txt file

    Access, about 1 hour

    A short Markdown summary of your site at /llms.txt. Cheap to add, but few AI systems request it yet, so it's worth 3 points.

    +3points
  11. Finding 11

    Add a meta description

    Structure, about 5 min

    A one-sentence summary that crawlers and link previews reuse.

    +3points
  12. Finding 12

    Add Open Graph tags

    Structure, about 5 min

    og:title, og:description and og:image control how your link looks when shared, including in AI chat apps.

    +2points
  13. Finding 13

    Add a canonical link

    Structure, about 5 min

    Tells crawlers which URL is the real one when the same page is reachable at several.

    +2points
  14. Finding 14

    Mirror llms.txt at /.well-known/llms.txt

    Access, about 5 min

    Some tools look in the .well-known folder instead. A copy of the same file covers both.

    +1points

We'll write these fixes for you

Send us your domain and we'll write the fixes, ready to paste:

  • an llms.txt file written for your site
  • the robots.txt rules to let AI crawlers in
  • Organization and WebSite structured data (JSON-LD)
  • a plain-English note on anything only JavaScript shows, and how to fix it

Delivered by email within 2 business days. We re-score your site once you've applied it. See a sample fix pack.

$149

one site, one payment, charged as R2,450 (South African rand)

Get the fix pack

Payments by Paystack

Money-back guarantee. Full refund if we miss the 2-day delivery, or if your score doesn't go up after you apply the fixes within 30 days. Refund policy

Get told when this changes

We're building monitoring: a re-score every week and an email when the score moves. Leave your email to get it first.

Show your score

Add this badge to your site or README. It always shows the latest score.

CrawlReadable score for example.com

6 checks passed
  • Let GPTBot crawl the site (6 points)
  • Let ClaudeBot crawl the site (6 points)
  • Let PerplexityBot crawl the site (6 points)
  • Stop serving an empty app shell (10 points)
  • Add real copy to the page (4 points)
  • Write a descriptive page title (3 points)