CrawlReadable
Certificate of AI readabilityNo. CR-20261019-robusty.io

Can AI crawlers read robusty.io?

AI crawlers can find and read this site well.

Site inspected
robusty.io
Fetched from
https://www.robusty.io/
Date
2026-10-19
Checks failed
3 of 20

Score by category

  • Access25/26
  • Content36/40
  • Structure20/20
  • Discovery6/10
  • Interface4/4

What an AI crawler gets

The first 400 characters of text in the HTML, before any JavaScript runs.

Robusty: Stop Testing Your App by Hand How it works Pricing Docs Book a demo Start for free Start for free Stop testing your app by hand Robusty is an AI-powered testing platform that runs your plain-English tests in a real browser to catch broken flows. Start for free Book a demo No credit card required R Move fast without breaking things In the AI era, you're building faster than ever, but ...

2,087 characters of readable text

Which AI crawlers are allowed

Crawlerrobots.txt says
GPTBotAllowed
OAI-SearchBotAllowed
ClaudeBotAllowed
Claude-SearchBotAllowed
PerplexityBotAllowed
Perplexity-UserAllowed
Google-ExtendedAllowed
CCBotAllowed
BytespiderAllowed
AmazonbotAllowed

Findings, biggest first

Each finding shows the points it would add to the score once fixed.

  1. Finding 1

    Add real copy to the page

    Content, about 1 hour

    Readable text is a tiny share of the page's code, or there's very little of it. Crawlers summarise words, not markup.

    +4points
  2. Finding 2

    Keep sitemap dates fresh

    Discovery, about 1 hour

    No lastmod date in the last 90 days, so crawlers can't tell what changed and may not come back.

    +4points
  3. Finding 3

    Mirror llms.txt at /.well-known/llms.txt

    Access, about 5 min

    Some tools look in the .well-known folder instead. A copy of the same file covers both.

    +1points

Get told when this changes

We're building monitoring: a re-score every week and an email when the score moves. Leave your email to get it first.

Show your score

Add this badge to your site or README. It always shows the latest score.

CrawlReadable score for robusty.io

17 checks passed
  • Publish a robots.txt file (4 points)
  • Let GPTBot crawl the site (6 points)
  • Let ClaudeBot crawl the site (6 points)
  • Let PerplexityBot crawl the site (6 points)
  • Add an llms.txt file (3 points)
  • Put your words in the HTML (18 points)
  • Stop serving an empty app shell (10 points)
  • Give the page an h1 heading (4 points)
  • Use semantic HTML tags (4 points)
  • Add structured data (JSON-LD) (6 points)
  • Describe your organisation in JSON-LD (4 points)
  • Write a descriptive page title (3 points)
  • Add a meta description (3 points)
  • Add Open Graph tags (2 points)
  • Add a canonical link (2 points)
  • Publish a sitemap.xml (6 points)
  • Offer a machine-readable interface (4 points)