Can AI crawlers read calmparent.app?
AI crawlers miss most of this site.
- Site inspected
- calmparent.app
- Fetched from
- https://calmparent.app/
- Date
- 2026-10-17
- Checks failed
- 12 of 20
AI crawlers miss most of this site.
The first 400 characters of text in the HTML, before any JavaScript runs.
Liviana — First aid for parenting
33 characters of readable text, under the 400 we count as readable
| Crawler | robots.txt says |
|---|---|
| GPTBot | Allowed |
| OAI-SearchBot | Allowed |
| ClaudeBot | Allowed |
| Claude-SearchBot | Allowed |
| PerplexityBot | Allowed |
| Perplexity-User | Allowed |
| Google-Extended | Allowed |
| CCBot | Allowed |
| Bytespider | Allowed |
| Amazonbot | Allowed |
A crawler that runs no JavaScript found under 400 characters of text. Whatever your page says after scripts run, it doesn't say to them.
Machine-readable facts about your business: name, logo, what you sell. AI systems use it to describe you accurately.
A list of every page you want found. Without one, crawlers only find what's linked.
Readable text is a tiny share of the page's code, or there's very little of it. Crawlers summarise words, not markup.
One clear main heading tells a crawler what the page is about.
Tags like main, nav, article and footer show which part is content and which is chrome.
Organization or WebSite objects tie your site to a known entity, so answers name you correctly.
No lastmod date in the last 90 days, so crawlers can't tell what changed and may not come back.
A valid /openapi.json or /llms-full.txt lets AI agents use your product or read your docs in full, not just your homepage.
A short Markdown summary of your site at /llms.txt. Cheap to add, but few AI systems request it yet, so it's worth 3 points.
Tells crawlers which URL is the real one when the same page is reachable at several.
Some tools look in the .well-known folder instead. A copy of the same file covers both.