What this AI crawler checker does
It fetches your page twice: once the way GPTBot does (the HTML your server sends, no JavaScript), and once in a real browser, the way visitors see it. Then it lists every heading and paragraph that only appears after JavaScript runs, checks whether your title, meta description, H1, structured data and links are in the HTML AI reads, and reads your robots.txt for each AI crawler.
Why it matters for AI answers
- AI crawlers don't run JavaScript. A React, Vue or Angular page built in the browser can look complete to you and nearly empty to ChatGPT, Claude and Perplexity.
- What isn't read can't be cited. If your pricing, features or FAQ load with a script, AI answers about your product are written without them.
- Firewalls block AI crawlers quietly. Many CDNs turn AI bots away by default, so a page can be open in robots.txt and still unreachable.
Frequently asked questions
Does GPTBot render JavaScript?
No. GPTBot, ClaudeBot and PerplexityBot read the HTML your server sends. Content a script adds afterwards isn't part of what they read. Googlebot does render JavaScript, but later and not always.
How do I make my page readable by AI crawlers?
Put the text in the HTML your server sends: server-side rendering or prerendering (Next.js, Nuxt, Astro, Angular SSR all do this). Serve the same content to everyone. Showing crawlers a different page from visitors is cloaking, which Google penalizes.
Should I block GPTBot in robots.txt?
GPTBot collects training data. Blocking it is your choice and doesn't remove you from ChatGPT search, which uses OAI-SearchBot and ChatGPT-User. Blocking those, or PerplexityBot, keeps your pages out of their answers.
Is this tool free?
Yes, a few pages a day with no sign-up. Sign up free to check every page of your site.