Startups All Tools Full Site Audit
Free AI crawler check

See your page the way GPTBot does

GPTBot, ClaudeBot and PerplexityBot don't run JavaScript. Enter a page and we'll read it twice, as an AI crawler and as a visitor, and show you exactly what AI misses.

Try:

What this AI crawler checker does

It fetches your page twice: once the way GPTBot does (the HTML your server sends, no JavaScript), and once in a real browser, the way visitors see it. Then it lists every heading and paragraph that only appears after JavaScript runs, checks whether your title, meta description, H1, structured data and links are in the HTML AI reads, and reads your robots.txt for each AI crawler.

Why it matters for AI answers

Frequently asked questions

Does GPTBot render JavaScript?

No. GPTBot, ClaudeBot and PerplexityBot read the HTML your server sends. Content a script adds afterwards isn't part of what they read. Googlebot does render JavaScript, but later and not always.

How do I make my page readable by AI crawlers?

Put the text in the HTML your server sends: server-side rendering or prerendering (Next.js, Nuxt, Astro, Angular SSR all do this). Serve the same content to everyone. Showing crawlers a different page from visitors is cloaking, which Google penalizes.

Should I block GPTBot in robots.txt?

GPTBot collects training data. Blocking it is your choice and doesn't remove you from ChatGPT search, which uses OAI-SearchBot and ChatGPT-User. Blocking those, or PerplexityBot, keeps your pages out of their answers.

Is this tool free?

Yes, a few pages a day with no sign-up. Sign up free to check every page of your site.

Related free tools

Robots.txt generator · H1 & heading checker · Structured data validator · All free SEO tools