Can ChatGPT, Claude, Perplexity and Google AI read your website?
Check your live robots.txt against every major AI crawler in seconds, and see which blocks actually hurt your AI visibility and which only opt you out of model training.
Free, no sign-up. We read your public robots.txt and homepage only.
Not every "AI bot" is the same.
AI companies run separate crawlers for search, for user-requested visits and for model training. Blocking the wrong one can quietly remove you from AI answers, while blocking the training-only ones does not.
| Type | Crawlers | If blocked |
|---|---|---|
| Search | OAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot, Bingbot | That engine can't index you for its answers |
| User visits | ChatGPT-User, Claude-User, Perplexity-User | May reduce visibility when users ask about you |
| Gemini grounding | Google-Extended | Gemini can't cite you; AI Overviews unaffected |
| Training only | GPTBot, ClaudeBot, Applebot-Extended, CCBot | No effect on AI search visibility |
Which AI crawlers matter for visibility?
The search crawlers: OAI-SearchBot (ChatGPT search), Claude-SearchBot (Claude), PerplexityBot (Perplexity), Googlebot (Google Search and AI Overviews) and Bingbot (Bing and Microsoft Copilot). If any of them is blocked, that engine cannot index your pages for its answers.
Should I block GPTBot and ClaudeBot?
That is your choice. According to OpenAI and Anthropic, GPTBot and ClaudeBot collect content for model training. Blocking them keeps your future content out of training data and does not remove you from ChatGPT or Claude search results.
Does blocking Google-Extended remove me from AI Overviews?
No. Google-Extended controls whether Gemini can use your pages for training and for grounding answers in the Gemini app. AI Overviews and AI Mode are part of Google Search and follow Googlebot and the normal Search controls.
Do ChatGPT-User and Perplexity-User follow robots.txt?
OpenAI and Perplexity say these user-requested fetchers may not follow robots.txt, because a person asked for the page. Anthropic says blocking Claude-User may reduce your visibility in user-directed search.
Is this check the same as a full AI visibility audit?
No. Crawler access is the first gate: if AI engines cannot read your site, nothing else matters. The AI Visibility Fixer goes further and checks your entity data, FAQs, local signals and profiles, then gives you fixes.
Learn the fixes
Crawlers allowed? Find what else is holding you back.
The AI Visibility Fixer checks entity data, FAQs, local signals and profiles, then gives you fixes.
Find My Visibility Gaps