Is your site blocked from ChatGPT, Claude and Perplexity?

Enter a URL to see which AI crawlers your robots.txt allows, whether your firewall or CDN turns them away, and how ready the page is to be quoted in AI answers.

Free, no signup. Checks one page.

What this tool checks

  • robots.txt rules for 14 AI crawlers, including OAI-SearchBot, ChatGPT-User, GPTBot, Claude-SearchBot, ClaudeBot, PerplexityBot and Google-Extended
  • Firewall and CDN blocks: we request the page as GPTBot, ClaudeBot and PerplexityBot and look for refusals or bot challenges
  • Whether /llms.txt is published
  • Answer-friendly signals: structured data, question-style headings, author and date information

Search bots vs. training bots

AI companies run two kinds of crawlers. Search and user-triggered bots (OAI-SearchBot, ChatGPT-User, Claude-SearchBot, PerplexityBot) fetch pages so an assistant can answer a question and cite the source. Training bots (GPTBot, ClaudeBot, CCBot, Google-Extended) collect data to train future models.

Blocking training bots is a legitimate choice and doesn't stop you from appearing in AI answers. Blocking search bots does: if ChatGPT or Perplexity can't read your page, they can't recommend it. This checker flags only blocked search bots as a problem.

A common surprise is Cloudflare's "Block AI bots" setting, which refuses AI crawlers at the edge even when robots.txt allows them. The firewall probe catches that.

Frequently asked questions

How do I let ChatGPT see my website?

Allow OAI-SearchBot and ChatGPT-User in robots.txt (or don't disallow them), make sure your CDN or firewall doesn't block them, and serve your main content in the HTML rather than only after JavaScript runs.

Should I block GPTBot?

GPTBot only collects training data. Blocking it doesn't remove you from ChatGPT search answers, which use OAI-SearchBot and ChatGPT-User. Decide based on whether you want your content used for model training.

What is llms.txt?

llms.txt is a short Markdown file at the root of your site that summarizes what you do and links to your most important pages, written for AI assistants and agents. It's optional and still an emerging convention.

Do AI crawlers run JavaScript?

Most don't. GPTBot, ClaudeBot and PerplexityBot read the raw HTML. If your content is only rendered client-side, they see an almost empty page. Server-side rendering or pre-rendering fixes this.

More free SEO tools

  • Meta Tag CheckerCheck any page's title tag, meta description, canonical, Open Graph and Twitter Card tags.
  • Robots.txt CheckerFetch and test any site's robots.txt.
  • Schema Markup CheckerSee which schema.org structured data a page has (JSON-LD, Microdata, RDFa), find invalid JSON-LD, and get the markup types you're missing..
  • Heading CheckerSee the H1–H6 outline of any page.