GPTBot: user agent, IP ranges and robots.txt

GPTBot collects public web pages that OpenAI may use to train its foundation models. Disallowing it tells OpenAI your content should not be used for training.

Operator
OpenAI
Type
Training · Collects content to train future models
robots.txt token
GPTBot
Follows robots.txt
Yes
IP list
https://openai.com/gptbot.json
Official docs
developers.openai.com

GPTBot user agent

User agent

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot

The version number changes over time, so match on "GPTBot" rather than the full string. When fetching robots.txt, OpenAI may add a robots.txt marker to the user agent.

How to block or allow GPTBot in robots.txt

OpenAI uses robots.txt as the opt-out for GPTBot.

Block GPTBot

User-agent: GPTBot
Disallow: /

Allow GPTBot

User-agent: GPTBot
Allow: /

Put the rules in /robots.txt at the root of each host. Check the result with our AI Crawler Checker, which also catches firewall and CDN blocks that robots.txt doesn't show.

Should you block GPTBot?

Block GPTBot if you don't want your content used to train OpenAI models. It has no effect on whether ChatGPT search shows or links to you: that is OAI-SearchBot. Many publishers block GPTBot and allow OAI-SearchBot.

If you allow both GPTBot and OAI-SearchBot, OpenAI says it may use the results of one crawl for both purposes, so you may see fewer requests than you expect from each.

How to verify real GPTBot traffic

Check the request IP against OpenAI's published GPTBot list. A request that says GPTBot from any other IP is not OpenAI.

Loading the current list…

To check every request in your server log at once, use the free AI Bot Log Analyzer. It runs in your browser and flags requests that use a bot's name from outside its official IP list.

Frequently asked questions

What is GPTBot?

GPTBot collects public web pages that OpenAI may use to train its foundation models. Disallowing it tells OpenAI your content should not be used for training.

How do I block GPTBot?

Add "User-agent: GPTBot" followed by "Disallow: /" to your robots.txt. To block only part of the site, disallow those paths instead of "/". OpenAI uses robots.txt as the opt-out for GPTBot.

Is a request claiming to be GPTBot really from OpenAI?

Not necessarily: anyone can copy a user agent. Check the request IP against OpenAI's published GPTBot list. A request that says GPTBot from any other IP is not OpenAI.

Does blocking GPTBot remove my site from ChatGPT?

No. ChatGPT search uses OAI-SearchBot to find pages and ChatGPT-User to open them when someone asks. Blocking GPTBot only opts you out of model training.

Checked against OpenAI's documentation on 2026-11-06. Vendors change user agents and policies, so confirm in the official docs before relying on them.

Related crawlers

Is GPTBot allowed on your site?

The free AI Crawler Checker tests robots.txt and firewall rules for 14 AI crawlers in one run.

Check my site