GPTBot: user agent, purpose, and whether to allow it
GPTBot crawls content that may be used to train OpenAI foundation models. Blocking it signals that your content should not be used in training.
Specification
| Operator | OpenAI |
|---|---|
| Purpose | TrainingCollects content that may be used to train future models. Blocking it removes your content from future training runs. It does not affect whether a model can find and cite you today. |
| Agent kind | CrawlerAutonomous crawler. Appears in your logs under its own user-agent. |
| User agent | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot |
| robots.txt token | GPTBot |
| robots.txt compliance | Documented: respects robots.txt |
| Verification | Published IP range filehttps://openai.com/gptbot.json |
| Sends referral traffic | No |
| Attaches a citation | No |
| Status | Active |
| Legacy Kymo category | training |
| Source | Operator documentationhttps://developers.openai.com/api/docs/bots |
| Verified on |
What blocking it costs you
Blocking GPTBot keeps your content out of future OpenAI training runs. It does not remove you from ChatGPT search answers, which are governed by OAI-SearchBot. OpenAI notes that if both are allowed, one crawl may serve both purposes.
robots.txt directives
Both blocks below address GPTBot only. Rules for one token never apply to another, even from the same operator.
Allow GPTBot
User-agent: GPTBot Allow: /
Block GPTBot
User-agent: GPTBot Disallow: /
Questions
What is the GPTBot user agent?
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot
Does GPTBot respect robots.txt?
Documented: respects robots.txt
What does blocking GPTBot cost me?
Blocking GPTBot keeps your content out of future OpenAI training runs. It does not remove you from ChatGPT search answers, which are governed by OAI-SearchBot. OpenAI notes that if both are allowed, one crawl may serve both purposes.
Related crawlers
All AI crawlers and control tokens
See this bot in your own logs
Kymo classifies every crawler hit server-side using this registry, so GPTBot shows up under OpenAI with its purpose attached. Read how crawler detection works, or run a free free AI visibility check to see what each crawler currently gets back from your server. No account needed for the check.