OAI-SearchBot: user agent, purpose, and whether to allow it
OAI-SearchBot surfaces websites in ChatGPT search results. It is not used to train foundation models.
Specification
| Operator | OpenAI |
|---|---|
| Purpose | Search indexBuilds the index an AI product cites from. Blocking it removes you from that product's answers. This is the expensive block, and the one most often made by accident. |
| Agent kind | CrawlerAutonomous crawler. Appears in your logs under its own user-agent. |
| User agent | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot |
| robots.txt token | OAI-SearchBot |
| robots.txt compliance | Documented: respects robots.txt |
| Verification | Published IP range filehttps://openai.com/searchbot.json |
| Sends referral traffic | Yes |
| Attaches a citation | Yes |
| Status | Active |
| Legacy Kymo category | indexing |
| Source | Operator documentationhttps://developers.openai.com/api/docs/bots |
| Verified on |
What blocking it costs you
Blocking OAI-SearchBot removes your site from ChatGPT search answers, though it can still appear as a navigational link. This is usually the costliest OpenAI block and the one made by accident. OpenAI notes robots.txt changes take roughly 24 hours to take effect for search.
robots.txt directives
Both blocks below address OAI-SearchBot only. Rules for one token never apply to another, even from the same operator.
Allow OAI-SearchBot
User-agent: OAI-SearchBot Allow: /
Block OAI-SearchBot
User-agent: OAI-SearchBot Disallow: /
Questions
What is the OAI-SearchBot user agent?
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
Does OAI-SearchBot respect robots.txt?
Documented: respects robots.txt
What does blocking OAI-SearchBot cost me?
Blocking OAI-SearchBot removes your site from ChatGPT search answers, though it can still appear as a navigational link. This is usually the costliest OpenAI block and the one made by accident. OpenAI notes robots.txt changes take roughly 24 hours to take effect for search.
Related crawlers
All AI crawlers and control tokens
See this bot in your own logs
Kymo classifies every crawler hit server-side using this registry, so OAI-SearchBot shows up under OpenAI with its purpose attached. Read how crawler detection works, or run a free free AI visibility check to see what each crawler currently gets back from your server. No account needed for the check.