OAI-SearchBot: OpenAI's Search Crawler
What is OAI-SearchBot?
OAI-SearchBot surfaces websites in ChatGPT search results. It is not used to train foundation models. OAI-SearchBot builds the index ChatGPT search cites from, so it is the one OpenAI token whose block removes you from answers a person can click.
Key facts
- OAI-SearchBot is operated by OpenAI, and its purpose is search index: builds the index an AI product cites from.
- The user agent for OAI-SearchBot is documented by OpenAI and contains the token OAI-SearchBot.
- The robots.txt token for OAI-SearchBot is OAI-SearchBot, and OpenAI documents that OAI-SearchBot respects that rule.
- OAI-SearchBot can send a visitor to your site, and OAI-SearchBot attaches a linked citation when it uses your content.
Specification
| Operator | OpenAI |
|---|---|
| Purpose | Search indexBuilds the index an AI product cites from. Blocking it removes you from that product's answers. This is the expensive block, and the one most often made by accident. |
| Agent kind | CrawlerAutonomous crawler. Appears in your logs under its own user-agent. |
| User agent | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot |
| robots.txt token | OAI-SearchBot |
| robots.txt compliance | Documented: respects robots.txt |
| Verification | Published IP range filehttps://openai.com/searchbot.json |
| Sends referral traffic | Yes |
| Attaches a citation | Yes |
| Status | Active |
| Legacy Kymo category | indexing |
| Source | Operator documentationhttps://developers.openai.com/api/docs/bots |
| Verified on |
What this looks like in your logs
203.0.113.42 - - [04/Aug/2026:09:14:07 +0000] "GET /pricing HTTP/1.1" 200 18432 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot"
The part that identifies the request is the token OAI-SearchBot, inside the quoted user-agent field at the end of the line. Everything before it is the ordinary shape of a Common Log Format entry, and the address shown is a documentation address rather than one OpenAI uses.
What OAI-SearchBot does not do
OAI-SearchBot does not guarantee a visit, because a read only becomes a visit if a person clicks the citation that came out of it.
OAI-SearchBot does not train a model on what it reads, so blocking it changes what OpenAI can answer and not what OpenAI learns.
OpenAI publishes no crawl frequency for OAI-SearchBot, so any schedule you read elsewhere is somebody's estimate rather than a documented figure.
How Kymo classifies it
Kymo files OAI-SearchBot under OpenAI as indexing, classified from the HTTP request on the server rather than from anything running in a browser.
A hit from OAI-SearchBot appears on the Crawlers dashboard and is deliberately left out of the AI Visibility numbers. An index sweep produces no click to miss, so counting it there would inflate the read number and break its relationship to the visitor number beside it.
Seen on our own sites
| Last seen | 15 Sep 2026 |
|---|---|
| Days active, last 30 days | 31 |
| Requests, last 30 days | 64,525 |
Counted by Kymo on kymo.in and remotestack.in, the two sites we run, up to 15 Sep 2026. Two sites is a small sample. Treat these numbers as proof the bot is active now.
Allow it or block it
Blocking OAI-SearchBot removes your site from ChatGPT search answers, though it can still appear as a navigational link. This is usually the costliest OpenAI block and the one made by accident. OpenAI notes robots.txt changes take roughly 24 hours to take effect for search.
| Allow it if | Block it if |
|---|---|
| You want to appear in OpenAI answers, because this crawler builds the index those answers cite. | You accept disappearing from OpenAI answers. This is the most expensive block on this page, and it is the one made most often by accident. |
robots.txt directives
Both blocks below address OAI-SearchBot only. Rules for one token never apply to another, even from the same operator.
Allow OAI-SearchBot
User-agent: OAI-SearchBot Allow: /
Block OAI-SearchBot
User-agent: OAI-SearchBot Disallow: /
Questions
How do I know a request claiming to be OAI-SearchBot is genuine?
OpenAI publishes the IP ranges OAI-SearchBot uses at https://openai.com/searchbot.json. Check the requesting address against that file. A user-agent string alone proves nothing, because anyone can send one.
Does blocking OAI-SearchBot also block GPTBot?
No. OAI-SearchBot and GPTBot are separate tokens, and OpenAI treats each one on its own. A rule naming OAI-SearchBot leaves GPTBot free to carry on collecting your content for OpenAI model training. To stop that as well, name GPTBot in a rule of its own.
Will I see OAI-SearchBot in Google Analytics?
No. Google Analytics runs a JavaScript tag in a visitor's browser, and OAI-SearchBot reads your HTML and leaves without running any script. Server-side logging is the only way to record it.
Does OAI-SearchBot send traffic back to my site?
Yes. OAI-SearchBot builds an index that OpenAI cites its sources from, and a citation there is a link a person can click. A crawl is still not a visit, so expect the read count to run far ahead of the click count.
Related crawlers
All AI crawlers and control tokens
See this bot in your own logs
OAI-SearchBot decides whether you appear in OpenAI answers, and a browser tag cannot tell you whether it ever came.
Kymo reads the HTTP request on your server, so a crawler that never runs JavaScript is still recorded. You see which bots reached the site, which pages they took and how often, next to your human traffic.
Start free → 14-day free trial. No card required.
No account yet? Run a free AI visibility check on your own site.