YouBot: You.com's Search Crawler
What is YouBot?
YouBot is the You.com web crawler that builds the search index behind real-time You.com search results. You.com documents that YouBot honours a crawl-delay directive, which Googlebot and the Amazon crawlers both document that they ignore.
Key facts
- YouBot is operated by You.com, and its purpose is search index: builds the index an AI product cites from.
- The user agent for YouBot is documented by You.com and contains the token YouBot.
- The robots.txt token for YouBot is YouBot, and You.com documents that YouBot respects that rule.
- YouBot can send a visitor to your site, and YouBot attaches a linked citation when it uses your content.
Specification
| Operator | You.com |
|---|---|
| Purpose | Search indexBuilds the index an AI product cites from. Blocking it removes you from that product's answers. This is the expensive block, and the one most often made by accident. |
| Agent kind | CrawlerAutonomous crawler. Appears in your logs under its own user-agent. |
| User agent | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; YouBot/1.0; +https://docs.you.com/youbot; env:prod) Chrome/X.X.X.X Safari/537.36 |
| robots.txt token | YouBot |
| robots.txt compliance | Documented: respects robots.txt |
| Verification | Reverse DNS, forward confirmedsearch.you.com |
| Sends referral traffic | Yes |
| Attaches a citation | Yes |
| Status | Active |
| Legacy Kymo category | indexing |
| Source | Operator documentationhttps://you.com/docs/youbot |
| Verified on |
What this looks like in your logs
203.0.113.42 - - [25/Oct/2026:09:14:07 +0000] "GET /pricing HTTP/1.1" 200 18432 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; YouBot/1.0; +https://docs.you.com/youbot; env:prod) Chrome/X.X.X.X Safari/537.36"
The part that identifies the request is the token YouBot, inside the quoted user-agent field at the end of the line. Everything before it is the ordinary shape of a Common Log Format entry, and the address shown is a documentation address rather than one You.com uses.
What YouBot does not do
YouBot does not guarantee a visit, because a read only becomes a visit if a person clicks the citation that came out of it.
YouBot does not train a model on what it reads, so blocking it changes what You.com can answer and not what You.com learns.
You.com publishes no crawl frequency for YouBot, so any schedule you read elsewhere is somebody's estimate rather than a documented figure.
How Kymo classifies it
Kymo files YouBot under You.com as indexing, classified from the HTTP request on the server rather than from anything running in a browser.
A hit from YouBot appears on the Crawlers dashboard and is deliberately left out of the reads on the AI Traffic page. An index sweep is bulk work done before anyone asks, so counting it there would inflate the read number. A person can still arrive later from an answer built on that index, and the people count includes them either way.
Seen on our own sites
Not seen on kymo.in or remotestack.in in the 90 days up to 25 Oct 2026.
Allow it or block it
Blocking YouBot removes your pages from the You.com search index, and with them from the answers that cite it. You.com documents that YouBot follows robots.txt, including crawl-delay, and caches the file for 30 minutes, so a change applies quickly. Genuine requests come from 68.67.112.0/24 and resolve to hostnames under search.you.com.
| Allow it if | Block it if |
|---|---|
| You want to appear in You.com answers, because this crawler builds the index those answers cite. | You accept disappearing from You.com answers. This is the most expensive block on this page, and it is the one made most often by accident. |
robots.txt directives
Both blocks below address YouBot only. Rules for one token never apply to another, even from the same operator.
Allow YouBot
User-agent: YouBot Allow: /
Block YouBot
User-agent: YouBot Disallow: /
Questions
How do I know a request claiming to be YouBot is genuine?
Run a reverse DNS lookup on the requesting address, confirm it ends in search.you.com, then run a forward lookup on that name and check it returns the same address. A user-agent string alone proves nothing.
Does blocking YouBot affect my Google ranking?
No. A Google ranking is decided by Googlebot, which is a separate crawler. YouBot is You.com's search crawler, so blocking it changes what You.com can do with your content and changes nothing in Google search results.
Will I see YouBot in Google Analytics?
No. Google Analytics runs a JavaScript tag in a visitor's browser, and YouBot reads your HTML and leaves without running any script. Server-side logging is the only way to record it.
Does YouBot send traffic back to my site?
Yes. YouBot builds an index that You.com cites its sources from, and a citation there is a link a person can click. A crawl is still not a visit, so expect the read count to run far ahead of the click count.
Related crawlers
All AI crawlers and control tokens
See this bot in your own logs
YouBot decides whether you appear in You.com answers, and a browser tag cannot tell you whether it ever came.
Kymo reads the HTTP request on your server, so a crawler that never runs JavaScript is still recorded. You see which bots reached the site, which pages they took and how often, next to your human traffic.
Start free → 14-day free trial. No card required.
No account yet? Run a free AI visibility check on your own site.