Claude-User: Anthropic's Answer Crawler
What is Claude-User?
Claude-User fetches a page when a person asks Claude a question that requires reading it. Claude-User fetches on a person's request, and Anthropic documents no robots.txt exemption for it, unlike the user-initiated fetchers at OpenAI and Perplexity.
Key facts
- Claude-User is operated by Anthropic, and its purpose is live fetch: fetches a page in real time because a person asked.
- Anthropic publishes no full user-agent string for Claude-User, so Kymo matches the token Claude-User observed in real logs.
- The robots.txt token for Claude-User is Claude-User, and Anthropic documents that Claude-User respects that rule.
- Claude-User can send a visitor to your site, and Claude-User attaches a linked citation when it uses your content.
Specification
| Operator | Anthropic |
|---|---|
| Purpose | Live fetchFetches a page in real time because a person asked. Traffic here tracks human interest, not a crawl schedule. |
| Agent kind | FetcherFetches on a person's request. Appears in your logs. Request volume follows human activity. |
| User agent | Not documented by Anthropic. Kymo matches this agent on the token Claude-User, observed in real logs rather than published in operator documentation. |
| robots.txt token | Claude-User |
| robots.txt compliance | Documented: respects robots.txt |
| Verification | Reverse DNS, forward confirmed |
| Sends referral traffic | Yes |
| Attaches a citation | Yes |
| Status | Active |
| Legacy Kymo category | ai_answer |
| Source | Operator documentationhttps://support.claude.com/en/articles/8896518 |
| Verified on |
What this looks like in your logs
grep -i 'Claude-User' access.log
Anthropic publishes no full user-agent string for Claude-User, so the rest of the line varies and only the token Claude-User is dependable. Kymo matches on that token, case-insensitively, for the same reason.
What Claude-User does not do
Claude-User does not guarantee a visit, because a read only becomes a visit if a person clicks the citation that came out of it.
Claude-User does not crawl on a schedule, because it only requests a page when a person has already asked something that needs it.
Anthropic publishes no crawl frequency for Claude-User, so any schedule you read elsewhere is somebody's estimate rather than a documented figure.
How Kymo classifies it
Kymo files Claude-User under Anthropic as ai_answer, classified from the HTTP request on the server rather than from anything running in a browser.
A hit from Claude-User counts in the reads on the AI Traffic page. An answer crawl is the only kind made for one person's question, so it is the only kind counted beside the people AI sent you.
Seen on our own sites
| Last seen | 2 Nov 2026 |
|---|---|
| Days active, last 30 days | 31 |
| Requests, last 30 days | 14,047 |
Counted by Kymo on kymo.in and remotestack.in, the two sites we run, up to 2 Nov 2026. Two sites is a small sample. Treat these numbers as proof the bot is active now.
Allow it or block it
Blocking Claude-User prevents Anthropic from retrieving your content in response to a user query, which Anthropic states may reduce visibility for user-directed web search. Unlike OpenAI, Anthropic does not document a user-initiated exemption from robots.txt for this agent.
| Allow it if | Block it if |
|---|---|
| You want Anthropic to read your live page when a person asks about you, which is the crawl most likely to send a real visitor. | You do not want your page read during a live question. Check the compliance row first: a fetcher started by a person may need an edge rule rather than a robots.txt rule. |
robots.txt directives
Both blocks below address Claude-User only. Rules for one token never apply to another, even from the same operator.
Allow Claude-User
User-agent: Claude-User Allow: /
Block Claude-User
User-agent: Claude-User Disallow: /
Questions
How do I know a request claiming to be Claude-User is genuine?
Run a reverse DNS lookup on the requesting address, confirm it ends in null, then run a forward lookup on that name and check it returns the same address. A user-agent string alone proves nothing.
Does blocking Claude-User also block Claude-SearchBot?
No. Claude-User and Claude-SearchBot are separate tokens, and Anthropic treats each one on its own. A rule naming Claude-User leaves Claude-SearchBot free to carry on indexing your pages for Anthropic answers. To stop that as well, name Claude-SearchBot in a rule of its own.
Will I see Claude-User in Google Analytics?
No. Google Analytics runs a JavaScript tag in a visitor's browser, and Claude-User reads your HTML and leaves without running any script. Server-side logging is the only way to record it.
Does Claude-User send traffic back to my site?
Yes. Claude-User builds an index that Anthropic cites its sources from, and a citation there is a link a person can click. A crawl is still not a visit, so expect the read count to run far ahead of the click count.
Related crawlers
All AI crawlers and control tokens
See this bot in your own logs
Claude-User arrives because a person just asked Anthropic something, and a browser tag never runs in time to record the moment.
Kymo reads the HTTP request on your server, so a crawler that never runs JavaScript is still recorded. You see which bots reached the site, which pages they took and how often, next to your human traffic.
Start free → 14-day free trial. No card required.
No account yet? Run a free AI visibility check on your own site.