Claude-User: Anthropic's Answer Crawler
What is Claude-User?
Claude-User fetches a page when a person asks Claude a question that requires reading it. Claude-User fetches on a person's request, and Anthropic documents no robots.txt exemption for it, unlike the user-initiated fetchers at OpenAI and Perplexity.
Key facts
- Claude-User is operated by Anthropic, and its purpose is live fetch: fetches a page in real time because a person asked.
- Anthropic publishes no full user-agent string for Claude-User, so Kymo matches the token Claude-User observed in real logs.
- The robots.txt token for Claude-User is Claude-User, and Anthropic documents that Claude-User respects that rule.
- Claude-User can send a visitor to your site, and Claude-User attaches a linked citation when it uses your content.
Specification
| Operator | Anthropic |
|---|---|
| Purpose | Live fetchFetches a page in real time because a person asked. Traffic here tracks human interest, not a crawl schedule. |
| Agent kind | FetcherFetches on a person's request. Appears in your logs. Request volume follows human activity. |
| User agent | Not documented by Anthropic. Kymo matches this agent on the token Claude-User, observed in real logs rather than published in operator documentation. |
| robots.txt token | Claude-User |
| robots.txt compliance | Documented: respects robots.txt |
| Verification | Reverse DNS, forward confirmed |
| Sends referral traffic | Yes |
| Attaches a citation | Yes |
| Status | Active |
| Legacy Kymo category | ai_answer |
| Source | Operator documentationhttps://support.claude.com/en/articles/8896518 |
| Verified on |
What this looks like in your logs
grep -i 'Claude-User' access.log
Anthropic publishes no full user-agent string for Claude-User, so the rest of the line varies and only the token Claude-User is dependable. Kymo matches on that token, case-insensitively, for the same reason.
What Claude-User does not do
Claude-User does not guarantee a visit, because a read only becomes a visit if a person clicks the citation that came out of it.
Claude-User does not crawl on a schedule, because it only requests a page when a person has already asked something that needs it.
Anthropic publishes no crawl frequency for Claude-User, so any schedule you read elsewhere is somebody's estimate rather than a documented figure.
How Kymo classifies it
Kymo files Claude-User under Anthropic as ai_answer, classified from the HTTP request on the server rather than from anything running in a browser.
A hit from Claude-User counts on the AI Visibility page, which is the only page that counts crawls at all. An answer crawl is the only kind a person can click through from, so it is the only kind that belongs beside a visitor number.
Seen on our own sites
| Last seen | 15 Sep 2026 |
|---|---|
| Days active, last 30 days | 31 |
| Requests, last 30 days | 841 |
Counted by Kymo on kymo.in and remotestack.in, the two sites we run, up to 15 Sep 2026. Two sites is a small sample. Treat these numbers as proof the bot is active now.
Allow it or block it
Blocking Claude-User prevents Anthropic from retrieving your content in response to a user query, which Anthropic states may reduce visibility for user-directed web search. Unlike OpenAI, Anthropic does not document a user-initiated exemption from robots.txt for this agent.
| Allow it if | Block it if |
|---|---|
| You want Anthropic to read your live page when a person asks about you, which is the crawl most likely to send a real visitor. | You do not want your page read during a live question. Check the compliance row first: a fetcher started by a person may need an edge rule rather than a robots.txt rule. |
robots.txt directives
Both blocks below address Claude-User only. Rules for one token never apply to another, even from the same operator.
Allow Claude-User
User-agent: Claude-User Allow: /
Block Claude-User
User-agent: Claude-User Disallow: /
Questions
How do I know a request claiming to be Claude-User is genuine?
Run a reverse DNS lookup on the requesting address, confirm it ends in null, then run a forward lookup on that name and check it returns the same address. A user-agent string alone proves nothing.
Does blocking Claude-User also block Claude-SearchBot?
No. Claude-User and Claude-SearchBot are separate tokens, and Anthropic treats each one on its own. A rule naming Claude-User leaves Claude-SearchBot free to carry on indexing your pages for Anthropic answers. To stop that as well, name Claude-SearchBot in a rule of its own.
Will I see Claude-User in Google Analytics?
No. Google Analytics runs a JavaScript tag in a visitor's browser, and Claude-User reads your HTML and leaves without running any script. Server-side logging is the only way to record it.
Does Claude-User send traffic back to my site?
Yes. Claude-User builds an index that Anthropic cites its sources from, and a citation there is a link a person can click. A crawl is still not a visit, so expect the read count to run far ahead of the click count.
Related crawlers
All AI crawlers and control tokens
See this bot in your own logs
Claude-User arrives because a person just asked Anthropic something, and a browser tag never runs in time to record the moment.
Kymo reads the HTTP request on your server, so a crawler that never runs JavaScript is still recorded. You see which bots reached the site, which pages they took and how often, next to your human traffic.
Start free → 14-day free trial. No card required.
No account yet? Run a free AI visibility check on your own site.