Google-Extended: robots.txt token, what it controls, and why it never appears in logs
Google-Extended is a robots.txt control token, not a crawler. It has no user-agent and never appears in your logs. It controls whether content Google already crawled may be used to train and ground Gemini.
Google-Extended has no user agent and will never appear in your logs. It is a robots.txt token only. It controls how content that a different, already permitted crawler collected may be used.
If a log line, analytics report or bot-detection tool claims a visit from Google-Extended, that record is wrong. Either the request is spoofed, or the tool is inventing a user agent that does not exist. Most published crawler directories get this wrong.
Specification
| Operator | |
|---|---|
| Purpose | TrainingCollects content that may be used to train future models. Blocking it removes your content from future training runs. It does not affect whether a model can find and cite you today. |
| Agent kind | Control tokenA robots.txt token only. It has no user-agent and will never appear in your logs. It controls how content already collected by another crawler may be used. If this string shows up in your access log, treat it as spoofed. |
| User agent | None. This entry is a robots.txt token and has no user agent. |
| robots.txt token | Google-Extended |
| robots.txt compliance | Documented: respects robots.txt |
| Verification | No published method, user agent is the only signal |
| Sends referral traffic | No |
| Attaches a citation | No |
| Status | Active |
| Legacy Kymo category | training |
| Source | Operator documentationhttps://developers.google.com/search/docs/crawling-indexing/google-common-crawlers |
| Verified on |
What blocking it costs you
Disallowing Google-Extended opts your content out of Gemini training and grounding. It has no effect on Google Search ranking or on Googlebot. If this string appears in your access log, it is spoofed traffic, because no real client sends it.
robots.txt directives
Both blocks below address Google-Extended only. Rules for one token never apply to another, even from the same operator.
Allow Google-Extended
User-agent: Google-Extended Allow: /
Block Google-Extended
User-agent: Google-Extended Disallow: /
Questions
Does Google-Extended appear in my server logs?
No. Google-Extended is a robots.txt token only. It has no user agent, so it never appears in an access log. A log line naming Google-Extended is spoofed or comes from a misconfigured client.
Does Google-Extended respect robots.txt?
Documented: respects robots.txt
What does blocking Google-Extended cost me?
Disallowing Google-Extended opts your content out of Gemini training and grounding. It has no effect on Google Search ranking or on Googlebot. If this string appears in your access log, it is spoofed traffic, because no real client sends it.
Related crawlers
All AI crawlers and control tokens
See this bot in your own logs
Kymo classifies every crawler hit server-side using this registry, so a spoofed request claiming this token is flagged rather than counted. Read how crawler detection works, or run a free free AI visibility check to see what each crawler currently gets back from your server. No account needed for the check.