Google-Extended: Google's AI training opt-out
Google-Extended is not a crawler: it's a control token used in robots.txt. You will never see it in your server logs — no user-agent carries this name. It tells Google whether content already crawled by its regular robots may be used to train the Gemini models and Vertex AI APIs. Google documents that this choice affects neither Search rankings nor inclusion in AI Overviews.
Technical profile
- Operator
- Role
- robots.txt token (Gemini training)
- robots.txt token
Google-Extended- User-agent (example)
- None — robots.txt token only, never visible in logs
- Respects robots.txt
- Yes — it is precisely a robots.txt token
- Official IP ranges
- Not published by the operator (IP verification impossible)
- Official documentation
- Google crawlers documentation
How a token works without a crawler
Google doesn't send a dedicated AI robot: Googlebot and the other crawlers collect content once, then Google-Extended acts as a downstream usage switch. Blocking Google-Extended in robots.txt says: “you may crawl me for Search, but not use my content to train Gemini”.
That's why searching for “Google-Extended” in a log file is futile — and why any user-agent presenting itself that way is, with certainty, an impostor.
What blocking changes (and doesn't)
Per the official documentation, blocking Google-Extended removes your content from Gemini's training data but changes nothing in Google Search: indexing, ranking and AI Overviews inclusion remain driven by Googlebot. It's one of the cleanest AI opt-outs available: an editorial choice with no documented SEO risk.
Verifying it's really Google-Extended
There is nothing to verify in your logs: Google-Extended sends no requests. If a “Google-Extended” shows up in your user-agent field, it's fake — probably a scraper hoping to exploit the confusion. Our analyzer verifies the authenticity of Google's real crawlers (Googlebot, GoogleOther) against their official IP ranges.
Blocking or allowing Google-Extended (robots.txt)
“Blocking” Google-Extended is the token's very purpose: a Disallow directive in robots.txt suffices, and Google honors it by design. A purely editorial decision — your SEO neither gains nor loses anything, per Google's documentation.
To block it:
User-agent: Google-Extended Disallow: /
To explicitly allow it:
User-agent: Google-Extended Allow: /
Frequently asked questions
Is Google-Extended a crawler?
No. Google's documentation describes it as a product token usable in robots.txt, not a robot: no Google-Extended user-agent exists, no request carries this name. Content is collected by Google's regular crawlers; the token controls its use for training Gemini.
Does blocking Google-Extended hurt my Google rankings?
No, per the official documentation: the token only governs content use for training Gemini and Vertex AI. Indexing, Search rankings and AI Overviews inclusion depend on Googlebot, which this directive does not affect.
How do I refuse Gemini training without touching my visibility?
Add to your robots.txt: “User-agent: Google-Extended” followed by “Disallow: /”. Your site keeps being crawled and ranked normally by Googlebot; only the training use is removed. It's the most cleanly scoped AI opt-out available.
Why do I never see Google-Extended in my logs?
Because it doesn't exist as an agent: it's a policy token, not a robot. Google requests in your logs come from Googlebot, GoogleOther and the other documented crawlers. A “Google-Extended” user-agent is a spoofing signal.
Sources
Every fact on this page is backed by a primary source — the operator's official documentation and, where it exists, its published IP list.
See what this bot does on your site
Drop your access.log files into the Crawl Lab log analyzer: it isolates Google-Extended, verifies its authenticity and lists the pages it requests — entirely in your browser, no upload.
Analyze my logs