What Google-Extended does
Google-Extended is a standalone product token: no separate crawler visits your site with this name. Adding it to robots.txt tells Google not to use content Googlebot has already fetched for training Gemini models or for grounding in Gemini apps and the Gemini API. It does not affect Google Search indexing or AI Overviews eligibility.
Should I opt out of Google-Extended?
This is the cleanest separation in the ecosystem: you can block Gemini training while keeping full Google Search and AI Overviews visibility. Disallowing Google-Extended does not change how your site appears in Search, but it does limit what Gemini models learn from your content.
How to control Google-Extended
Google-Extended is a robots.txt control token, not a crawler: no separate agent visits your site under this name. Add the rule below to opt out; omitting it (the default) leaves the content usable.
Opt out
User-agent: Google-Extended
Disallow: /There is no Google-Extended user agent in server logs; the token is honored by Google's crawl infrastructure on content fetched by Googlebot.
Google-Extended FAQ
What is Google-Extended?
Google-Extended is not a separate crawler and has no HTTP user-agent string of its own; it is a standalone robots.txt product token that controls whether content Google crawls from your site may be used to train future generations of Gemini models and for grounding in Gemini Apps and on Vertex AI. Crawling itself is done with existing Google user agents.
How do I block Google-Extended?
Add a robots.txt group with "User-agent: Google-Extended" followed by "Disallow: /". This opts your content out of Gemini training and grounding use but does not stop Google from crawling your pages, and per Google's documentation it does not impact your site's inclusion in Google Search and is not used as a ranking signal.
About AI Training Crawlers
Crawlers that collect content to train foundation models. Data gathered today shapes what future model versions know about your brand.
Blocking these doesn't affect live citations, but it limits what future models learn about you from your own site, leaving third-party sources to fill the gap.
