AI crawlers·Google
Google-Extended
Not a crawler but a robots.txt token. It tells Google whether pages its other crawlers fetch may be used to train future Gemini models and to ground Gemini’s answers.
- Operator
- robots.txt token
Google-Extended- Follows robots.txt
- Yes
- IP ranges
- Not published
- In Spreelytics
- Never seen in a request
- Source
- Google’s documentation
Does Google-Extended have a user agent?
No. Google says Google-Extended has no user agent string of its own: Google’s existing crawlers do the fetching, and the token only matters in robots.txt. It never appears in a request.
How do I block Google-Extended?
Disallowing Google-Extended tells Google not to use your pages to train future Gemini models, or to ground answers in Gemini Apps and Grounding with Google Search on Vertex AI. Google says it does not affect whether your site appears in Google Search and is not a ranking signal. It does not control AI Overviews or AI Mode either: those follow Googlebot’s rules.
Add this to the robots.txt file at the root of your site:
User-agent: Google-Extended
Disallow: /Sources: developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers#google-extended, developers.google.com/search/docs/appearance/ai-features
See your own numbers today.
Free up to 100k events a month · no credit card