Google-CloudVertexBot

What user agent does Google-CloudVertexBot use?

Google publishes this user agent string for Google-CloudVertexBot.

Google’s crawler list gives only the name. The full string is from Google Cloud’s documentation, and its Chrome version will change.

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.7559.132 Mobile Safari/537.36 (compatible; Google-CloudVertexBot; +https://cloud.google.com/enterprise-search)

How do I block Google-CloudVertexBot?

Google says its common crawlers always obey robots.txt when they crawl automatically. Google-CloudVertexBot also follows rules written for Googlebot, and Google Cloud fetches a data store’s sitemap with Googlebot.

Add this to the robots.txt file at the root of your site:

User-agent: Google-CloudVertexBot
Disallow: /

How do I check that a Google-CloudVertexBot visit is real?

Anyone can put a crawler’s name in a user agent, so the name alone proves nothing.

Google publishes the IP ranges its common crawlers use. A reverse DNS lookup of a genuine request’s IP address gives a host name under googlebot.com, and a forward lookup of that name returns the same address.

Spreelytics runs this check for you. Every crawl that claims to be Google-CloudVertexBot is compared with the IP ranges Google publishes, refreshed every 12 hours, and one from outside them counts as spoofed.

How do I see Google-CloudVertexBot visits on my site?

Your server’s access logs record the user agent of each request, so you can search them for “Google-CloudVertexBot”.

Or add the Spreelytics server middleware, @spreelytics/ai-crawl. It recognises Google-CloudVertexBot by its user agent and counts each visit on the Bots page under Training, apart from your visitor numbers.

Sources: developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers#google-cloudvertexbot, docs.cloud.google.com/generative-ai-app-builder/docs/prepare-data#website, developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests