Amzn-SearchBot

What user agent does Amzn-SearchBot use?

Amazon publishes this user agent string for Amzn-SearchBot.

Amazon writes W.X.Y.Z where the Chrome version goes, because it changes.

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amzn-SearchBot/0.1) Chrome/W.X.Y.Z Safari/537.36

How do I block Amzn-SearchBot?

Amazon says its crawlers follow robots.txt allow and disallow rules but not Crawl-delay, that each of its bots is controlled separately, and that a change can take about 24 hours. Each subdomain needs its own robots.txt. When your robots.txt does not name Amzn-SearchBot but allows other search bots, it follows their rules. Blocking Amazonbot does not block it.

Add this to the robots.txt file at the root of your site:

User-agent: Amzn-SearchBot
Disallow: /

How do I check that a Amzn-SearchBot visit is real?

Anyone can put a crawler’s name in a user agent, so the name alone proves nothing.

Amazon publishes a separate list of IP addresses for each of its bots, this one for Amzn-SearchBot.

How do I see Amzn-SearchBot visits on my site?

Your server’s access logs record the user agent of each request, so you can search them for “Amzn-SearchBot”.

Or add the Spreelytics server middleware, @spreelytics/ai-crawl. It recognises Amzn-SearchBot by its user agent and counts each visit on the Bots page under Indexing, apart from your visitor numbers.

Sources: developer.amazon.com/amazonbot