Applebot

What user agent does Applebot use?

Apple publishes these user agent strings for Applebot.

The first is Apple’s general format, the other two its desktop and mobile examples. Apple says the browser version in them changes from time to time.

Mozilla/5.0 (Device; OS_version) AppleWebKit/WebKit_version (KHTML, like Gecko)Version/Safari_version [Mobile/Mobile_version] Safari/WebKit_version (Applebot/Applebot_version; +http://www.apple.com/go/applebot)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot)
Mozilla/5.0 (iPhone; CPU iPhone OS 17_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Mobile/15E148 Safari/604.1 (Applebot/0.1; +http://www.apple.com/go/applebot)

How do I block Applebot?

Apple says Applebot follows robots.txt rules aimed at it, and follows Googlebot’s rules when a robots.txt has none for Applebot. It does not follow Crawl-delay. To keep your pages out of model training while staying in Apple’s search, Apple points to Applebot-Extended instead.

Add this to the robots.txt file at the root of your site:

User-agent: Applebot
Disallow: /

How do I check that a Applebot visit is real?

Anyone can put a crawler’s name in a user agent, so the name alone proves nothing.

Apple publishes the IP ranges Applebot crawls from. A reverse DNS lookup of a genuine request’s IP address gives a host name under applebot.apple.com, and a forward lookup of that name returns the same address.

How do I see Applebot visits on my site?

Your server’s access logs record the user agent of each request, so you can search them for “Applebot”.

Or add the Spreelytics server middleware, @spreelytics/ai-crawl. It recognises Applebot by its user agent and counts each visit on the Bots page under Indexing, apart from your visitor numbers.

Sources: support.apple.com/en-us/119829