meta-externalfetcher

What user agent does meta-externalfetcher use?

Meta publishes these user agent strings for meta-externalfetcher.

Meta says the user agent in your logs will be similar to these.

meta-externalfetcher/1.1 (+/documentation/sharing/webmasters/web-crawlers)
meta-externalfetcher/1.1

How do I block meta-externalfetcher?

Meta says meta-externalfetcher may bypass robots.txt, because it fetches links a person asked for.

This is the robots.txt rule that names it:

User-agent: meta-externalfetcher
Disallow: /

How do I check that a meta-externalfetcher visit is real?

Anyone can put a crawler’s name in a user agent, so the name alone proves nothing.

Meta does not publish IP ranges or a DNS check for meta-externalfetcher, so there is no way to confirm a request really came from it.

How do I see meta-externalfetcher visits on my site?

Your server’s access logs record the user agent of each request, so you can search them for “meta-externalfetcher”.

Or add the Spreelytics server middleware, @spreelytics/ai-crawl. It recognises meta-externalfetcher by its user agent and counts each visit on the Bots page under AI answers, as Meta AI, apart from your visitor numbers.

Sources: developers.facebook.com/documentation/sharing/webmasters/web-crawlers