OAI-SearchBot

What user agent does OAI-SearchBot use?

OpenAI publishes these user agent strings for OAI-SearchBot.

OpenAI says the version number may change. The second form, with robots.txt in it, is the one it may send when it fetches your robots.txt file.

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; robots.txt; +https://openai.com/searchbot

How do I block OAI-SearchBot?

OpenAI says a site that blocks OAI-SearchBot is not shown in ChatGPT search answers, though it can still appear as a plain navigational link, and that a change takes about 24 hours to show. Blocking it does not block GPTBot, or the other way round.

Add this to the robots.txt file at the root of your site:

User-agent: OAI-SearchBot
Disallow: /

How do I check that a OAI-SearchBot visit is real?

Anyone can put a crawler’s name in a user agent, so the name alone proves nothing.

OpenAI publishes the IP addresses OAI-SearchBot crawls from, as JSON.

Spreelytics runs this check for you. Every crawl that claims to be OAI-SearchBot is compared with the IP ranges OpenAI publishes, refreshed every 12 hours, and one from outside them counts as spoofed.

How do I see OAI-SearchBot visits on my site?

Your server’s access logs record the user agent of each request, so you can search them for “OAI-SearchBot”.

Or add the Spreelytics server middleware, @spreelytics/ai-crawl. It recognises OAI-SearchBot by its user agent and counts each visit on the Bots page under Indexing, apart from your visitor numbers.

Sources: developers.openai.com/api/docs/bots, help.openai.com/en/articles/9237897-searching-the-web-with-chatgpt