AI crawlers·You.com
YouBot
The crawler behind You.com’s search engine. It finds and indexes pages so You.com can give its users real-time search results.
- Operator
- You.com
- robots.txt token
YouBot- Follows robots.txt
- Yes
- IP ranges
- Published
- In Spreelytics
- Indexing tab
- Source
- You.com’s documentation
What user agent does YouBot use?
You.com publishes this user agent string for YouBot.
You.com writes X.X.X.X where the Chrome version goes.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; YouBot/1.0; +https://docs.you.com/youbot; env:prod) Chrome/X.X.X.X Safari/537.36How do I block YouBot?
You.com says YouBot honours robots.txt, including Crawl-delay, and re-reads the file every 30 minutes, so a change takes effect within half an hour.
Add this to the robots.txt file at the root of your site:
User-agent: YouBot
Disallow: /How do I check that a YouBot visit is real?
Anyone can put a crawler’s name in a user agent, so the name alone proves nothing.
You.com recommends checking the request’s Web Bot Auth signature against the keys it publishes. A reverse DNS lookup of a genuine request gives a host name ending in search.you.com that resolves back to the same address, and its requests come from 68.67.112.0/24.
How do I see YouBot visits on my site?
Your server’s access logs record the user agent of each request, so you can search them for “YouBot”.
Or add the Spreelytics server middleware, @spreelytics/ai-crawl. It recognises YouBot by its user agent and counts each visit on the Bots page under Indexing, apart from your visitor numbers.
Sources: you.com/docs/youbot
See your own numbers today.
Free up to 100k events a month · no credit card