What YandexBot does
YandexBot is Yandex's main indexing robot, responsible for crawling pages for the Yandex search index. It identifies itself with the user agent string "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)", and Yandex recommends not matching on the browser version in that string when searching server logs because it may change. The bot respects robots.txt directives, and Yandex documents a DNS-based procedure for confirming that traffic claiming to be YandexBot is genuine.
How to control YandexBot
Target YandexBot in your robots.txt with the YandexBot token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow YandexBot
User-agent: YandexBot
Allow: /Block YandexBot
User-agent: YandexBot
Disallow: /Yandex documents authenticity verification via reverse DNS lookup, where genuine robot hostnames end in yandex.ru, yandex.net, or yandex.com and a forward DNS lookup of that hostname must return the original IP, with an IP address check tool also available.
How to identify YandexBot
Temso identifies YandexBottraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
YandexBotYandexBot FAQ
What is YandexBot?
YandexBot is the main indexing robot of the Yandex search engine, crawling pages to build the Yandex search index. It appears in logs as "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)", though Yandex advises not to match on the browser version in the string since it can change.
How do I block YandexBot?
Add a robots.txt group with "User-agent: YandexBot" followed by "Disallow: /" (or specific paths); YandexBot respects robots.txt directives. Because some robots disguise themselves as Yandex robots, verify suspicious traffic with a reverse DNS lookup (genuine hostnames end in yandex.ru, yandex.net, or yandex.com, confirmed by a matching forward lookup) rather than trusting the user agent string.
About Search Engine Crawlers
Classic search indexers that now also feed AI features: Google's AI Overviews and AI Mode build on Googlebot's index, and Microsoft Copilot builds on Bingbot's.
Blocking these removes you from both traditional rankings and the AI answers built on top of those indexes, usually the most costly block of all.
