What AhrefsBot does
AhrefsBot is the web crawler that powers Ahrefs, a marketing intelligence platform, and Yep, an independent search engine. It indexes information about websites, their content, and how they link to each other. It obeys robots.txt and crawl-delay directives, caches frequently requested files to reduce server load, and automatically slows down when it encounters error responses. Its user agent string is "Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)".
How to control AhrefsBot
Target AhrefsBot in your robots.txt with the AhrefsBot token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow AhrefsBot
User-agent: AhrefsBot
Allow: /Block AhrefsBot
User-agent: AhrefsBot
Disallow: /Ahrefs publishes the bot's IP addresses and ranges via API and supports reverse DNS verification (hostnames end in ahrefs.com or ahrefs.net), and notes the bot may need some time to pick up robots.txt changes.
How to identify AhrefsBot
Temso identifies AhrefsBottraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
AhrefsBotAhrefsBot FAQ
What is AhrefsBot?
It is the web crawler that builds the link and content index behind Ahrefs, a marketing intelligence platform, and also powers Yep, an independent search engine. It indexes information about websites, their content, and their linking relationships.
How do I block AhrefsBot?
Add a robots.txt group with "User-agent: AhrefsBot" followed by "Disallow: /". Ahrefs notes the bot may take some time to pick up robots.txt changes, and that crawl-delay cannot be followed while it renders JavaScript, since page assets may be requested simultaneously.
About SEO & Analytics Crawlers
Crawlers behind SEO and marketing-intelligence platforms: backlink databases, site audits, rank tracking, and brand monitoring.
Blocking these has no effect on search or AI visibility. It mainly limits what third-party tools (including your competitors' tools) can see about your site.
