How to control anthropic-ai
Target anthropic-ai in your robots.txt with the anthropic-ai token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow anthropic-ai
User-agent: anthropic-ai
Allow: /Block anthropic-ai
User-agent: anthropic-ai
Disallow: /How to identify anthropic-ai
Temso identifies anthropic-aitraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
anthropic-aianthropic-ai FAQ
What is anthropic-ai?
anthropic-ai is a user agent token associated with Anthropic, the maker of the Claude AI models. Anthropic's current crawler documentation describes ClaudeBot (which collects web content that could contribute to model training), Claude-User, and Claude-SearchBot, and does not document the anthropic-ai token specifically.
How do I block anthropic-ai?
Add a robots.txt group with User-agent: anthropic-ai followed by Disallow: /, and apply it on every subdomain you want to opt out. Anthropic states that its bots respect do-not-crawl signals by honoring industry standard robots.txt directives; requests from Anthropic can also be checked against the IP list it publishes at claude.com/crawling/bots.json.
About AI Training Crawlers
Crawlers that collect content to train foundation models. Data gathered today shapes what future model versions know about your brand.
Blocking these doesn't affect live citations, but it limits what future models learn about you from your own site, leaving third-party sources to fill the gap.
