What Slackbot does
Slackbot is Slack's general-purpose agent, used for requests that are not covered by Slack's more specific agents. According to Slack's documentation, this includes API calls for integrated services and outgoing webhooks that users have configured. It identifies itself with a user-agent such as Slackbot 1.0 with a link to api.slack.com/robots.
How to control Slackbot
Target Slackbot in your robots.txt with the Slackbot token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow Slackbot
User-agent: Slackbot
Allow: /Block Slackbot
User-agent: Slackbot
Disallow: /Slack states it does not currently honor robots.txt files, on the grounds that its requests are made on behalf of users rather than by a crawler.
How to identify Slackbot
Temso identifies Slackbottraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
SlackbotSlackbot FAQ
What is Slackbot?
Slackbot is Slack's general-purpose agent for HTTP requests not handled by its specialized agents, including API calls for integrated services and outgoing webhooks. Requests from it originate from actions taken by Slack users and integrations.
How do I block Slackbot?
A robots.txt group with "User-agent: Slackbot" will not stop it, because Slack states it does not currently honor robots.txt files, describing its requests as made on behalf of users rather than crawling. To have your site blocklisted, Slack directs owners to contact its help system or @slackapi on Twitter.
About Operational Bots
Monitoring, security scanning, link previews, webhooks, and feed fetchers: the utility traffic that keeps integrations and shared links working.
Most of this traffic is service-critical or user-triggered. Webhooks and previews don't follow robots.txt, and blocking them tends to break things you rely on.
