What Slackbot-LinkExpanding does
Slackbot-LinkExpanding is the Slack agent that responds to links Slack users post into their channels, fetching the page to extract metadata for an unfurled preview. It reads oEmbed, Twitter Card, and Open Graph tags, and if those tags reference media files it fetches them to validate and extract additional metadata. It uses HTTP Range headers to minimize data transfer, and responses are cached globally across the service for around 30 minutes. It identifies itself as Slackbot-LinkExpanding 1.0 with a link to api.slack.com/robots.
How to control Slackbot-LinkExpanding
Target Slackbot-LinkExpanding in your robots.txt with the Slackbot-LinkExpanding token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow Slackbot-LinkExpanding
User-agent: Slackbot-LinkExpanding
Allow: /Block Slackbot-LinkExpanding
User-agent: Slackbot-LinkExpanding
Disallow: /Slack states it does not currently honor robots.txt files, on the grounds that these fetches are made on behalf of users rather than by a crawler.
How to identify Slackbot-LinkExpanding
Temso identifies Slackbot-LinkExpandingtraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
Slackbot-LinkExpandingSlackbot-LinkExpanding FAQ
What is Slackbot-LinkExpanding?
Slackbot-LinkExpanding is the Slack agent that fetches links posted in Slack channels to build unfurled previews. It extracts oEmbed, Twitter Card, and Open Graph metadata, and its responses are cached across Slack for around 30 minutes.
How do I block Slackbot-LinkExpanding?
A robots.txt group with "User-agent: Slackbot-LinkExpanding" will not stop it, because Slack states it does not currently honor robots.txt files, treating these fetches as user-initiated rather than crawling. To have your site blocklisted, Slack directs owners to contact its help system or @slackapi on Twitter.
About Operational Bots
Monitoring, security scanning, link previews, webhooks, and feed fetchers: the utility traffic that keeps integrations and shared links working.
Most of this traffic is service-critical or user-triggered. Webhooks and previews don't follow robots.txt, and blocking them tends to break things you rely on.
