What Google-Safety does
Google-Safety is one of Google's special-case crawlers and handles abuse-specific crawling, such as malware discovery for publicly posted links on Google properties. It identifies itself with the user agent "Google-Safety" and, unlike Google's common crawlers, it ignores robots.txt rules. It crawls from IP ranges distinct from Google's common crawlers, published in the special-crawlers.json object, with reverse DNS hostnames matching the pattern rate-limited-proxy-***-***-***-***.google.com.
How to control Google-Safety
Target Google-Safety in your robots.txt with the Google-Safety token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow Google-Safety
User-agent: Google-Safety
Allow: /Block Google-Safety
User-agent: Google-Safety
Disallow: /Google-Safety ignores robots.txt rules entirely; genuine requests can be verified against the IP ranges published in Google's special-crawlers.json object or by reverse DNS hostnames matching rate-limited-proxy-***-***-***-***.google.com.
How to identify Google-Safety
Temso identifies Google-Safetytraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
Google-SafetyGoogle-Safety FAQ
What is Google-Safety?
Google-Safety is a special-case Google crawler used for abuse-specific crawling, such as malware discovery for publicly posted links on Google properties. It identifies itself with the user agent "Google-Safety".
How do I block Google-Safety?
Robots.txt will not block it: Google's documentation states that the Google-Safety user agent ignores robots.txt rules, so a "User-agent: Google-Safety" group has no effect. Blocking it would require server or firewall rules based on the IP ranges Google publishes in special-crawlers.json, which are separate from the ranges its common crawlers use.
About Operational Bots
Monitoring, security scanning, link previews, webhooks, and feed fetchers: the utility traffic that keeps integrations and shared links working.
Most of this traffic is service-critical or user-triggered. Webhooks and previews don't follow robots.txt, and blocking them tends to break things you rely on.
