What Google Site Verifier does
Google-Site-Verification is one of Google's user-triggered fetchers. It retrieves Search Console verification tokens when a site owner requests verification of domain ownership through Google's tools, so each fetch is initiated by a user action rather than by automated crawling. It identifies itself with the user-agent string Mozilla/5.0 (compatible; Google-Site-Verification/1.0).
How to control Google Site Verifier
Target Google Site Verifier in your robots.txt with the Google-Site-Verification token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow Google Site Verifier
User-agent: Google-Site-Verification
Allow: /Block Google Site Verifier
User-agent: Google-Site-Verification
Disallow: /Because the fetch is requested by a user, Google states its user-triggered fetchers generally ignore robots.txt rules, and Google publishes their IP ranges in JSON files (such as user-triggered-fetchers.json) with reverse DNS hostnames under gae.googleusercontent.com or google.com for verification.
How to identify Google Site Verifier
Temso identifies Google Site Verifiertraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
Google-Site-VerificationGoogle Site Verifier FAQ
What is Google Site Verifier?
Google-Site-Verification is a Google fetcher that retrieves Search Console verification tokens when a site owner asks Google to confirm ownership of a domain. It only fetches in response to that user request.
How do I block Google Site Verifier?
You can add a robots.txt group with "User-agent: Google-Site-Verification" and Disallow rules, but Google states that its user-triggered fetchers generally ignore robots.txt because the fetch was requested by a user. Since the fetch only happens when someone attempts Search Console verification for your domain, seeing it usually means a verification attempt was made. Google also notes the user-agent string can be spoofed and publishes IP ranges for verifying genuine requests.
About Operational Bots
Monitoring, security scanning, link previews, webhooks, and feed fetchers: the utility traffic that keeps integrations and shared links working.
Most of this traffic is service-critical or user-triggered. Webhooks and previews don't follow robots.txt, and blocking them tends to break things you rely on.
