What Google-Agent does
Google-Agent is the fetcher used by agents hosted on Google infrastructure when they navigate the web and perform actions that a user has requested. It identifies itself with mobile and desktop user agent strings that contain "compatible; Google-Agent". Because its fetches are triggered by a user action rather than autonomous crawling, Google states that it generally ignores robots.txt rules. Google is also experimenting with the Web Bot Auth protocol, using the https://agent.bot.goog identity, to authenticate these requests.
How to control Google-Agent
Target Google-Agent in your robots.txt with the Google-Agent token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow Google-Agent
User-agent: Google-Agent
Allow: /Block Google-Agent
User-agent: Google-Agent
Disallow: /Google publishes the IP addresses Google-Agent requests originate from in its user-triggered-agents.json file and recommends verifying requests against them because user agent strings can be spoofed.
How to identify Google-Agent
Temso identifies Google-Agenttraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
Google-AgentGoogle-Agent FAQ
What is Google-Agent?
Google-Agent is a fetcher deployed by agents hosted on Google infrastructure to navigate the web and perform actions when users request them. Its requests come from IP addresses Google publishes in the user-triggered-agents.json file.
How do I block Google-Agent?
You can add a robots.txt group with User-agent: Google-Agent followed by Disallow: /, but Google's documentation states that because these fetches are initiated by a user action, user-triggered fetchers like Google-Agent generally ignore robots.txt rules. Blocking reliably therefore requires server-side measures, such as filtering the IP ranges Google publishes in user-triggered-agents.json.
About AI Assistant Fetchers
Agents that fetch a page in real time because a user asked an AI assistant about it. They don't build an index; each visit maps to a live conversation.
Blocking these removes your pages from live AI answers at the exact moment a user is asking about you or your category.
