What Mediapartners-Google does
Mediapartners-Google is the Google AdSense crawler. It visits participating publisher sites to determine page content so that relevant ads can be served, and it only crawls pages that display Google ads, indexing by URL so site.com and www.site.com are treated as distinct. Publishers have no control over crawl frequency, and page changes can take up to 1 or 2 weeks to be reflected in the AdSense index. As a special-case crawler it ignores the global User-agent: * group in robots.txt and only obeys rules that name it.
How to control Mediapartners-Google
Target Mediapartners-Google in your robots.txt with the Mediapartners-Google token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow Mediapartners-Google
User-agent: Mediapartners-Google
Allow: /Block Mediapartners-Google
User-agent: Mediapartners-Google
Disallow: /It ignores the global User-agent: * group in robots.txt and must be blocked by name, and genuine requests can be verified against Google's published special-crawlers.json IP ranges, with reverse DNS resolving to rate-limited-proxy-***-***-***-***.google.com.
How to identify Mediapartners-Google
Temso identifies Mediapartners-Googletraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
Mediapartners-GoogleMediapartners-Google FAQ
What is Mediapartners-Google?
Mediapartners-Google is the Google AdSense crawler that visits participating sites to determine their content so relevant ads can be shown. It is separate from Google's search crawler, so its activity does not affect search rankings, although the two systems share a cache to conserve bandwidth.
How do I block Mediapartners-Google?
Add a robots.txt group with User-agent: Mediapartners-Google and Disallow: /. It must be named explicitly because the AdSense crawler still crawls pages that are only disallowed under the global User-agent: * group; blocking it prevents AdSense from determining page content to serve relevant ads.
About Advertising Bots
Bots run by ad platforms to check landing pages, verify ad placements, and analyze content for brand safety and targeting.
If you run ads on a platform, its ad bots usually need access; blocking them can break ad approval and quality checks for your own campaigns.
