What IAS Crawler does
Integral Ad Science operates two crawlers. The IAS Crawler (ias_crawler) performs the primary indexing that supports IAS content rating and certification services for brands, agencies, ad networks, and publishers, while a secondary crawler, IAS Wombles (ias_wombles), captures data used to refine IAS invalid traffic detection models. Both adhere to the Standard for Robot Exclusion and fetch a copy of a site's robots.txt file before each crawl. IAS notes that blocking the IAS Crawler makes a site inaccessible to its partner advertisers.
How to control IAS Crawler
Target IAS Crawler in your robots.txt with the ias_crawler token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow IAS Crawler
User-agent: ias_crawler
Allow: /Block IAS Crawler
User-agent: ias_crawler
Disallow: /IAS publishes its crawler IP ranges in a JSON file linked from its site indexing policy so administrators can verify crawler origin, and the crawlers pick up a fresh copy of robots.txt before each crawl.
How to identify IAS Crawler
Temso identifies IAS Crawlertraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
ias_crawlerIAS Crawler FAQ
What is IAS Crawler?
IAS Crawler (ias_crawler) is Integral Ad Science's indexing crawler. It analyzes page content to support IAS content rating and certification services used by brands, agencies, ad networks, and publishers.
How do I block IAS Crawler?
Add a robots.txt group with User-agent: ias_crawler and Disallow: / (IAS also operates ias_wombles, which needs its own group if you want to block it too). The crawlers fetch robots.txt before each crawl; IAS asks administrators who change their crawling instructions to contact IAS directly so the revised directives are implemented promptly, and notes that blocking the IAS Crawler makes a site inaccessible to its partner advertisers.
About Advertising Bots
Bots run by ad platforms to check landing pages, verify ad placements, and analyze content for brand safety and targeting.
If you run ads on a platform, its ad bots usually need access; blocking them can break ad approval and quality checks for your own campaigns.
