What Cookiebot does
The Cookiebot scanner crawls websites for the Cookiebot consent management platform to find the cookies and tracking technologies in use. Scans are performed from a published pool of static IP addresses, and Cookiebot states it is technically impossible for its scanner to operate from any other address. The scanner identifies itself with a user agent containing 'Cookiebot/1.0; +http://cookiebot.com/', and Cookiebot recommends matching the 'Cookiebot/1.0' substring because the full user agent string may change.
How to identify Cookiebot
Temso identifies Cookiebottraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
Cookiebot\/\dCookiebot FAQ
What is Cookiebot?
Cookiebot is the scanner behind the Cookiebot consent management platform; it crawls a customer's website to detect the cookies and tracking technologies in use. It identifies itself with a user agent containing 'Cookiebot/1.0' and scans only from a published pool of static IP addresses.
Can I block Cookiebot with robots.txt?
Cookiebot's documentation does not describe a robots.txt token or robots.txt handling for its scanner, which visits sites in connection with the site's own Cookiebot account. The documented ways to filter or block it are matching 'Cookiebot/1.0' in the user agent or using the published static IP list.
About Operational Bots
Monitoring, security scanning, link previews, webhooks, and feed fetchers: the utility traffic that keeps integrations and shared links working.
Most of this traffic is service-critical or user-triggered. Webhooks and previews don't follow robots.txt, and blocking them tends to break things you rely on.
