What OhDearBot does
The Oh Dear crawler visits sites monitored by Oh Dear customers to detect broken links, find mixed content on HTTPS pages, validate sitemaps, and check outgoing links from other monitored sites that point to yours. Its requests appear as standard HTTP GET requests, with a user-agent string that varies by check type but always begins with "OhDear.app" followed by a documentation URL. The crawler throttles itself and respects robots.txt by default, so its crawling should cause minimal server impact.
How to identify OhDearBot
Temso identifies OhDearBottraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
OhDear\.appOhDearBot FAQ
What is OhDearBot?
OhDearBot is the crawler of the Oh Dear monitoring service, which visits customer sites to detect broken links and mixed content, validate sitemaps, and verify outgoing links. It identifies itself with user-agent strings that start with "OhDear.app" and link to the relevant feature documentation.
Can I block OhDearBot with robots.txt?
Oh Dear states that the crawler respects robots.txt by default, so disallow rules apply to its crawling. You can also filter or limit it by matching its user-agent strings, which all begin with "OhDear.app", and Oh Dear provides guidance for handling the crawler in web application firewalls such as Cloudflare.
About Operational Bots
Monitoring, security scanning, link previews, webhooks, and feed fetchers: the utility traffic that keeps integrations and shared links working.
Most of this traffic is service-critical or user-triggered. Webhooks and previews don't follow robots.txt, and blocking them tends to break things you rely on.
