What Brightbot does
Brightbot is Bright Data's data collection crawler. It has a built-in cache layer that prevents repeated downloads of the same data within a 24-hour period, and it treats data behind logins as private, blocking authentication cookies and password entry by default. It identifies itself with the user agent "Brightbot 1.0" and crawls from the IP subnet 82.97.199.0/24. Bright Data also runs health monitors that enforce rate limits if a domain's responsiveness degrades in correlation with Brightbot traffic.
How to control Brightbot
Target Brightbot in your robots.txt with the Brightbot token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow Brightbot
User-agent: Brightbot
Allow: /Block Brightbot
User-agent: Brightbot
Disallow: /Brightbot can be verified by combining its "Brightbot 1.0" user agent with its published source IP subnet 82.97.199.0/24, and Bright Data documents a collectors.txt file as an additional way for webmasters to set data collection guidelines.
How to identify Brightbot
Temso identifies Brightbottraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
BrightbotBrightbot FAQ
What is Brightbot?
Brightbot is Bright Data's web crawler for data collection, identified by the user agent "Brightbot 1.0" and the source IP subnet 82.97.199.0/24. It caches data so the same content is not downloaded again within 24 hours and does not collect data behind logins.
How do I block Brightbot?
Add a robots.txt group with "User-agent: Brightbot" followed by "Disallow: /". Bright Data also documents a collectors.txt file for defining data collection guidelines (such as PII protection and endpoint restrictions) and a WebMaster Console, verified via a DNS token, for tracking and controlling collection on your site.
About SEO & Analytics Crawlers
Crawlers behind SEO and marketing-intelligence platforms: backlink databases, site audits, rank tracking, and brand monitoring.
Blocking these has no effect on search or AI visibility. It mainly limits what third-party tools (including your competitors' tools) can see about your site.
