What Coveobot does
Coveobot is the crawler behind Coveo's Web source, which indexes website content into the Coveo Platform. It works like a search engine bot: it starts from a configured URL and discovers other pages by following site navigation and hyperlinks. By default it obeys all crawler directives it encounters, including robots.txt Allow and Disallow rules, noindex and nofollow meta tags, X-Robots-Tag headers, and rel="nofollow" anchors, though the Coveo customer who configured the source can enable override settings for these as a last resort. The crawler is coded to request no more than one page per second.
How to control Coveobot
Target Coveobot in your robots.txt with the Coveobot token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow Coveobot
User-agent: Coveobot
Allow: /Block Coveobot
User-agent: Coveobot
Disallow: /Coveobot honors a robots.txt Crawl-delay directive whenever it is slower than its built-in one-page-per-second limit, and it supports coveobot-specific rules in robots.txt, meta tags, and X-Robots-Tag headers that take precedence over all-robot directives.
How to identify Coveobot
Temso identifies Coveobottraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.
CoveobotCoveobot FAQ
What is Coveobot?
Coveobot is the crawler for Coveo's Web source, which indexes website content into the Coveo Platform. It needs only a starting URL and then discovers pages by following site navigation and hyperlinks, similar to a web search engine bot.
How do I block Coveobot?
Add a robots.txt group with "User-agent: coveobot" and Disallow rules for the paths you want off limits; you can also target it with coveobot-named meta tags or X-Robots-Tag directives. Note that Coveo documents override settings that let a source administrator bypass robots.txt and meta directives, though Coveo describes overriding as a last resort.
About AI Search Crawlers
Crawlers that build the retrieval indexes behind AI search products like ChatGPT search and Perplexity. They decide which pages can be cited as sources.
Blocking these keeps your content out of the source index: AI search products can't cite pages they can't crawl.
