Algolia Crawler

Operated by AlgoliaSearch Engine

Extracts site content and makes it searchable via Algolia.

How to identify Algolia Crawler

Temso identifies Algolia Crawlertraffic by matching the request's User-Agent header against this pattern. User-agent strings can be spoofed, so treat a match as an indication, not proof.

Algolia Crawler

About Search Engine Crawlers

Classic search indexers that now also feed AI features: Google's AI Overviews and AI Mode build on Googlebot's index, and Microsoft Copilot builds on Bingbot's.

Blocking these removes you from both traditional rankings and the AI answers built on top of those indexes, usually the most costly block of all.

Other Search Engine Crawlers

Googlebot

Google's main indexer: the crawl behind Search, AI Overviews, and AI Mode.

GoogleOther

Google's generic crawler for product teams: research and development fetches outside Search.

Bingbot

Microsoft's indexer: the crawl behind Bing and the grounding index for Copilot.

See which bots visit your site

Temso helps marketers track every AI crawler and bot that visits your site, and runs regular audits to make sure your website is always performing at its full potential.

Explore Website Audits
Website audit dashboard showing bot visits, citation-bot visits, and a table of AI bots crawling the site
← Back to the AI Bot Directory