What OAI-SearchBot does
OAI-SearchBot crawls the web to build and maintain the index used by ChatGPT search. OpenAI documents that it is used to surface and link to websites in search results, and is not used to train OpenAI's models. OpenAI has since changed the user agent: it now opens with a full desktop Chrome string and reports version 1.4, where earlier requests used a short "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0" form. Log filters and firewall rules written against that older shape will miss current traffic, so match on the OAI-SearchBot token rather than the whole string.
Should I allow OAI-SearchBot to crawl my site?
This is the crawler that decides whether ChatGPT search can find and cite you at all. Blocking OAI-SearchBot removes your pages from ChatGPT's search index, independent of any GPTBot training decision.
How to control OAI-SearchBot
Target OAI-SearchBot in your robots.txt with the OAI-SearchBot token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow OAI-SearchBot
User-agent: OAI-SearchBot
Allow: /Block OAI-SearchBot
User-agent: OAI-SearchBot
Disallow: /OpenAI publishes OAI-SearchBot's IP ranges so crawl traffic can be verified against the official list.
User-agent string
As published by OpenAI. Remember that user-agent strings can be spoofed, so verify suspicious traffic against the operator's published IP ranges.
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbotOAI-SearchBot FAQ
What is OAI-SearchBot?
OAI-SearchBot is an OpenAI crawler that builds the search index behind ChatGPT search by surfacing websites in search results. It is separate from GPTBot, the crawler OpenAI uses to gather content for training its models, and is not used for model training.
How do I block OAI-SearchBot?
Add a robots.txt group with "User-agent: OAI-SearchBot" and "Disallow: /". OpenAI says it takes about 24 hours for its systems to adjust after a robots.txt change. Sites that block the bot will not appear in ChatGPT search answers, though they may still show as navigational links.
Why does OAI-SearchBot look like Chrome in my logs?
OpenAI changed the user agent so it now begins with a full desktop Chrome string on macOS before the "compatible; OAI-SearchBot/1.4" segment. Earlier requests used a much shorter form starting "Mozilla/5.0 AppleWebKit/537.36". Both contain the OAI-SearchBot token, so match on that token rather than on the leading browser string, which will keep working across future changes.
About AI Search Crawlers
Crawlers that build the retrieval indexes behind AI search products like ChatGPT search and Perplexity. They decide which pages can be cited as sources.
Blocking these keeps your content out of the source index: AI search products can't cite pages they can't crawl.
