What GPTBot does
GPTBot crawls publicly available web content that may be used to improve and train OpenAI's foundation models. It is distinct from OAI-SearchBot (search index) and ChatGPT-User (live user fetches): blocking GPTBot does not remove a site from ChatGPT search.
Should I allow GPTBot to crawl my site?
GPTBot shapes what future GPT models inherently know about your brand. Blocking it doesn't affect today's citations, but it means the next model generation learns about you only from third-party sources, and you give up authorship of your own story in the training data.
How to control GPTBot
Target GPTBot in your robots.txt with the GPTBot token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow GPTBot
User-agent: GPTBot
Allow: /Block GPTBot
User-agent: GPTBot
Disallow: /OpenAI publishes GPTBot's IP ranges so crawl traffic can be verified against the official list.
User-agent string
As published by OpenAI. Remember that user-agent strings can be spoofed, so verify suspicious traffic against the operator's published IP ranges.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.2; +https://openai.com/gptbotGPTBot FAQ
What is GPTBot?
GPTBot is OpenAI's crawler that collects public web content that may be used to train its generative AI foundation models. It is distinct from OAI-SearchBot, which powers ChatGPT search, and from ChatGPT-User, which is triggered by individual user actions.
How do I block GPTBot?
Add a robots.txt group with "User-agent: GPTBot" and "Disallow: /". OpenAI states that disallowing GPTBot signals a site's content should not be used in training generative AI foundation models. You can verify genuine GPTBot requests against the IP ranges published at openai.com/gptbot.json.
About AI Training Crawlers
Crawlers that collect content to train foundation models. Data gathered today shapes what future model versions know about your brand.
Blocking these doesn't affect live citations, but it limits what future models learn about you from your own site, leaving third-party sources to fill the gap.
