What Applebot-Extended does
Applebot-Extended is a control token, not a separate crawler. Disallowing it tells Apple not to use content crawled by Applebot for training Apple's foundation models, while leaving Applebot's crawling for Siri, Spotlight, and other Apple services untouched.
Should I opt out of Applebot-Extended?
Like Google-Extended, this separates training from visibility: you can opt out of Apple Intelligence training without giving up presence in Siri and Spotlight results.
How to control Applebot-Extended
Applebot-Extended is a robots.txt control token, not a crawler: no separate agent visits your site under this name. Add the rule below to opt out; omitting it (the default) leaves the content usable.
Opt out
User-agent: Applebot-Extended
Disallow: /No separate user agent appears in logs; Apple applies the token to content already fetched by Applebot.
Applebot-Extended FAQ
What is Applebot-Extended?
Applebot-Extended is a secondary Apple user agent that does not crawl webpages; it is only used to determine how data already crawled by Applebot may be used. Disallowing it in robots.txt opts your content out of training Apple's general purpose foundation models that power generative AI features, including Apple Intelligence, Services, and Developer Tools.
How do I block Applebot-Extended?
Add a robots.txt group with "User-agent: Applebot-Extended" followed by "Disallow: /". Because Applebot-Extended does not crawl, this only controls data usage: Applebot may still crawl your pages, and content that disallows Applebot-Extended can still be included in search results and remains discoverable through Spotlight, Siri, and Safari.
About AI Training Crawlers
Crawlers that collect content to train foundation models. Data gathered today shapes what future model versions know about your brand.
Blocking these doesn't affect live citations, but it limits what future models learn about you from your own site, leaving third-party sources to fill the gap.
