What Meta-ExternalAgent does
Meta-ExternalAgent crawls the web for use cases such as training Meta's AI models and improving products by indexing content directly. Meta documents it alongside its other crawlers with supported robots.txt controls.
Should I allow Meta-ExternalAgent to crawl my site?
Meta's Llama models are among the most widely deployed open-weight models. Allowing Meta-ExternalAgent puts your content in the training pool for models that run far beyond Meta's own apps.
How to control Meta-ExternalAgent
Target Meta-ExternalAgent in your robots.txt with the Meta-ExternalAgent token. Rules apply per bot, so you can allow this one while blocking others, or the reverse.
Allow Meta-ExternalAgent
User-agent: Meta-ExternalAgent
Allow: /Block Meta-ExternalAgent
User-agent: Meta-ExternalAgent
Disallow: /User-agent string
As published by Meta. Remember that user-agent strings can be spoofed, so verify suspicious traffic against the operator's published IP ranges.
meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)Meta-ExternalAgent FAQ
What is Meta-ExternalAgent?
Meta-ExternalAgent is Meta's crawler that crawls the web for use cases such as training foundation AI models or improving Meta products by indexing content directly. It identifies itself with user-agent strings such as "meta-externalagent/1.1".
How do I block Meta-ExternalAgent?
Add a robots.txt group with "User-agent: meta-externalagent" followed by "Disallow: /" (Meta's documentation uses the lowercase token in its examples). Allow up to 24 hours for changes to take effect, because Meta's crawlers may cache the contents of your robots.txt file for up to 24 hours.
About AI Training Crawlers
Crawlers that collect content to train foundation models. Data gathered today shapes what future model versions know about your brand.
Blocking these doesn't affect live citations, but it limits what future models learn about you from your own site, leaving third-party sources to fill the gap.
