meta-externalfetcher: user agent, IP ranges and robots.txt
meta-externalfetcher fetches individual links at a user's request, for Meta's AI agent features.
- Operator
- Meta
- Type
- User request · Fetches a page live because someone asked an assistant about it
- robots.txt token
meta-externalfetcher- Follows robots.txt
- Not always (user-triggered)
- IP list
- not published
- Official docs
- developers.facebook.com
meta-externalfetcher user agent
User agent
meta-externalfetcher/1.1
Meta also publishes a longer form with a link to its crawler documentation. Match on "meta-externalfetcher".
How to block or allow meta-externalfetcher in robots.txt
Meta: "this crawler may bypass robots.txt rules", because a user requested the fetch.
Block meta-externalfetcher
User-agent: meta-externalfetcher Disallow: /
Allow meta-externalfetcher
User-agent: meta-externalfetcher Allow: /
Put the rules in /robots.txt at the root of each host. Check the result with our AI Crawler Checker, which also catches firewall and CDN blocks that robots.txt doesn't show.
Should you block meta-externalfetcher?
Usually allow it, like other user-triggered fetchers. Blocking it takes a firewall rule on the user agent.
How to verify real meta-externalfetcher traffic
Meta doesn't publish an IP list for it.
To check every request in your server log at once, use the free AI Bot Log Analyzer. It runs in your browser and flags requests that use a bot's name from outside its official IP list.
Frequently asked questions
What is meta-externalfetcher?
meta-externalfetcher fetches individual links at a user's request, for Meta's AI agent features.
How do I block meta-externalfetcher?
Add "User-agent: meta-externalfetcher" followed by "Disallow: /" to robots.txt, but it may not be respected: Meta: "this crawler may bypass robots.txt rules", because a user requested the fetch. For a hard block, add a firewall or CDN rule that refuses requests whose user agent contains "meta-externalfetcher".
Is a request claiming to be meta-externalfetcher really from Meta?
You can't be sure. Meta doesn't publish an IP list for it. Treat the user agent as a claim, not proof.
Checked against Meta's documentation on 2026-11-06. Vendors change user agents and policies, so confirm in the official docs before relying on them.
Related crawlers
- meta-externalagentMeta · Training
- ChatGPT-UserOpenAI · User request
- Perplexity-UserPerplexity · User request
- Claude-UserAnthropic · User request
- All AI crawlersThe full list with user agents and purposes.
The free AI Crawler Checker tests robots.txt and firewall rules for 14 AI crawlers in one run.