FacebookBot
SafeUser-agent: FacebookBot
Category: Social Media Bots
SEO impact if blocked: 0/10
Meta
5 crawlers operated by Meta
https://www.facebook.com/Who Meta is, for site owners
Meta operates 5 crawlers in our directory, filed under Social Media Bots and AI & LLM Bots. Of those 5, 4 rated safe, 1 rated caution for crawl behaviour.
Blocking Meta carries an SEO impact score of 0/10 to 2/10 across its crawlers. 4 of them share the highest blocking impact here: Varies - Evaluate before blocking.
Our recommendation for Meta, set by its least reassuring crawler (Meta Web Indexer, rated caution): Generally safe to allow; provides legitimate crawling value.
Meta at a glance
| Operator | Meta |
|---|---|
| Crawlers in directory | 5 |
| Categories | Social Media Bots, AI & LLM Bots |
| SEO impact if blocked | 0/10 to 2/10 |
| Crawl behaviour | 4 rated safe, 1 rated caution |
| Website | www.facebook.com |
| Official crawler docs | Meta crawler documentation |
Crawlers operated by Meta
facebookexternalhit
SafeUser-agent:
facebookexternalhitCategory: Social Media Bots
SEO impact if blocked: 0/10
Meta Web Indexer
CautionUser-agent:
meta-webindexerCategory: AI & LLM Bots
SEO impact if blocked: 2/10
Meta-ExternalAgent
SafeUser-agent:
meta-externalagentCategory: AI & LLM Bots
SEO impact if blocked: 0/10
Meta-ExternalFetcher
SafeUser-agent:
meta-externalfetcherCategory: AI & LLM Bots
SEO impact if blocked: 0/10
How to block or allow Meta in robots.txt
These rules cover all 5 crawlers Meta operates. Add them to the robots.txt file at the root of your domain.
Block Meta completely
User-agent: FacebookBot
User-agent: facebookexternalhit
User-agent: meta-webindexer
User-agent: meta-externalagent
User-agent: meta-externalfetcher
Disallow: /
Allow Meta explicitly
User-agent: FacebookBot
User-agent: facebookexternalhit
User-agent: meta-webindexer
User-agent: meta-externalagent
User-agent: meta-externalfetcher
Allow: /
robots.txt is a request, not an enforcement mechanism. A crawler that ignores it needs a block at the server or CDN layer. Rules are case-sensitive on the path and matched against the user-agent token, not the full user-agent string.
Frequently asked questions about Meta
What crawlers does Meta run?
Meta operates 5 crawlers in our directory: FacebookBot (user-agent FacebookBot), facebookexternalhit (user-agent facebookexternalhit), Meta Web Indexer (user-agent meta-webindexer), Meta-ExternalAgent (user-agent meta-externalagent), Meta-ExternalFetcher (user-agent meta-externalfetcher).
Should I block Meta?
Generally safe to allow; provides legitimate crawling value. That guidance follows this operator's least reassuring crawler, Meta Web Indexer, rated caution, because allowing an operator allows every crawler it runs. The SEO impact score for blocking is 0/10 to 2/10, where 10 means blocking costs you the most visibility.
How do I block Meta in robots.txt?
Add the following to robots.txt at your domain root: User-agent: FacebookBot then User-agent: facebookexternalhit then User-agent: meta-webindexer then User-agent: meta-externalagent then User-agent: meta-externalfetcher then Disallow: /. One Disallow line covers all 5 user-agent tokens.
Does blocking Meta hurt my search rankings?
4 of these share the highest blocking impact here, rated: Varies - Evaluate before blocking. Google's own ranking crawler is Googlebot, which this operator does not run, so blocking Meta does not by itself remove you from Google's index. What it can change is whether AI assistants and analytics tools can read your pages.