AhrefsBot
SafeUser-agent: AhrefsBot
Operator: Ahrefs
SEO Tools Directory
Commercial SEO platform crawlers (Ahrefs, Semrush etc.). Blocking hides your data from competitors.
21 total
17 safe
4 caution
0 aggressive
All 21 SEO Tools
AudigentAdBot
SafeUser-agent:
AudigentAdBotOperator: Audigent
AwarioRssBot
SafeUser-agent:
AwarioRssBotOperator: Awario
AwarioSmartBot
SafeUser-agent:
AwarioSmartBotOperator: Awario
Brandwatch
SafeUser-agent:
BrandwatchOperator: Brandwatch
Chrome-Lighthouse
SafeUser-agent:
Chrome-LighthouseOperator: Google
DataForSeoBot
SafeUser-agent:
DataForSeoBotOperator: DataForSEO
DotBot
CautionUser-agent:
DotBotOperator: Moz
Google Page Speed Insights
SafeUser-agent:
Google Page Speed InsightsOperator: Google
Jetslide
SafeUser-agent:
JetslideOperator: Unknown
Meltwater
SafeUser-agent:
MeltwaterOperator: Meltwater
MJ12bot
CautionUser-agent:
MJ12botOperator: Majestic
NeticleBot
SafeUser-agent:
NeticleBotOperator: Neticle
Netvibes
SafeUser-agent:
NetvibesOperator: Unknown
peer39_crawler/1.0
CautionUser-agent:
peer39_crawlerOperator: Unknown
Screaming Frog SEO Spider
SafeUser-agent:
Screaming Frog SEO SpiderOperator: Screaming Frog
SearchmetricsBot
SafeUser-agent:
SearchmetricsBotOperator: Searchmetrics
SemrushBot
SafeUser-agent:
SemrushBotOperator: Semrush
SemrushBot-OCOB
CautionUser-agent:
SemrushBot-OCOBOperator: Semrush
SemrushBotSwa
SafeUser-agent:
SemrushBotSwaOperator: Semrush
Sidetrade Indexer Bot
SafeUser-agent:
Sidetrade Indexer BotOperator: Sidetrade
FAQ
How many SEO Tools are tracked?
We currently track 21 SEO Tools with user-agent strings, safety ratings, and blocking rules.
Which SEO Tools should I never block?
Never block bots with 'critical' impact rating - there are 0 in this category. Blocking them removes your site from their search results.
Are all SEO Tools safe?
Out of 21 bots in this category: 17 safe, 4 caution, 0 aggressive.
How do I block all SEO Tools at once?
Generate a combined robots.txt block rule for all SEO Tools using our bulk robots.txt generator, or manually list each User-agent + Disallow: / directive.