Caution SEO Tools

DotBot

Operated by Moz

Quick Facts

User-Agent:
DotBot
Category:
SEO Tools
Operator:
Moz
Safety:
Caution
Blocking Impact:
Low - No SEO ranking impact
SEO Impact Score:
2/10
Official docs:
moz.com

What is DotBot?

DotBot is the web crawler for Moz, a leading SEO software company. It gathers data for their Link Explorer and other tools.

DotBot is the web crawler for Moz, a leading SEO software company. It gathers data for their Link Explorer and other tools. DotBot is a commercial SEO analytics crawler operated by Moz. It builds backlink graphs, crawls for technical SEO issues, and tracks keyword rankings. The user-agent DotBot is well-known and respected in the SEO industry. Blocking it removes your domain from Moz's index, preventing competitors from analysing your backlink profile via their platform. However, other Moz users also lose visibility into links pointing TO your site - weigh this trade-off carefully.

What happens if you block DotBot?

✅ **No SEO Impact** - Blocking DotBot will not affect your search rankings. The only consequence is that your domain's data will not appear in Moz's platform. This can be advantageous if you want to conceal your backlink profile or keyword strategy from competitors who use Moz.
Consider blocking based on your content strategy.

How to block DotBot with robots.txt

<code>User-agent: DotBot</code> - Matching is case-insensitive. Robots.txt is fetched from the root of each subdomain separately.

Block completely (robots.txt)
User-agent: DotBot Disallow: /
Allow all (robots.txt)
User-agent: DotBot Allow: /
Block private only (robots.txt)
User-agent: DotBot Disallow: /private/ Disallow: /api/ Disallow: /admin/ Allow: /
Nginx server block
# Nginx: Hard-block DotBot if ($http_user_agent ~* "DotBot") { return 403 "Bot blocked"; }
Apache .htaccess
# Apache: Hard-block DotBot SetEnvIfNoCase User-Agent "DotBot" bad_bot Order Allow,Deny Allow from all Deny from env=bad_bot
Meta robots tag
<meta name="robots" content="noindex, nofollow">
X-Robots-Tag header
X-Robots-Tag: noindex, nofollow

Is DotBot safe to allow?

⚠️ **Use Caution with DotBot.** While operated by Moz for stated legitimate purposes, this bot collects your content for uses you may not want to support (commercial data aggregation). It generally respects robots.txt but may revisit pages more frequently than needed. Evaluate your content strategy: if you're concerned about your data being used for these purposes, block it.

What does DotBot do?

Understanding DotBot's purpose helps you decide whether to allow or block it.

  • Analyzes website structure and SEO health
  • Tracks backlinks and referring domains
  • Monitors keyword rankings and competitor analysis
  • Crawls sites for technical SEO audits
  • Generates reports for SEO professionals

Frequently Asked Questions

What is the official user-agent string for DotBot?
The official user-agent string for DotBot is: DotBot. This is the exact string you must use in robots.txt, Nginx, Apache, or Cloudflare firewall rules to target this bot. User-agent matching in robots.txt is case-insensitive, but the string must be spelled correctly. You can verify that a request genuinely comes from DotBot by performing a reverse-DNS lookup on the source IP - legitimate bots resolve back to their operator's domain.
Is DotBot safe?
⚠️ **Use Caution with DotBot.** While operated by Moz for stated legitimate purposes, this bot collects your content for uses you may not want to support (commercial data aggregation). It generally respects robots.txt but may revisit pages more frequently than needed. Evaluate your content strategy: if you're concerned about your data being used for these purposes, block it.
Will blocking DotBot hurt my SEO?
✅ **No SEO Impact** - Blocking DotBot will not affect your search rankings. The only consequence is that your domain's data will not appear in Moz's platform. This can be advantageous if you want to conceal your backlink profile or keyword strategy from competitors who use Moz.
How do I block DotBot in robots.txt?
Add the following lines to your /robots.txt file: 
User-agent: DotBot
Disallow: /
This instructs DotBot not to crawl any path on your site. The Disallow: / directive covers the entire domain including subfolders. To only block specific sections, replace / with the path (e.g., Disallow: /blog/). Note: robots.txt is publicly readable - any bot or human can inspect it at yourdomain.com/robots.txt.
Does DotBot respect robots.txt?
⚠️ DotBot may not always respect robots.txt. For guaranteed blocking, combine robots.txt with server-level rules (Nginx if/return 403, Apache SetEnvIf, or Cloudflare WAF).
How do I verify if DotBot is crawling my site?
Search your web server access logs for the string DotBot (case-insensitive grep: grep -i "DotBot" /var/log/nginx/access.log). You can also check Google Search Console → Coverage → Crawl Stats for Googlebot variants. For DotBot specifically, filter by user-agent in your log analysis tool (GoAccess, AWStats, etc.).
What is the crawl frequency of DotBot?
Crawl frequency data for DotBot is not publicly documented. Monitor your logs to understand actual visit patterns.
Can I block DotBot from specific pages only?
Yes. Instead of a global Disallow: / you can restrict DotBot to specific paths: 
User-agent: DotBot
Disallow: /private/
Disallow: /staging/
Allow: /
This allows DotBot everywhere except the listed paths. Path matching in robots.txt uses prefix matching - Disallow: /private/ blocks /private/page.html but NOT /public/private/.
Why would I want to block DotBot?
There are two main reasons to block DotBot: 1. **Competitive intelligence**: Moz users (your competitors) can analyse your backlink profile, keyword rankings, and site structure. Blocking prevents this. 2. **Crawl budget**: If your site is large, SEO tool crawlers consume crawl bandwidth without contributing to search rankings. Blocking frees up server resources for search engine bots. The trade-off: you also become invisible in Moz's backlink database, meaning links pointing TO your site from other domains won't appear in competitor reports either.

Related Bots