AddSearch AI
SafeUser-agent: AddSearchBot
Operator: AddSearch
AI training crawlers collecting content for LLM datasets. Blocking has zero SEO impact.
User-agent: AddSearchBot
Operator: AddSearch
User-agent: AI2Bot
Operator: Allen Institute for AI
User-agent: Ai2Bot-Dolma
Operator: Allen Institute for AI
User-agent: aiHitBot
Operator: aiHit
User-agent: AmazonBuyForMe
Operator: Amazon
User-agent: Amzn-User
Operator: Amazon
User-agent: Andibot
Operator: Andi
User-agent: anthropic-ai
Operator: Anthropic
User-agent: bedrockbot
Operator: Amazon
User-agent: AzureAI-SearchBot
Operator: Microsoft
User-agent: Big Sur AI
Operator: Big Sur AI
User-agent: Bravebot
Operator: Brave
User-agent: ChatGLM-Spider
Operator: Zhipu AI
User-agent: ChatGPT Agent
Operator: OpenAI
User-agent: ChatGPT Operator
Operator: OpenAI
User-agent: ChatGPT-User
Operator: OpenAI
User-agent: Claude-Code
Operator: Anthropic
User-agent: Claude-SearchBot
Operator: Anthropic
User-agent: Claude-User
Operator: Anthropic
User-agent: Claude-Web
Operator: Anthropic
User-agent: ClaudeBot
Operator: Anthropic
User-agent: Cloudflare-AutoRAG
Operator: Cloudflare
User-agent: cohere-ai
Operator: Cohere
User-agent: cohere-training-data-crawler
Operator: Cohere
User-agent: DeepSeekBot
Operator: DeepSeek
User-agent: Devin
Operator: Cognition AI
User-agent: DigitalOceanGenAICrawler
Operator: DigitalOcean
User-agent: DuckAssistBot
Operator: DuckDuckGo
User-agent: ExaBot
Operator: Exa
User-agent: FirecrawlAgent
Operator: Firecrawl
User-agent: Google-Gemini-CLI
Operator: Google
User-agent: GoogleAgent-URLContext
Operator: Google
User-agent: Google-Firebase
Operator: Google
User-agent: Google-Extended
Operator: Google
User-agent: GPTBot
Operator: OpenAI
User-agent: Grok
Operator: xAI
User-agent: GrokBot
Operator: xAI
User-agent: iAskBot
Operator: iAsk
User-agent: img2dataset
Operator: Open Source Community
User-agent: kagi-fetcher
Operator: Kagi
User-agent: Kimi-User
Operator: Moonshot AI
User-agent: KlaviyoAIBot
Operator: Klaviyo
User-agent: LINER Bot
Operator: LINER
User-agent: LinkupBot
Operator: Linkup
User-agent: Manus-User
Operator: Manus
User-agent: meta-webindexer
Operator: Meta
User-agent: meta-externalagent
Operator: Meta
User-agent: meta-externalfetcher
Operator: Meta
User-agent: MistralAI-Index
Operator: Mistral AI
User-agent: MistralAI-User
Operator: Mistral AI
User-agent: MyCentralAIScraperBot
Operator: Unknown
User-agent: NotebookLM
Operator: Google
User-agent: OAI-SearchBot
Operator: OpenAI
User-agent: OAI-AdsBot
Operator: OpenAI
User-agent: OpenAI
Operator: OpenAI
User-agent: Operator
Operator: OpenAI
User-agent: PanguBot
Operator: Huawei
User-agent: Perplexity-User
Operator: Perplexity AI
User-agent: PerplexityBot
Operator: Perplexity AI
User-agent: PhindBot
Operator: Phind
User-agent: GoogleAgent-Mariner
Operator: Google
User-agent: QuillBot
Operator: QuillBot
User-agent: quillbot.com
Operator: QuillBot
User-agent: SBIntuitionsBot
Operator: Unknown
User-agent: TavilyBot
Operator: Tavily
User-agent: CloudVertexBot
Operator: Google
User-agent: WRTNBot
Operator: WRTN
User-agent: YouBot
Operator: You