Claude-SearchBot: Anthropic's search crawler
Claude-SearchBot is Anthropic's search-dedicated agent: it explores the web to improve the quality and relevance of the results Claude offers when searching for sources for its users. It is distinct from ClaudeBot (model training) and Claude-User (live browsing) — three agents, three robots.txt tokens, three independent decisions.
Technical profile
- Operator
- Anthropic
- Role
- Search result quality
- robots.txt token
Claude-SearchBot- User-agent (example)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-SearchBot/1.0; +Claude-SearchBot@anthropic.com)- Respects robots.txt
- Yes (documented by Anthropic)
- Official IP ranges
- Not published by the operator (IP verification impossible)
- Official documentation
- Anthropic crawling documentation
Its place in Claude's search
When Claude runs a web search to answer, it relies on results whose quality is shaped by Claude-SearchBot. Allowing this agent maximizes the chances your pages are well understood, properly ranked and offered as sources in Claude's answers — Anthropic's equivalent of OAI-SearchBot's role at OpenAI.
For a site pursuing GEO visibility, it's a bot to let through: it belongs to the “citation” family, the one that can send readers your way, as opposed to the “training” family, which doesn't.
Claude-SearchBot in your logs
Its “Claude-SearchBot” signature appears in your server logs' user-agent field. Its volume is usually lower than ClaudeBot's: it targets understanding pages useful to search rather than broad collection. As with other Anthropic agents, no official IP ranges are published to formally verify it.
Verifying it's really Claude-SearchBot
Since Anthropic publishes no official IP ranges, a “Claude-SearchBot” hit cannot be formally authenticated. Label it “unverifiable” and judge by behavior: moderate volume and targeted crawling are consistent; a flood of requests under this name is not. Our analyzer applies exactly this caution.
Blocking or allowing Claude-SearchBot (robots.txt)
Blocking Claude-SearchBot degrades your potential visibility in Claude's sourced answers, with nothing gained — the training opt-out goes through ClaudeBot, not this agent. Unless your content should be excluded from every engine, let it access the site.
To block it:
User-agent: Claude-SearchBot Disallow: /
To explicitly allow it:
User-agent: Claude-SearchBot Allow: /
Frequently asked questions
What's the difference between Claude-SearchBot and ClaudeBot?
ClaudeBot collects content to train Claude models; Claude-SearchBot improves the quality of search results Claude offers its users. The former sends you no readers, the latter is part of the circuit that can cite your pages. Two distinct robots.txt tokens, two independent decisions.
Should I block Claude-SearchBot?
No, unless you refuse all visibility in AI engines: blocking it reduces your chances of appearing as a source in Claude's answers, with nothing gained. If your goal is a training opt-out, block ClaudeBot and let Claude-SearchBot through.
How do I detect Claude-SearchBot in my logs?
Look for the “Claude-SearchBot” substring in your access.log user-agent field — or drop your files into our analyzer, which isolates each Anthropic agent, shows the pages visited and honestly flags that these bots cannot be verified by IP.
Sources
Every fact on this page is backed by a primary source — the operator's official documentation and, where it exists, its published IP list.
See what this bot does on your site
Drop your access.log files into the Crawl Lab log analyzer: it isolates Claude-SearchBot, verifies its authenticity and lists the pages it requests — entirely in your browser, no upload.
Analyze my logs