Claude-SearchBot
Operated by Anthropic · Powers Claude search & citations
Claude-SearchBot navigates the web to improve search result quality for Claude users. Anthropic states that disabling it may reduce your site's visibility and accuracy in Claude's search results. Unlike Claude-User, it isn't triggered by a specific user request; unlike ClaudeBot, it doesn't feed model training.
What it does
Claude-SearchBot navigates and analyzes web content to improve the relevance and accuracy of Claude's search responses, similar to how a traditional search engine crawler works.
It is not used to train Anthropic's underlying models — that's ClaudeBot's job — and it operates independently of Claude-User, which only fetches pages on demand for a specific user request.
Why we recommend keeping it open
- Anthropic says disabling it may reduce your site's visibility and accuracy in search results
- Not used to train Anthropic's models — indexing only
- Operates independently of ClaudeBot (training) and Claude-User (live fetches)
Identification & robots.txt
Claude-SearchBot
Additional verification information is available in Anthropic's official documentation.
User-agent: Claude-SearchBot
Allow: /