ClaudeBot is Anthropic's training crawler. It gathers publicly available web content used to train Claude models. It is a training crawler, not a search or answer crawler, so it has no direct role in what Claude retrieves or cites in real time.
Blocking ClaudeBot is a content-licensing decision: it keeps your material out of Anthropic's training data while leaving Claude's search and user-action fetchers unaffected if those remain allowed.
How to check and control it
To check, look in your robots.txt for a rule targeting this user-agent. To allow or block it:
User-agent: ClaudeBot
Disallow: / # blocks it — remove or set "Allow: /" to permitFor an instant check across your whole site, use SEO AEO Specialist's free AI crawler checker, and see the llms.txt & AI crawler guide.
FAQ
Does blocking ClaudeBot remove me from Claude's answers?
No. ClaudeBot only feeds model training. Claude's live search and on-demand browsing use Claude-SearchBot and Claude-User, which are separate user-agents. Blocking ClaudeBot opts you out of training but leaves those retrieval paths intact if they remain allowed.
Why would I block ClaudeBot?
Publishers block ClaudeBot when they don't want their content used to train Claude models, often for licensing or proprietary-content reasons. It has no impact on Google or Bing rankings. SEO AEO Specialist audits which AI training crawlers you allow so the choice is deliberate.
See where you stand
SEO AEO Specialist runs a free AI-visibility audit and hands you the exact fixes. One-off report €9, unlimited €19/mo.
Run your free audit →