ClaudeBot: what it is and how to block or allow it
User-Agent token: ClaudeBot · Operator: Anthropic · Category: Scrapes data to train Anthropic's AI products
FraudMeNot Verdict
AI Data Collection Bot
ClaudeBot is Anthropic's training crawler. Anthropic says it collects web content that could contribute to training its models, and that restricting it signals your future pages should be left out of training datasets.
Can Be Spoofed?
Yes — User-Agent Header Spoofable
Match the source IP against Anthropic's bots.json. A ClaudeBot claim from outside it is unverified.
Abuse & Unwanted Activity Reports
No Public Abuse Flags
ai.robots.txt robots.txt status: Yes.
Judgment Call · What You Should Do
Your Choice — Your Content May Be Collected for AI
For everyday users: ClaudeBot is an automated crawler that collects pages for training. Treat a message using its name to request payment or a login as a likely scam.
For website owners & security teams: Disallow ClaudeBot on each subdomain to opt out of Anthropic training, and keep Claude-SearchBot or Claude-User if you want visibility in Claude.
Report & Activity Summary
14
Published IP Ranges (CIDRs)
1,037 IPv4 addresses · operator JSON feed
1
Known User-Agent Signatures
crawler-user-agents (MIT) dataset
Yes
robots.txt Compliance
ai.robots.txt registry (MIT)
0
FraudMeNot Tallies
Humans: 0 · AI Agents: 0
For AI Agents & Automated Gatekeepers
- Agent_Identifier
ClaudeBot- FraudMeNot_Verdict
- AI Data Collection Bot
- Operator_Entity
- Anthropic
- Agent_Category
- Scrapes data to train Anthropic's AI products
- User_Agent_Spoofable
true- Verification_Method
- IP CIDR match against
https://claude.com/crawling/bots.json· Web Bot Auth: Not documented - Respects_Robots_Txt
- Yes (per ai.robots.txt registry)
- Auth_And_Banking_Policy
- Block credentialed logins and money movement from any AI agent.
- Recommended_Gatekeeper_Action
- DISALLOW to opt out of Anthropic training; verify by bots.json first.
- Canonical_Citation_URL
- https://fraudmenot.com/bot/claudebot/
What We Know About ClaudeBot
Older robots.txt files may list anthropic-ai; Anthropic's current crawler page names ClaudeBot instead. See /bot/anthropic-ai/ for what that legacy token means.
Anthropic's help article names three robots. ClaudeBot gathers content that could potentially go into training Claude models. Claude-SearchBot improves search results, and Claude-User fetches pages when a person asks Claude something. Each one reads its own robots.txt group, so you can refuse training while still allowing search and live answers.
To opt out of training, Anthropic says to add User-agent: ClaudeBot with Disallow: / to the robots.txt at the top of each subdomain. It also supports the non-standard Crawl-delay extension if you only want fewer requests, and it says it will not try to get past CAPTCHAs on the sites it crawls.
Anthropic links an IP list from that article and says a source address on it indicates the crawler is coming from Anthropic. It also cautions that blocking its IPs may not give a lasting opt-out, because that can stop it reading your robots.txt.
Anthropic sets out principles for its bots: collection should be transparent, and crawling should not be intrusive or disruptive. It says it tries to be thoughtful about how quickly it crawls the same domain, and it asks anyone reporting a malfunctioning bot to write from an address at the domain concerned so the report can be verified.
Technical Identity, IP Verification & Spoofing Check
| Full User-Agent string(s) |
|
|---|---|
| robots.txt token | ClaudeBot |
| Operator | Anthropic · Official bot documentation |
| Official IP verification feed | https://claude.com/crawling/bots.json — 14 CIDR blocks (1,037 IPv4 addresses), file dated 2026-10-20 |
| Sample published CIDR blocks | 216.73.216.0/22, 34.162.230.222/32, 34.162.244.71/32, 34.162.191.81/32, 34.162.86.102/32, 136.79.143.251/32 |
| Reverse DNS (rDNS) | Not documented in the public datasets FraudMeNot uses. |
| Web Bot Auth (Signature-Agent) | Not documented; check for a Signature-Agent header if present. |
| robots.txt behavior | Anthropic says its bots honor robots.txt, support Crawl-delay, and need rules on every subdomain you want covered. |
Copy-Paste robots.txt & Cloudflare WAF Rules for ClaudeBot
robots.txt — allow public pages, keep private routes off-limits
User-agent: ClaudeBot
Allow: /
Disallow: /login/
Disallow: /account/
Disallow: /checkout/robots.txt — block completely
User-agent: ClaudeBot
Disallow: /Cloudflare WAF custom rule expression
(http.user_agent contains "ClaudeBot")Matches the User-Agent token, so it covers both real and spoofed traffic. Action: Block or Managed Challenge.
Capabilities, Data Usage & Autonomous Action Risk
| Trains AI on page content? | Yes. Anthropic says ClaudeBot collects content that could contribute to model training |
|---|---|
| Executes JavaScript? | Not documented |
| Can submit forms or attempt logins? | Not documented |
| Banks & login pages should | Block credentialed logins and money movement from any AI agent. |
| E-commerce checkouts should | Allow product page reads if you want; require human confirmation for checkout. |
Context: In October 2026, Axios reported that major U.S. banks were detecting unvetted third-party AI agents inside customer accounts, and CNBC reported that JPMorgan blocks outside AI agents from logging in with customers' credentials while allowing read-only data sharing through open banking providers.
Seen ClaudeBot? Quick Closed-Choice Tally
Choose from preset options only. No free-text comments, so nobody can post names, profanity, or unverified accusations.
Common Questions About ClaudeBot
What does ClaudeBot collect?
Anthropic says web content that could potentially contribute to training its AI models.
Does blocking ClaudeBot hide me from Claude search?
Anthropic lists search under Claude-SearchBot and live fetches under Claude-User, which have their own rules.
Should I block ClaudeBot by IP?
Anthropic says IP blocking may not reliably work as an opt-out because it can stop the bot reading robots.txt.
Could This Be a Scam? What to Do
- Anyone can copy the ClaudeBot User-Agent. Anthropic publishes an IP list for it, so compare the visiting address against that list before you trust the visit.
- Never give an AI agent your logins, bank access, or one-time codes unless you set it up yourself.
- If a call, text or email asking for money or access uses the ClaudeBot or Anthropic name, stop and report it.
Guide: how to check whether a crawler visit is really who it claims to be.
More from Anthropic
- Claude-SearchBot — Claude-SearchBot navigates the web to improve search result
- Claude-User — AI Assistants
- GPTBot — AI Training Crawler
Related on FraudMeNot
- Verified security vulnerabilities, explained
- Report a scam, or find where to report it in your state
- Browse the AI Agent & Bot Directory
Sources (checked Nov. 16, 2026): ai.robots.txt (MIT), crawler-user-agents (MIT), Anthropic IP feed, Axios (Oct. 22, 2026), CNBC (Oct. 13, 2026), Anthropic Help Center: Does Anthropic crawl data from the web?, Anthropic: bots.json IP list. FraudMeNot is not affiliated with Anthropic. Product and company names are trademarks of their owners and are used here only to identify the bots. General information, not security advice.