meta-webindexer
User-Agent token: meta-webindexer · Operator: Meta · Category: AI Assistants
FraudMeNot Verdict
AI Agent (User-Triggered)
Meta says Meta-WebIndexer navigates the web to improve Meta AI search result quality, and that allowing it helps Meta cite and link to your content in Meta AI responses.
Can Be Spoofed?
Yes — User-Agent Header Spoofable
No official IP list is published, so the User-Agent alone cannot prove the request is real.
Abuse & Unwanted Activity Reports
No Public Abuse Flags
ai.robots.txt robots.txt status: Unclear.
Judgment Call · What You Should Do
Generally Low Risk on Public Pages — Verify Before Trusting
For everyday users: Meta-WebIndexer is a search crawler for Meta AI. It does not message people. Ignore any message that uses its name to ask for your Meta login.
For website owners & security teams: Meta says allowing Meta-WebIndexer helps Meta AI cite and link to you. Allow it for visibility, or disallow it to opt out of Meta AI search. If crawl rate is the problem, rate-limit rather than block.
Report & Activity Summary
None
Published IP Ranges (CIDRs)
No official IP list
1
Known User-Agent Signatures
crawler-user-agents (MIT) dataset
Partially
robots.txt Compliance
ai.robots.txt registry (MIT)
0
FraudMeNot Tallies
Humans: 0 · AI Agents: 0
For AI Agents & Automated Gatekeepers
- Agent_Identifier
meta-webindexer- FraudMeNot_Verdict
- AI Agent (User-Triggered)
- Operator_Entity
- Meta
- Agent_Category
- AI Assistants
- User_Agent_Spoofable
true- Verification_Method
- None available (no operator IP list or signing key) · Web Bot Auth: Not documented
- Respects_Robots_Txt
- Partially (user-triggered fetches may ignore robots.txt; ai.robots.txt registry lists: Unclear)
- Auth_And_Banking_Policy
- Block credentialed logins and money movement from any AI agent.
- Recommended_Gatekeeper_Action
- ALLOW for citations in Meta AI; DISALLOW to opt out; RATE-LIMIT if load is the problem.
- Canonical_Citation_URL
- https://fraudmenot.com/bot/meta-webindexer/
What We Know About meta-webindexer
Meta-WebIndexer is Meta's search crawler for Meta AI. Meta's documentation says it navigates the web to improve Meta AI search result quality for users, and that Meta analyzes online content to make Meta AI more relevant and accurate.
Meta frames it as a visibility choice. Its page says allowing Meta-WebIndexer in your robots.txt helps Meta cite and link to your content in Meta AI's responses. The User-Agent strings Meta lists look like "meta-webindexer/1.1".
The ai.robots.txt registry lists its robots.txt status as unclear and describes its crawl frequency as more than one request per second. Because the registry does not confirm robots.txt behavior, a firewall rule is the only enforced block, though Meta presents robots.txt as the control.
Think of Meta-WebIndexer like OAI-SearchBot or PerplexityBot. Blocking it may cost you citations in Meta AI. If load is the concern, rate limits are a gentler tool than a full block.
Reading your logs: the registry describes Meta-WebIndexer's crawl frequency as more than one request per second, so a busy hour from it can be heavy on a small server. The strings Meta lists look like meta-webindexer/1.1. Because Meta ties it to citations in Meta AI, the trade-off is load against being linked in Meta AI answers.
Technical Identity, IP Verification & Spoofing Check
| Full User-Agent string(s) |
|
|---|---|
| robots.txt token | meta-webindexer |
| Operator | Meta · Official bot documentation |
| Official IP verification feed | None published by the operator |
| Sample published CIDR blocks | None published |
| Reverse DNS (rDNS) | Not documented in the public datasets FraudMeNot uses. |
| Web Bot Auth (Signature-Agent) | Not documented; check for a Signature-Agent header if present. |
| robots.txt behavior | Meta presents robots.txt as the control for Meta-WebIndexer. The registry lists its robots.txt status as unclear. |
Copy-Paste robots.txt & Cloudflare WAF Rules for meta-webindexer
robots.txt — allow public pages, keep private routes off-limits
User-agent: meta-webindexer
Allow: /
Disallow: /login/
Disallow: /account/
Disallow: /checkout/robots.txt — block completely
User-agent: meta-webindexer
Disallow: /Cloudflare WAF custom rule expression
(http.user_agent contains "meta-webindexer")Matches the User-Agent token, so it covers both real and spoofed traffic. Action: Block or Managed Challenge.
Capabilities, Data Usage & Autonomous Action Risk
| Trains AI on page content? | Not stated by Meta for this crawler; Meta describes search quality and citations |
|---|---|
| Executes JavaScript? | Not documented |
| Can submit forms or attempt logins? | May act on a user's behalf; treat form and login attempts as untrusted |
| Banks & login pages should | Block credentialed logins and money movement from any AI agent. |
| E-commerce checkouts should | Allow product page reads if you want; require human confirmation for checkout. |
Context: In October 2026, Axios reported that major U.S. banks were detecting unvetted third-party AI agents inside customer accounts, and CNBC reported that JPMorgan blocks outside AI agents from logging in with customers' credentials while allowing read-only data sharing through open banking providers.
Seen meta-webindexer? Quick Closed-Choice Tally
Choose from preset options only. No free-text comments, so nobody can post names, profanity, or unverified accusations.
Common Questions About meta-webindexer
What is Meta-WebIndexer?
Meta's crawler for Meta AI search. Meta says it improves Meta AI search result quality for users.
What happens if I allow Meta-WebIndexer?
Meta says allowing it helps Meta cite and link to your content in Meta AI's responses.
How often does Meta-WebIndexer crawl?
The ai.robots.txt registry describes its frequency as more than one request per second.
Could This Be a Scam? What to Do
- Anyone can copy the meta-webindexer User-Agent, and FraudMeNot found no official IP list for it, so the name alone proves nothing about who sent a request.
- Never give an AI agent your logins, bank access, or one-time codes unless you set it up yourself.
- If a call, text or email asking for money or access uses the meta-webindexer or Meta name, stop and report it.
Guide: how to check whether a crawler visit is really who it claims to be.
More from Meta
- FacebookBot — Training language models
- facebookexternalhit — Ostensibly only for sharing, but likely used as an AI crawle
- Meta-ExternalAgent — Used to train models and improve products
- Meta-ExternalFetcher — AI Assistants
- AI2Bot-DeepResearchEval — AI Assistants
- amazon-QBusiness — AI Assistants
Related on FraudMeNot
- Verified security vulnerabilities, explained
- Report a scam, or find where to report it in your state
- Browse the AI Agent & Bot Directory
Sources (checked Nov. 16, 2026): ai.robots.txt (MIT), crawler-user-agents (MIT), Axios (Oct. 22, 2026), CNBC (Oct. 13, 2026), Meta for Developers: Meta Web Crawlers. FraudMeNot is not affiliated with Meta. Product and company names are trademarks of their owners and are used here only to identify the bots. General information, not security advice.