How to get Meta AI to recommend your brand

Meta AI recommends brands it can cite. It is the assistant inside WhatsApp, Instagram, Facebook, Messenger, and meta.ai, and it does not read from a single index: Meta routes queries between backends by topic, combining Bing results (a partnership dating to 2023), Google results (added April 2024), and its own crawler, meta-webindexer, which Meta says improves Meta AI search results and which Meta says helps it cite and link your content when you allow it. If that crawler is blocked, or your pages only exist in one backend's index, Meta AI cites a competitor it can reach instead.

A product note before anything else: Meta AI is not Muse. Meta AI is the assistant inside Meta's apps; Muse is Meta's separate personal-agent product. Both use the Muse Spark model family, but they are different products and this guide is about Meta AI only.

Why Meta AI skips your brand

Meta AI's failure modes are its own:

1. meta-webindexer is the one allowlist that directly controls citations, and most sites block it without knowing. Meta documents this crawler as the one that improves Meta AI search results and says allowing it helps Meta AI cite and link to your content. Generic "block AI bots" rules catch it along with the training crawlers.
2. Your pages are in one index but not the other. Because Meta AI routes between the Bing and Google supplier paths by query topic, a site indexed only by Google can be invisible to Meta AI on the queries that route to Bing. No other engine splits its retrieval this way.
3. JavaScript shells are invisible to Meta's own crawlers. Meta's crawlers do not execute JavaScript, so a product page that renders prices and specs client-side arrives as an empty shell to them. The saving detail: rendered content can still surface through the Google and Bing supplier paths, since both are fully rendered indexes. JavaScript matters less for Meta AI than for Claude, but the readable-HTML rule still applies.
4. Each surface may not read the same index. Meta has not said whether WhatsApp, Instagram, and Facebook all pull from the same retrieval mix, so a brand recommended on one surface and skipped on another is a real, unverified possibility, not a debugging error.

Step 1: Allow meta-webindexer, by name

Check your robots.txt at [yourdomain.com]/robots.txt, replacing yourdomain.com with your actual domain. If nothing loads, create the file at the root of your site. Then add these rules near the top, before any broad disallow rules:

User-agent: meta-externalagent/1.1
Allow: /

User-agent: meta-webindexer/1.1
Allow: /

User-agent: meta-externalads/1.1
Allow: /

Know what each one does. meta-webindexer is the citation crawler: it feeds Meta AI search results, and blocking it is the single most common reason Meta AI cannot cite you. meta-externalagent feeds training and product improvement; meta-externalads feeds advertising products. The full user-agent form Meta documents looks like meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler), but matching on the short token is enough.

Two tokens you do not put on this list: meta-externalfetcher/1.1, which fetches individual links when a user requests them including for AI agents, and facebookexternalhit/1.1, which fetches link previews for shared URLs. Meta documents that both may bypass robots.txt, so allow rules for them are not reliably enforced; they do their own jobs either way. (These crawler details follow Meta's developer documentation for web crawlers, as consistently transcribed by third parties; our direct fetch of the page failed, so if Meta revises the docs, this page follows.)

Step 2: Check the edge, not just the file

The robots.txt file says allow, but the edge can still be the blocker: Cloudflare Bot Fight Mode and WAF bot rules challenge or block Meta's crawlers even when robots.txt permits them. The file is what you declare; the edge is what enforces. In Cloudflare, open Security > Bots and confirm AI crawlers are not being challenged or blocked. Robots.txt changes take up to 24 hours to take effect, so wait before re-testing. Then verify in your own logs:

grep -i "meta-external" /var/log/nginx/access.log

Watch for meta-webindexer specifically: it is the traffic that feeds the AI-search path. No hits after several days means something upstream is still blocking, regardless of what the file says.

Step 3: Cover Bing and Google, not just one

Since Meta AI routes between the Bing and Google supplier paths by query topic, you need to be indexed in both. Register the site in Bing Webmaster Tools and confirm your key pages are indexed there, and do the same in Google Search Console. Brands that only watch Google Search Console miss half of a retrieval mix that no other engine duplicates in quite this way.

Step 4: Verify the whole path, surface by surface

Meta publishes no crawler IP ranges and no webmaster tool for its AI crawlers, so verification takes two passes:

1. Authenticate crawler traffic. Log tokens can be spoofed, so run the requesting IPs through whois and confirm they fall on Meta's network block, AS32934. A token match without the whois check proves nothing.
2. Test each surface. For link previews on Facebook and Instagram, use Meta's Sharing Debugger. Then ask Meta AI a branded query inside WhatsApp and inside Instagram and check whether your brand is cited on both. A brand cited on one surface and skipped on another is the multi-backend retrieval at work, not a failure of your allow rules.

What this guide does not cover

Getting cited gets you into Meta AI's answers. The retrieval mix changes: Meta is openly building its own index alongside the Bing and Google integrations, and your catalog drifts, which is why monitoring exists. And answers are not orders: letting the agent complete the purchase is separate work. Muse, Meta's personal agent, is a different product with no allowlist lever at all; it gets its own guide.

For the ChatGPT version of this playbook, see our guide to getting ChatGPT to recommend your brand.