There is no crawler to allowlist. Muse is not a search engine you get indexed into; it is a personal agent that shops for the user, and it browses your site with its own browser, on a Muse Secure VM, the way a human would. It recommends brands whose pages render in a real browser, whose price and stock it can verify in structured data, and whose checkout it can actually complete. Everything about robots.txt allowlisting in the other guides in this series does not apply here.
One clarification: Muse is a separate product from Meta AI, not a rename. It is Meta's personal AI agent, launched September 8, 2026, and Meta AI gets its own guide. Meta documents one adjacent fetcher token, meta-externalfetcher/1.1, for user-initiated fetches "including for AI agents," which may bypass robots.txt, but Meta has not said which path Muse uses for a given fetch, so there is nothing to build on there. Meta's launch materials describe a second agent, Sentinel, monitoring the VM and gating sensitive actions, and purchases running through one-time virtual cards via Stripe Link. That is context, not something you optimize for.
Why Muse skips your brand
Muse's failure modes are its own, and none of them involve a crawler:
- Bot protection fires on it. Muse presents a standard browser user-agent, so a bot challenge that fires on real browsers fires on Muse too. There is no allowlist token to add, and blocking it would mean blocking real shoppers, so the fix lives in your edge rules.
- The checkout assumes a logged-in human. Account-required checkout, shipping options hidden until sign-in, sessions that depend on earlier visits: all of these stall when Muse arrives in a fresh browser session. It can recommend you and still abandon the cart.
- There is nothing to verify against. Muse reads the page like a human, but it cross-checks what it reads against your structured data. Prices baked into images and pages with no product schema give it nothing to confirm, and unconfirmed facts lose to brands it can verify.
- The page breaks outside your test environment. JS-driven widgets and checkout flows that fail in a clean session fail for Muse the same way.
Step 1: Test through Muse itself
There are no webmaster tools and no log tokens for Muse, so the test is the verification. Ask Muse to research your product and work toward a purchase, then watch where it stalls: the search step, the price check, or the checkout. Your server logs will only show normal browser traffic, so skip the log grep entirely.
Step 2: Make the site survive a real browser
Open [yourdomain.com] in a private window, replacing yourdomain.com with your actual domain, logged out, and complete the path a shopper takes. The checkout must finish without a login, price and stock must be text rather than images, and any bot protection or challenge must not fire on an ordinary browser session. If it breaks for you there, it breaks for Muse.
Step 3: Give it schema to verify against
Muse cross-checks the page against your structured data, so the schema has to match the visible page exactly: price, availability, SKU, no conflicts between the two. A mismatch between what the page shows and what the schema claims is a trust failure, and the agent picks the brand it can trust.
Step 4: Walk the full transaction path
Product to cart to checkout to confirmation, as a guest, in a fresh browser session, and confirm the order actually goes through. This is the step the other engine guides never ask for, and for Muse it is the one that matters most: recommendation without transactability is a dead end.
A hypothetical, to make it concrete
Say you run a direct-to-consumer luggage brand. The site looks perfect, the product pages rank well, and the checkout is a polished single-page app. But shipping options only appear after sign-in, some prices live inside lifestyle images, and there is no product schema on the page. A scan would flag the pattern in one pass: nothing here is a crawl problem, so no allowlist can fix it. The fix is the transaction path: guest checkout that completes in a fresh browser session, prices as text with matching schema, and edge rules that treat Muse's browser like any shopper's.
The fundamentals
These mechanics are shared across every engine and covered in full in their own guides:
- •llms.txt: the file format, a complete sample, and per-platform publish steps, see our guide to writing a good llms.txt file
- •Product schema: the JSON-LD markup Muse reads to verify what the page claims, see our product schema guide for AI search
- •The robots.txt pillar matters less here than for any other engine: there is no Muse crawler token to allowlist. Check your edge rules instead, since anything that blocks real browsers blocks Muse, see our robots.txt guide for AI search
What this guide does not cover
Getting Muse to recommend you is only half its job, because Muse does not stop at recommending. It goes on to buy, which makes this guide the closest in the series to the transaction layer: keeping catalog data accurate as it changes, handling variants and pricing reliably, and making the purchase path work at scale is ASO work, and this series does not cover it. It also does not cover monitoring and drift: your site changes, the agent changes, and neither waits for you.
Frequently Asked Questions
How does Muse discover ecommerce brands?
Does Muse enforce strict schema validation?
What is the best way to verify if Muse can crawl my site?
Check your own brand now
Run the free scan. It takes seconds, no signup, and shows exactly where Muse stands with your site: aizii.ai/aeo-scan
Run the free scan