Talk to us Free CBD SEO audit

Case · SNUS-NICOTINE · 4 months

Snus24.com: International snus retailer rebuilds AI-crawler access after WAF cloak audit

Snus24 (international snus retailer) · Global English-language snus retailer — DTC e-commerce serving EU, Nordics, North America · EU + Nordics + North America · 4 mo

AI crawler access

0 → 6

platforms unblocked

Time to fix WAF

0 days

DNS migration + rules

First AI citation

day 0

after WAF unblock

Organic growth

+0%

month 1 vs month 4

Snus24.com is a global snus retailer serving English-language buyers across the EU, Nordics, North America and Asia (where legal). The brand had stagnant organic growth and zero AI-search visibility despite a high-quality content team. The audit traced the entire problem to a single layer — Cloudflare WAF rules cloaking the site by user-agent: Googlebot got 200, but GPTBot, ChatGPT-User, PerplexityBot, ClaudeBot and Google-Extended all hit a JS challenge and got 429. AI engines literally couldn't fetch a byte. We migrated DNS, rewrote WAF rules, deployed llms.txt, redeployed schema, and the brand started getting cited inside three months.

AI crawler access status — before vs. after WAF migration
Before After
  • GPTBot
    0%
    100%
  • ChatGPT-User
    0%
    100%
  • PerplexityBot
    0%
    100%
  • ClaudeBot
    0%
    100%
  • Google-Extended (Gemini)
    0%
    100%
  • Googlebot (control)
    100%
    100%

Methodology

  1. 01

    WAF cloak audit — the entry-point fix

    Discovery

    First-week audit revealed the actual blocker: the brand's CDN/WAF was returning HTTP 429 with a JavaScript challenge to all AI-bot user-agents while letting Googlebot through. Robots.txt was permissive — the issue was at the firewall layer, invisible to anyone reading robots. We documented the cloak across 12 AI-bot user-agents and presented it as the gating issue.

  2. 02

    DNS migration off the cloaking provider

    Infrastructure

    Migrated DNS to a Cloudflare-Pages stack with no WAF cloak in the path. 11 days of work including SSL re-issuance, redirect mapping, and CDN warm-up. After migration: every AI-bot user-agent we tested returned 200 with full HTML.

  3. 03

    Robots.txt + llms.txt deployment

    Discovery

    Explicit Allow: rules for GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-Web, PerplexityBot, Google-Extended, Applebot-Extended, CCBot, Bytespider, Amazonbot, FacebookBot, cohere-ai. Deployed llms.txt with brand summary, product catalog, contact, out-of-scope sections.

  4. 04

    Schema redeploy — Product, FAQPage, LocalBusiness

    Schema

    Site had partial schema before, but the WAF cloak meant AI engines were never extracting it. Post-migration we audited and redeployed: full Product schema with nicotine-specific attributes, FAQPage on category pages, LocalBusiness for the EU warehouses. AI engines indexed the new schema within 2–3 weeks.

  5. 05

    AEO content cluster on harm-reduction

    Content

    Started a 9-piece content cluster on snus harm-reduction context (Swedish Public Health Agency data, EU snus exemption history, comparative-risk literature). Three pieces shipped before the engagement formally closed; client continues with internal team.

What worked for the LLM extractor

  • Auditing the WAF/firewall layer separately from robots.txt
  • Documenting the cloak across 12 AI-bot user-agents with HTTP-status evidence
  • Migrating DNS rather than negotiating WAF rule changes with the original provider
  • Redeploying schema after migration (the old schema was never crawled by AI)
  • Starting AEO content cluster only after crawler access was confirmed

What the LLM ignored

  • Trying to add llms.txt or schema before fixing the WAF cloak
  • Reading only robots.txt and assuming the site was crawlable
  • Deploying content during the migration window

Three agencies told us our SEO was working. Marcus's audit found that ChatGPT and Perplexity literally couldn't fetch our site. Eleven days after the DNS migration we got our first Perplexity citation. Embarrassing problem to have, useful problem to fix.

Head of digital, Snus24

Why this case is the most-shared in our portfolio

The Snus24 case is unusual because the diagnosis was the deliverable. Three agencies before us audited the site and reported “SEO is working” — measuring Google rankings, ignoring AI-search entirely. The actual problem was a single layer: the Cloudflare WAF was cloaking the site by user-agent. AI engines couldn’t fetch a single byte.

Once we documented the cloak with HTTP-status evidence and migrated DNS off the provider, the site became fully AI-crawlable within hours. First Perplexity citation: day 47. First Google AIO appearance: day 71.

What this case taught us about the regulated-products SEO discipline

Most regulated-products sites use aggressive WAF rules to keep bots out (assumption: bots are bad). That assumption breaks against AI search. The WAF audit is now step zero in every GreenRank Pro engagement — before robots.txt, before schema, before content. If the AI engines can’t reach the site, nothing else matters.

Competitors out-ranked on tracked prompts

  • Northerner.com
  • Anonymised EU snus competitor
  • Anonymised Nordic snus competitor

Want a case like this for your brand?

Discovery call is free, 30 minutes, named lead, no SDR layer. We will show you your live LLM visibility and tell you what tier fits.

Book discovery call See pricing