PerplexityBot: What It Crawls and Why It Matters

PerplexityBot is the crawler Perplexity uses to read your site, but how it behaves depends on which of Perplexity's two agents showed up. This matters because one of them can send you a customer and the other only reads.

Two crawlers, not one

Perplexity runs distinct user agents for distinct jobs. In your logs they look like relatives, and they do different work.

A third name, PerplexityBot-User, appears in some published crawler lists. It is not a real agent. The current token is Perplexity-User.

Kymo tracks both separately. In your dashboard each one shows up under its own classification, so you can tell an index crawl from a live answer fetch without guessing.

What PerplexityBot actually collects

Neither agent collects training data, because Perplexity does not train foundation models. Both are retrieval: one builds the index, the other reads a page on demand. So the usual "should I let the trainers in" question does not really apply to Perplexity. The question is whether you want to be citable.

Compliance is the real caveat. Perplexity documents that Perplexity-User generally ignores robots.txt, on the grounds that a person requested the fetch. And in August 2025 Cloudflare published evidence that Perplexity used undeclared crawlers to reach pages that had disallowed it. Perplexity disputed the finding. Either way, if you need a hard block, enforce it at your edge or firewall rather than in robots.txt.

The bot does not execute heavy JavaScript well. If your content requires client-side rendering to appear, PerplexityBot may see an empty page. That is a common reason why pages get fewer AI mentions than they should.

Live answers vs index crawls

The Perplexity-User agent is the one that matters most for your traffic. When someone asks Perplexity a question, it runs a live fetch of the pages it thinks are relevant. That fetch reads your content in real time, then Perplexity answers the user with a citation.

If your page is included, the user sees a numbered source link that points straight to your site. Clicking it sends a referral visit. This is the closest thing to "AI traffic" that actually shows up in your analytics as real people arriving at your URL.

The index crawl is different. PerplexityBot reads your pages so they are available to cite later. No user is waiting on that fetch, and no referral comes back from the crawl itself. It is a necessary step rather than a result: a page that is never indexed is never cited, and a page that is indexed may still never be chosen.

That distinction is why you need a tool that separates the two. Raw analytics tools lump all crawlers together and call it bot traffic, which tells you nothing about whether AI is driving real visits. Reading the AI report shows you exactly which pages get fetched for live answers versus which ones only get bulk crawled.

Why Perplexity matters for your traffic

Perplexity has become a real referral source for many small sites. Unlike a Google featured snippet, where you might get a click without a clear attribution, Perplexity shows visible citations that users can click before or after reading the answer.

Those clicks are warm leads. Someone has already read a summary that cites you. They are clicking through because they want more detail. That is higher intent than a random search result.

The flip side is that Perplexity answers often satisfy the user without any click at all. If the answer is complete, the citation goes unvisited. That is the reality of AI discovery, and it is why you need to optimize for being cited in the first place.

How to read it: perplexitybot in your logs

If you check your server logs, you will see hits from PerplexityBot with different paths. The user agent string is your first clue.

A PerplexityBot user agent is an index crawl. A Perplexity-User agent is a live answer fetch. The distinction is in the user agent, and Kymo reads it server-side to classify each hit correctly. Perplexity also publishes the IP ranges for each agent, so you can verify a claimed Perplexity hit instead of trusting a string anyone can copy.

You can see these hits in your Kymo dashboard alongside your human traffic. No more digging through raw logs trying to decode user agent strings. The Kymo documentation explains exactly how the classification works and what each category means.

Perplexity referrals are more valuable than they look

A Perplexity referral visit behaves differently from a Google visit. The user already saw your content summarized. They are coming to you with context. Session duration tends to be longer, and bounce rate tends to be lower, because the page matches what they expected to find.

That makes Perplexity referrals worth tracking as their own segment. If you see a steady trickle of Perplexity-User fetches on a specific page, that page has become an authority source for whatever topic it covers. You should keep it updated and consider writing more content in the same area.

This is core to answer engine optimization. The question is not whether you can rank on Perplexity, but whether your pages get picked up as cited sources at all. The Small Business Guide to AEO walks through the mechanics of making your content more citable for AI answers.

The limits of PerplexityBot data

PerplexityBot does not crawl everything. It is selective. Pages with thin content get skipped. Pages buried under heavy JavaScript get skipped. Pages with slow response times get skipped.

You can check whether Perplexity can see your site using the free AI visibility check. It reads a URL from public signals only, so your site does not need Kymo installed. The report arrives by a magic link over email. Use it to see whether PerplexityBot can even parse your content.

If the checker finds nothing, your content is likely invisible to AI crawlers. That is fixable. Static HTML, clear headings, and direct answers to common questions all help.

The bandwidth vs citations tradeoff

You can set separate rules for each agent, because each has its own robots.txt token. Perplexity documents both, so PerplexityBot and Perplexity-User can be treated differently in the same file.

Blocking all of Perplexity is a legitimate choice if the crawler load costs you real money. The cost is that Perplexity stops citing you, and citations are one of the more reliable AI referral sources available today. You decide which side of that matters more to you.

Most small businesses keep both agents allowed and simply watch the numbers. Kymo does not make the decision for you. It gives you the data to make it yourself.

What to do next

If you want to know whether Perplexity sees your site at all, run the free AI visibility check. It only needs a URL and an email address.

Then look at your actual traffic data. If you already have Kymo installed, check the AI Visibility report for Perplexity hits. You will see which pages get fetched for live answers, and how many confirmed click-throughs came from Perplexity referrals. That tells you what is working and what is invisible.

If you are still relying on Google Analytics, keep in mind it does not classify AI crawlers this way. It shows PerplexityBot as one generic bot entry, which leaves you blind to the difference between an index crawl and a user waiting on an answer. Privacy-first analytics handles this without cookies or personal data storage. Privacy-First Analytics: What It Actually Means explains the difference.

Perplexity is not one bot. Treating it as one hides the part that matters, which is whether live answer fetches are turning into visits. The analysis on ChatGPT traffic covers the same distinction for OpenAI's crawlers, and the pattern holds across all AI platforms.

Track it or guess

You can keep guessing whether Perplexity sends you traffic, or you can measure it. Kymo classifies PerplexityBot into its three categories and shows you which pages AI assistants fetch, the likely prompts behind those fetches, and the click-throughs that actually arrive.

The free plan lets you see this clearly. Claude-SearchBot (Anthropic's search index crawler) behaves similarly, and the same dashboard tracks them all.

If you run a content site and want to know whether AI answers send you visitors, start tracking it to see which of these crawlers actually hit your site and which ones send people back. If you have not installed anything yet, the free AI visibility check is the fastest way to find out whether Perplexity can even see your content. Read what AEO means in plain English if you need the bigger picture first.

See the data yourself

Start with the free AI visibility check to see if Perplexity can read your site. Then install Kymo to get the full classification of every crawler and every AI referral. Both take minutes.

Start free → 14-day free trial. No card required.