AI Search Monitoring From Your Own Site, Not a Prompt List
- A browser analytics snippet never fires for an AI crawler, because crawlers do not run JavaScript. A server-side record does see them.
- Prompt-panel tools type the same prompts on a schedule and count brand mentions in the reply. That measures the assistant, not your site.
- In a 2026 SparkToro study run with Gumshoe.ai, 2,961 prompt runs returned the same brand list in the same order fewer than 1 time in 1,000.
- Kymo records which AI agents read which pages and which ones later sent a visitor who clicked. It does not track mentions.
- AI crawler tracking does not count against your event limit, on either plan.
Monitoring AI search from your own site means logging what AI engines did on your server: which page was fetched, by which agent, in which category, and whether a click followed. The other school of ai search monitoring asks the assistants on a schedule and counts how often your brand turns up in the reply. That measures the assistant. It never touches your server.
Both methods exist. One of them produces a record you can check.
Two ways to monitor AI search, and what each one sees
| Prompt-panel monitoring | Server-side monitoring | |
|---|---|---|
| What runs | A tool types fixed prompts into ChatGPT, Claude or Perplexity | A receiver on your site logs each request and classifies the user agent |
| What gets recorded | Whether your brand appeared in the answer, and where | Which URL was fetched, by which crawler, in which category, at what time |
| Sample size | However many prompts the vendor ships | Every request that reaches your site |
| Ties a read to a click or a sale | No | Yes, when the referral and the goal both land |
| Survives a model update | No, the answers shift underneath it | Yes, the log stays a log |
Neither method is a scam. They answer different questions. A prompt panel tells you what one assistant said on one day to a synthetic question nobody typed. A server log tells you which of your pages AI systems pulled, and whether anyone came back.
The prompt-panel numbers move on their own
Model output is a shaky measurement surface. Thinking Machines Lab ran the same prompt 1,000 times at temperature 0 in September 2025 and got 80 unique completions. The cause was server load changing the batch size, nothing to do with the prompt.
Models also drift in quality. Chen, Zaharia and Zou found GPT-4 scoring 84% on a prime-number task in March 2023 and 51% on the same task in June 2023. OpenAI rolled back a GPT-4o update on April 29, 2025 because the model had become overly flattering.
Stack the sampling problem on top. The SparkToro and Gumshoe.ai study had 600 volunteers run 12 prompts across ChatGPT, Claude, Google AI Overviews and AI Mode 2,961 times. Fewer than 1 in 100 runs returned the same list of brands. Fewer than 1 in 1,000 returned that list in the same order. The study discloses that Gumshoe sells AI tracking.
Percentage visibility held up better than position across runs. Position is the number most prompt panels put on a dashboard. Rand Fishkin, one of the study's authors, described tools that report a ranking position in AI as 'full of baloney'.
A browser script cannot see an AI crawler
Your analytics tag lives in the page. It fires when a browser loads that page and runs the JavaScript. AI crawlers do not do that. They fetch the HTML, read it and leave. No browser, no scripts, no tag.
A client-side snippet can tell you plenty about humans and exactly nothing about an AI fetch. That includes single-purpose AI trackers built on a browser script. They cannot see what never runs.
Kymo takes the other route. A server-side receiver on your site handles the request, reads the user agent and files the event. That works for a crawler in a way a page snippet never will.
What the server-side record actually holds
Kymo's receiver sorts AI agents into three categories.
| Category | What it means | Example token |
|---|---|---|
| ai_answer | A live fetch made because one person asked a question right now | Perplexity-User |
| indexing | A crawler building a search index for later answers | OAI-SearchBot |
| training | A bulk crawl collecting model training data | GPTBot |
The distinction matters when you decide what to block in robots.txt. GPTBot (OpenAI's training crawler) takes content for training and sends no traffic. OAI-SearchBot is a different agent on a different token and can send click-throughs. Blocking one does not remove you from the other. Providers with both a search agent and a live-fetch agent behave the same way, which is ClaudeBot vs Claude-SearchBot: What Each One Does on Your Site.
From there you can see which pages get fetched, which AI referral traffic arrives, and whether it converts. If you have ever wondered, Does ChatGPT Send Traffic to Your Site? (How to Check) walks through the referral side. The AI Traffic page shows, per page, how often AI engines read it, how many people AI answers sent there, and which of them completed a goal. Reading the AI report covers how to interpret it, and Kymo documentation covers setup.
One warning on the metric everyone still quotes: Bounce Rate for Small Business Sites: What's Normal is worth a read before you panic about AI referrals bouncing.
What mention tracking can tell you, and what it cannot
Mention tracking has a place. It is just not a measurement of your site. Pew Research Center found in July 2025, across 900 US adults, that users clicked a result in 8% of visits to a Google page with an AI summary against 15% without one. They clicked a link inside the summary in 1% of visits. Fewer clicks is the environment you are measuring in, whatever method you pick.
Citation sets differ by surface too. Ahrefs compared 540,000 query pairs in December 2025 and found Google AI Mode and AI Overviews cited the same URLs only 13.7% of the time, with 86% semantic similarity. And Digital Applied scored one brand on one dataset at 20% mention-based share of voice, 16.8% position-weighted and 31.4% citation-based. Three numbers, three stories, same brand, same data.
Kymo does not track mentions or what an assistant said. It records what AI engines did on your site: a read, a visit, a sale, per page. That is the whole claim.
What to do this month
Start with measurement, then decide. Run Kymo's free AI visibility checker on your own domain. It reads a URL from public signals, so the site does not need Kymo installed, and the report arrives by email.
Then set tracking up so the numbers keep coming. AI crawler events do not count against your event limit. Every feature is included on both plans: Solo at $9 a month or $90 a year for up to 10,000 events a month, and Studio at $29 a month or $290 a year for up to 100,000 events a month. Both start with 14 days free, no card required. Exceed the limit and the dashboard pauses, and your data stays put.
If you are new to this side of SEO, The Small Business Guide to AEO is a plain-language starting point. Publishing an llms.txt file is a cheap bet, though adoption is partial and no major operator has publicly committed to honouring it. Publishing the file is not the same as a crawler fetching it, and only your server sees which one happened.
Do the server-side monitoring first if you want evidence. Run the prompt panels too if you enjoy them, but treat the output as a mood ring, not a scoreboard.
Stop guessing. Watch.
Most AI visibility tools ask a chatbot and report a guess. Kymo watches your own site instead: which pages AI engines read, how many people their answers send you, and which of those people buy. Every number comes from your server, not from a prompt someone else chose.
Start free → 14-day free trial. No card required.