What Is AI Visibility? (And Why Most Scores Are a Guess)
- AI visibility covers two different things: what an AI engine says about your brand, and what an AI engine does on your site.
- Most tools measure the first one by typing prompts into assistants and counting brand mentions, so the score moves when the model changes.
- A January 2026 SparkToro study ran 2,961 prompt runs and found fewer than 1 in 100 returned the same list of brands, and fewer than 1 in 1,000 returned the same list in the same order.
- Recording crawler fetches on your own server gives you a number tied to your pages instead of somebody else's prompt set.
- Kymo records AI crawler reads, the referrals they send, and normal human analytics in one dashboard.
Search for what is AI visibility and you will find a dozen definitions, most of them describing a score. The plain version: AI visibility is how present your brand is inside AI answers. Where it gets messy is measurement, because two people using that phrase are often measuring different events.
Two different things both get called AI visibility
One is what an AI engine says about you. The other is what an AI engine does on your site. The first happens inside someone else's model. The second happens on your server, and you can log it.
| What an AI engine says | What an AI engine does | |
|---|---|---|
| Where the data lives | Inside the provider's system | Your server logs |
| How you get it | Type prompts, count brand mentions | Record each crawler fetch and referral |
| What moves the number | Model updates, prompt wording, random sampling | Your pages, your robots.txt, real user questions |
| Can it show a sale | No | Yes, when an AI referral converts |
| Repeatable | Rarely | Yes, same log, same numbers |
What an engine says about you is one sample of one moment
Tools that sell an AI visibility score are sampling. They write prompts, run them against ChatGPT, Claude, Perplexity or Google's AI surfaces, count which brands appear, and turn the count into a percentage. The work behind it is real. The problem is that the sample wobbles in ways the score cannot show you.
SparkToro and Gumshoe.ai ran 12 prompts across ChatGPT, Claude and Google AI Overviews and AI Mode 2,961 times in January 2026 with 600 volunteers. Fewer than 1 in 100 runs returned the same list of brands. Fewer than 1 in 1,000 returned the same list in the same order. The study discloses that Gumshoe sells AI tracking, so read it with that in mind. Visibility percentage held up better across runs than any ranking position did, which is why Rand Fishkin called tools that report an AI ranking position "full of baloney."
Model drift makes it worse. Thinking Machines Lab ran the same prompt 1,000 times at temperature 0 in September 2025 and got 80 unique completions. The cause was server load changing the batch size, nothing to do with the content. Chen, Zaharia and Zou found GPT-4 scoring 84% on a prime-number task in March 2023 and 51% on the same task that June. OpenAI rolled back a GPT-4o update on April 29, 2025 because the model had become overly flattering.
Your brand's AI visibility score can fall 20 points because an operator shipped a new checkpoint on a Tuesday.
What an engine does on your site can be recorded
Kymo samples nothing. A server-side receiver classifies crawler user agents and writes one row per fetch, tied to the page that was requested. Three categories. ai_answer is a live fetch made for one person's question, like ChatGPT-User (OpenAI's live fetcher). indexing is crawlers working ahead of time, like OAI-SearchBot (OpenAI's search index crawler) and PerplexityBot. training is bulk crawling, like ClaudeBot (Anthropic's training crawler).
Each row tells you which page was read, and whether a visit or a completed goal followed. That is the part a prompt-based score cannot produce, because the sale happens on your checkout page, not inside a chatbot.
The human side runs without cookies and without localStorage. The visitor identifier is a salted server-side hash that rotates at UTC midnight, so a unique visitor means unique per day. Raw IPs are never stored. There is a full explanation under cookieless identity. Crawler user agents are kept on the crawler's own event row so a classification stays auditable months later.
One limit worth stating: Kymo does not track mentions, and it does not record what an assistant said about you. It cannot tell you that ChatGPT called you the best in your category. Other tools try to, with all the sampling problems above. The trade is honesty about what a server can actually see.
The numbers your own site can produce
Pew Research Center tracked 900 US adults in July 2025. They clicked a result in 8% of visits to a Google page with an AI summary, against 15% without one. They clicked a link inside the summary in 1% of visits.
So AI traffic is small and specific. It arrives on particular pages. If those pages are not labelled, or if your setup funnels everything into direct, you will never see it. That is the setup covered in How to See AI Traffic in Your Analytics.
The surfaces disagree with each other too. Ahrefs looked at 540,000 query pairs in December 2025 and found Google AI Mode and AI Overviews cited the same URLs only 13.7% of the time, despite 86% semantic similarity. Two AI features on one search engine, pointing at different pages.
Same brand, same data, three different scores
Digital Applied scored one brand on one dataset in 2026 and got three answers depending on the method.
| Method | Score for the same brand |
|---|---|
| Mention-based share of voice | 20% |
| Position-weighted | 16.8% |
| Citation-based | 31.4% |
Same underlying data. The number is a choice, and most of the time nobody tells you which choice they made.
Reading any AI visibility claim without getting fooled
Ask what the number is built from. If the answer is "we prompt the assistants", you are looking at a sample. If the answer is "we read your logs", you are looking at a record. Only one of them changes when you change your site.
Then check the crawler assumptions. Blocking a training crawler does not remove your site from that operator's AI answers, because the indexing and live-fetch agents are separate robots.txt tokens. Every AI Crawler That Might Visit Your Site in 2026 lays them out. Perplexity-User is documented as exempt from robots.txt because a user asked for the fetch, and Cloudflare published evidence in August 2025 of undeclared Perplexity crawlers, a finding Perplexity disputed.
On llms.txt, keep expectations low. Publishing one is a cheap bet. Adoption is partial. No major operator has publicly committed to honouring llms.txt, and neither OpenAI, Anthropic nor Google documents it as a signal their crawlers follow. Several documentation platforms and developer tools do read it. Writing the file is not the same as a crawler fetching it, and only your own server sees which happened. Steps are in How to add an llms.txt file to Shopify.
If you want the longer measurement picture, there is a whole category on Measuring AI visibility.
What to do first
Fix the measurement before you write anything new. A small team with no analyst cannot act on a percentage it cannot reproduce. Kymo has two plans. Solo is $9 a month, or $90 a year, for up to 10,000 events a month. Studio is $29 a month, or $290 a year, for up to 100,000 events a month. Every feature is on both plans, and they differ only by event volume. AI crawler tracking does not count against your event limit, so a heavy month of GPTBot activity will not eat your quota. Exceed the limit and the dashboard pauses. Your data stays put. Both plans start with 14 days free, no card required.
You can see the shape of it on the demo dashboard with no login, or run a free AI visibility check that reads a URL from public signals. Your site does not need Kymo installed, and the report arrives by email. For the content side, work through The Small Business Guide to AEO.
Track first if you are a solo builder who has never seen a single AI crawler hit on your own pages.
Stop guessing. Watch.
Most AI visibility tools ask a chatbot and report a guess. Kymo watches your own site instead: which pages AI engines read, how many people their answers send you, and which of those people buy. Every number comes from your server, not from a prompt someone else chose.
Start free → 14-day free trial. No card required.