Skip to content
Measure

Free Ways to Check If AI Search Sees Your Site

TL;DR
  • Run a site: search on Google to see what a crawler can find.
  • Ask ChatGPT or Claude directly if they can see your site. Their live fetchers will tell you.
  • Check your robots.txt file to see which AI crawlers you are blocking, or allowing by default.
  • Look at your server logs or a free tool like the Kymo AI visibility check for a structured report.
  • The free visibility check at Kymo reads a URL from public signals and emails you the report. No account needed.

The simplest answer is to ask an AI assistant directly whether it can see your site. If it can fetch your page, you are visible. If it cannot, you have a fixable problem.

You spent years getting discovered through Google. Now a growing share of discovery happens inside ChatGPT, Perplexity, and Claude. You cannot improve what you do not measure, so this guide walks through four genuinely free ways to run an ai visibility audit free. Each one costs nothing and takes under five minutes.

Run a site: search to see what Google has indexed

Google still matters. It also feeds a lot of AI answer engines. A quick site:yourdomain.com search shows you what Googlebot has crawled and stored.

Do that now. If your important pages appear, your indexing baseline is healthy. If not, start with Google Search Console and submit your sitemap.

This method is free and instant, but it only shows Google's view. It does not tell you what ChatGPT or Perplexity can see. That is the gap you need to close.

Ask an AI assistant directly if it can see your site

Open ChatGPT, Claude, or Perplexity and type a prompt like this: "Can you access [your URL]? What does the page contain?"

  • ChatGPT uses its live fetcher, ChatGPT-User (OpenAI's live fetcher), to pull the page on demand.
  • Claude does the same with Claude-User.
  • Perplexity uses Perplexity-User for live fetches.

If the assistant summarizes your content accurately, your site is fetchable. If it says it cannot access the page, something is blocking it. The usual culprits are robots.txt directives, Cloudflare bot rules, or a misconfigured server that returns an error to those user agents.

One thing to keep in mind: Perplexity-User is documented as exempt from robots.txt because a user asked for the fetch. Cloudflare published evidence in August 2025 of undeclared Perplexity crawlers, which Perplexity disputed. That is a disputed finding, so treat it as context, not settled fact.

This direct test is the quickest real-world check. It shows you exactly what a person using that assistant would experience.

Check your robots.txt file for AI crawler rules

Your robots.txt file, usually at yourdomain.com/robots.txt, tells crawlers what they may and may not access. Many AI companies run three separate agents: one for training, one for indexing, and one for live answer fetches.

Here are the key entries you need to know:

CrawlerOperatorCategoryRobots.txt tokenSends traffic?
GPTBotOpenAITrainingGPTBotNo
OAI-SearchBotOpenAIIndexingOAI-SearchBotYes
ChatGPT-UserOpenAILive fetchChatGPT-UserYes
ClaudeBotAnthropicTrainingClaudeBotNo
Claude-SearchBotAnthropicIndexingClaude-SearchBotYes
Claude-UserAnthropicLive fetchClaude-UserYes
PerplexityBotPerplexityIndexingPerplexityBotYes
Perplexity-UserPerplexityLive fetchPerplexity-UserYes
BytespiderByteDanceTrainingBytespiderNo

If your robots.txt blocks a training crawler like GPTBot, your content is not removed from AI answers. The indexing and live-fetch agents use separate tokens. Blocking GPTBot only stops bulk training data collection.

Google-Extended is a different animal. It is not a crawler at all. It is a robots.txt control token with no user agent, and it never appears in a server log. Disallowing it tells Google how content it already crawled may be used. It does not block a visit. If you are deciding whether to block it, understand that first.

Some operators run agents you may not have heard of. Bytespider (ByteDance) is a training crawler that does not send traffic. ClaudeBot and Claude-SearchBot do different jobs, and mixing them up leads to bad robots.txt decisions. ClaudeBot vs Claude-SearchBot: What Each One Does on Your Site explains the split in detail.

The default stance for most small sites is to allow everyone. If you block the wrong token, you can accidentally cut off indexing or live fetch access. If you use Cloudflare or another proxy, check whether its bot rules are filtering AI user agents before they reach your server.

Look at your server logs or use the free Kymo visibility check

Server logs contain every request your site receives, including those from AI crawlers. If you have access to your logs, search for user agents like GPTBot, ClaudeBot, or OAI-SearchBot. The problem is that logs are noisy and hard to read, especially if you are not technical.

The free visibility check at Kymo does this for you. It reads a URL from public signals only, so the site does not need Kymo installed. You submit your URL, and the report arrives by a magic link over email. No account, no card, nothing to install.

The report tells you which AI agents can see your page, which ones are blocked, and how your robots.txt rules look. It is a clean starting point for an ai visibility audit free of guesswork.

What to do with the results

Once you know whether AI search can see your site, act on the gaps.

If your page is blocked by robots.txt, decide whether that block is intentional. If it is accidental, fix it. If you want to be found, allow the indexing and live-fetch agents while still blocking training crawlers if that matters to you.

If your page is fetchable but not getting referenced, the issue is content structure. AI answers favor clear headings, direct answers, and self-contained explanations. The Small Business Guide to AEO walks through how to structure content so assistants can cite it cleanly.

If your content is solid, track the trend over time. A one-time check tells you where you stand today. It does not tell you which AI engines are crawling you weekly, which pages they prioritize, or whether their referral traffic is growing. That requires continuous measurement.

This matters because the AI ecosystem changes quickly. A crawler that ignores you today may index you tomorrow. PerplexityBot and ChatGPT-User operate differently, and understanding those differences helps you set the right rules. You can check the PerplexityBot registry page for details on its behavior. For ongoing tracking, Reading the AI report explains what the data shows and how to act on it. For context on why traditional analytics miss this entirely, Why GA4 Can't Show You AI Citations is worth a read. And if you have ever found GA4 itself painful, Why Is GA4 So Complicated? (And What Small Sites Actually Need) explains what you actually need to track.

Free tools have limits, and that is fine

The free methods in this guide give you a snapshot. They answer one question: can AI search see your site right now? That is valuable, and it is the right place to start.

A snapshot does not show trends, click-throughs from AI referrals, or which questions triggered the fetch. Those require continuous tracking. The good news is that you can run the free check today to establish a baseline, then decide whether you need ongoing visibility.

Kymo's paid plans start at $9 a month for Solo, up to 10,000 events, or $90 a year. Studio is $29 a month, or $290 a year, up to 100,000 events. Both plans include every feature, and the plans differ only by event volume. Both start with 14 days free, no card required, and AI crawler tracking does not count against your event limit. For the broader small business context, the AI search for small business archive has more on how this shift affects discovery.

The recommendation

Run the free visibility check on your homepage and your two most important service pages today. If you are a solo builder or run a small site, start with the free check to find immediate blocks, then decide if you need continuous tracking.

If AI referrals are becoming a meaningful part of where your customers come from, upgrade to continuous tracking. If you just want to know whether you have a problem, the free check answers that in minutes.

Get a clear answer on your site's AI visibility

Run the free check on your own site to see which AI agents can access it, or start tracking it continuously to see which AI engines already crawl you. The free check reads a URL from public signals, so your site does not need Kymo installed, and the report arrives by email. You will know exactly what is visible and what is blocked.

Start free → 14-day free trial. No card required.

Published Aug 23, 2026·All posts·The AEO guide