Omgilibot: user agent, purpose, and whether to allow it
Omgilibot collects web content for a data-licensing corpus that has been resold to model builders.
Specification
| Operator | Omgili |
|---|---|
| Purpose | DatasetCollects content into a public dataset that many model builders draw from. Blocking it has broad downstream effect and no direct citation cost. |
| Agent kind | CrawlerAutonomous crawler. Appears in your logs under its own user-agent. |
| User agent | Not documented by Omgili. Kymo matches this agent on the token Omgilibot, observed in real logs rather than published in operator documentation. |
| robots.txt token | Omgilibot |
| robots.txt compliance | No operator statement located |
| Verification | No published method, user agent is the only signal |
| Sends referral traffic | No |
| Attaches a citation | No |
| Status | Active |
| Legacy Kymo category | training |
| Source | Observed in real logs, no operator documentation locatedOmgili publishes no crawler documentation that Kymo could locate. Everything on this page comes from observed traffic, not from a vendor page. |
| Verified on |
What blocking it costs you
Blocking it removes your content from a corpus sold on to third parties. There is no live citation surface, so there is no direct visibility cost.
robots.txt directives
Both blocks below address Omgilibot only. Rules for one token never apply to another, even from the same operator.
Allow Omgilibot
User-agent: Omgilibot Allow: /
Block Omgilibot
User-agent: Omgilibot Disallow: /
Questions
What is the Omgilibot user agent?
Omgili documents the robots.txt token Omgilibot but does not publish a full user-agent string for it. Kymo records this agent from observation rather than from documentation.
Does Omgilibot respect robots.txt?
No operator statement located
What does blocking Omgilibot cost me?
Blocking it removes your content from a corpus sold on to third parties. There is no live citation surface, so there is no direct visibility cost.
Related crawlers
All AI crawlers and control tokens
See this bot in your own logs
Kymo classifies every crawler hit server-side using this registry, so Omgilibot shows up under Omgili with its purpose attached. Read how crawler detection works, or run a free free AI visibility check to see what each crawler currently gets back from your server. No account needed for the check.