Omgilibot is the crawler operated by Webz.io, formerly known as Omgili. It gathers large volumes of web data that Webz.io packages into datasets and feeds licensed to AI, research, and analytics customers, so it is a data-collection crawler rather than a consumer search engine.
The consequence of blocking is that your content stops flowing into Webz.io's datasets, so the many downstream products and models built on that data no longer include your material. Site owners who want to limit commercial data resale of their content use this Disallow rule to opt out.
How to check and control it
To check, look in your robots.txt for a rule targeting this user-agent. To allow or block it:
User-agent: Omgilibot
Disallow: / # blocks it — remove or set "Allow: /" to permitFor an instant check across your whole site, use SEO AEO Specialist's free AI crawler checker, and see the llms.txt & AI crawler guide.
FAQ
Who operates Omgilibot?
Omgilibot is run by Webz.io, previously called Omgili. It collects web data that Webz.io compiles into commercial datasets and feeds for AI, research, and analytics customers. Blocking it opts your content out of those datasets, keeping it out of the products built on Webz.io data.
What happens to my content in existing Webz.io datasets?
Blocking Omgilibot prevents future collection of your pages. Data already gathered and packaged into earlier datasets is not retroactively removed by a robots.txt change. The Disallow rule takes effect going forward, gradually reducing your presence as Webz.io's datasets refresh.
See where you stand
SEO AEO Specialist runs a free AI-visibility audit and hands you the exact fixes. One-off report €9, unlimited €19/mo.
Run your free audit →