The cohere-ai user-agent is operated by Cohere, a company that builds enterprise large language models. Its crawler gathers web content connected to Cohere's models and services, so it is oriented toward feeding AI systems rather than powering a consumer search engine.
The consequence of blocking is that your material is withheld from Cohere's crawling going forward, keeping it out of the pipeline behind Cohere's models. Businesses that want to be represented in Cohere-powered applications keep it allowed; those preferring to opt out apply the Disallow rule.
How to check and control it
To check, look in your robots.txt for a rule targeting this user-agent. To allow or block it:
User-agent: cohere-ai
Disallow: / # blocks it — remove or set "Allow: /" to permitFor an instant check across your whole site, use SEO AEO Specialist's free AI crawler checker, and see the llms.txt & AI crawler guide.
FAQ
Who operates the cohere-ai crawler?
The cohere-ai user-agent belongs to Cohere, a provider of enterprise large language models. Its crawler collects web content associated with Cohere's models and services, so allowing or blocking it determines whether your pages are part of the data behind Cohere-powered products.
Does the user-agent name need exact capitalization?
Robots.txt user-agent matching is case-insensitive, so cohere-ai and Cohere-AI are treated the same. What matters is that the token itself matches the crawler's declared user-agent. Use the exact string cohere-ai in your rule to be certain it applies.
See where you stand
SEO AEO Specialist runs a free AI-visibility audit and hands you the exact fixes. One-off report €9, unlimited €19/mo.
Run your free audit →