Run free audit
How it works What we check Pricing
Guides
AEO GEO AI Search Visibility SEO Audit AI Citations llms.txt
Free tools
AI Visibility Check llms.txt Generator Robots Checker FAQ Schema
Learn
Blog Glossary FAQ AEO checklist About Sign in
Guides/check/is google cloudvertexbot blocked

Is Google-CloudVertexBot Blocked on Your Site? | SEO AEO Specialist SEO & AEO Specialist

In short

To find out, open your robots.txt and look for a rule naming Google-CloudVertexBot with Disallow: /. If it exists, this crawler is blocked. Blocking limits Vertex AI's ability to ground answers and agents on your content when a customer points it at your site.

Google-CloudVertexBot is operated by Google. It fetches pages on behalf of Google Cloud's Vertex AI customers who build retrieval-augmented generation (RAG) and agent applications, so it acts on a customer's instruction rather than crawling the whole web for training.

The consequence of blocking is that Vertex AI grounding and agents cannot pull your pages as a live source, so answers built on your content in those apps become less accurate or drop your material entirely. This crawler is distinct from Googlebot and Google-Extended, so blocking it does not affect ordinary Google Search.

How to check and control it

To check, look in your robots.txt for a rule targeting this user-agent. To allow or block it:

User-agent: Google-CloudVertexBot
Disallow: /        # blocks it — remove or set "Allow: /" to permit

For an instant check across your whole site, use SEO AEO Specialist's free AI crawler checker, and see the llms.txt & AI crawler guide.

01

FAQ

Is Google-CloudVertexBot the same as Googlebot?

No. Googlebot indexes the web for Google Search. Google-CloudVertexBot fetches pages specifically for Vertex AI customers building RAG and agent apps. They are separate user-agents, so blocking CloudVertexBot limits Vertex AI grounding on your content without touching your Google Search visibility.

Why would Vertex AI need to crawl my site?

When a Google Cloud customer builds an agent or RAG app grounded on your pages, Google-CloudVertexBot fetches that content so the model can cite it accurately. Blocking the crawler means those applications cannot use your live content as a grounded source.

See where you stand

SEO AEO Specialist runs a free AI-visibility audit and hands you the exact fixes. One-off report €9, unlimited €19/mo.

Run your free audit →