Google Site Verifier User Agent Added To Google Crawlers

Google has added a new Google crawler, a new user agent, to the Google spider list. Its name is Google Site Verifier User Agent. This user agent ignores robots.txt rules, by the way, and I assume it is used for verifying sites with Search Console and other Google tools and services.

Google posted the new user agent over here in the Google Crawlers help documentation.

The user agent token is: Google-Site-Verification

The full user agent string is: Mozilla/5.0 (compatible; Google-Site-Verification/1.0)

And Google noted with a “caution” that the “Google Site Verifier ignores robots.txt rules.”

So keep an eye out for it on a very limited basis.

Forum discussion at Twitter.

Update: My assumption that this was for Search Console verification was correct but John Mueller of Google said the user agent is not new, the new part is that it was added to the docs:

It’s for Search Console verification. Not new, just newly listed.

— 🌽〈link href=//johnmu.com rel=canonical 〉🌽 (@JohnMu) September 20, 2022