Tag Archives: Googlebot

Google No Longer Has A Favicon User Agent & Required Googlebot-Image and Googlebot Crawling

Google has updated its favicon search developer documentation to remove the section for the Google Favicon user agent and to clarify that if you want Google to show your favicon you must allow both Googlebot-Image and Googlebot to crawl your pages. Here is what changed in the favicon documentation – you can also compare yourself […]

Google Says Cease Utilizing 403s or 404s To Cut back Googlebot Crawl Charges

Gary Illyes posted a new blog post on the Google Search Central site asking all of you to stop using 403 and 404 server status codes to reduce the crawl rate of Googlebot. He said they have seen an uptick in the number of sites and CDNs doing this and they need to cut it […]

Google Say Most Widespread Cause For Blocking Googlebot Are Firewalls or CDNs Points

Gary Illyes from Google posted a new PSA on LinkedIn saying that the most common reason a site unexpectedly blocks Googlebot from crawling is due to a misconfiguration of a firewall or CDN. Gary wrote, “check what traffic your firewalls and CDN are blocking.” “By far the most common issue in my inbox is related […]

Google Says If Your Utilizing A CDN Or Cloud Internet hosting Service, Set Up Routine Exams To Guarantee Googlebot Is Not Being Blocked

Gary Illyes from Google loves posting his PSAs on Mastodon. His next PSA is that if you are using a CDN or cloud hosting service, then make sure to test up a routine test to ensure that the service is not blocking Googlebot or other search engine crawlers that you want to crawl and index […]

Bizarre Google Algorithm Patterns, New Google Advert Label, Uncommon Googlebot Crawling Situation, Search Console Loss & Extra Messages

The Google algorithm ranking tracking tools seems to be having fun, or maybe they are off, I am seeing mixed signals of Google ranking updates. Google is testing a way more visible Google ad label with a green curved border, shocking. Google had some weird crawling issue last week, they called it a temporary drop […]

Google Core Replace for November, Google Ads CPC Error, Googlebot Challenge from SiteGround and Snowden on Google

What a week – Google actually released a major core update days before the big online Christmas shopping season – the November 2021 Google core update is live and rolling out. I share my thoughts on the timing, release, more on core updates and the impact so far. SiteGround, a hosting platform that runs 2 […]

Google November Spam Replace Accomplished, Google Testing IndexNow, Googlebot IP Addresses, and Extra

I’ll start by announcing the winners of the Squid Game contest I had last week. Google completed the rollout for the November spam update after 8 days. This November Google spam update seems big, at least it has some legs. Some suggest that this update also targets camouflaged spam content with inserted links. Google said […]