Tag Archives: 403s

Google Says Cease Utilizing 403s or 404s To Cut back Googlebot Crawl Charges

Gary Illyes posted a new blog post on the Google Search Central site asking all of you to stop using 403 and 404 server status codes to reduce the crawl rate of Googlebot. He said they have seen an uptick in the number of sites and CDNs doing this and they need to cut it […]