Tag Archives: IfModifiedSince

Google has added a new section to the crawl budget management help document. The new section is for the If-Modified-Since request header. The section is titled “Specify content changes with HTTP status codes.” Google wrote“Google generally supports the If-Modified-Since and If-None-Match HTTP request headers for crawling. Google’s crawlers don’t send the headers with all crawl […]