Ping search engines like google with urls for fast indexing

Microsoft Bing and Yandex announced a new initiative called IndexNow that enables website owners to ping IndexNow’s log for instant indexing with participating search engines. The participating search engines are currently only Bing and Yandex; Google is not on the list, and other search engines are not yet.

There’s a website at IndexNow.org That sums up what this is all about: “IndexNow is an easy way for website owners to instantly notify search engines of the latest content changes on their website. In its simplest form, IndexNow is a simple ping to let search engines know a URL and whose content has been added, updated, or deleted so that search engines can quickly reflect this change in their search results. “

Well the method of pinging and submitting the urls is pretty simple and it is documented here.

To send a URL with an HTTP request (replace with the one provided by the search engine), send your request to the following URL: https: /// indexnow? Url = url-changed & key = your-key

So at the moment it would be either https://www.bing.com/IndexNow?url=url-changed&key=your-key or https://yandex.com/indexnow?url=url-changed&key=your-key

To send a series of URLs with one HTTP request, send your POST JSON request to the URL provided by search engines. Just replace with the host name of the search engine. You can submit up to 10,000 URLs per post and mix http and https URLs if necessary.

How do you create your key to verify website ownership? Two methods:

(1) Hosting a text key file in the root directory of your host.

(2) Hosting a text key file on your host.

As I said, only Bing and Yandex are included. Google was asked, but they are not there. Baidu, I don’t think so. Others may also have been asked. In order for search engines to adopt the IndexNow protocol, they must agree that submitted URLs are automatically shared with all other participating search engines. In order to participate, search engines must have a noticeable presence in at least one market.

You can the FAQs here.

CMS platforms, CDNs, and SEO tools are all interested in working with this IndexNow protocol, I was told.

This does not replace the Bing URL Submission API, that’s a bit double for this overhead, but the Bing URL Submission API only works with the Bing Index. This also has no effect on the API for submitting Bing content. It just seems like Bing is doubling down on the push-versus-pull method of indexing and is trying to get other search engines to adopt it. It makes sense, it’s more of an efficient way to get new pages – but will there be enough uptake?

For your information, I checked this quickly because I used this site CloudFlare who added support with this announcement.

Why not google? They have their own Indexing API with limited use. I suspect that Google is getting a lot of signals from crawling and is not ready to give up those signals. Google’s John Mueller joked about the announcement:

😉 we too would like to be crawled less often.

– 🧀 John 🧀 (@JohnMu) October 18, 2021

Forum discussion at Twitter & WebmasterWorld.

Leave a Reply

Your email address will not be published. Required fields are marked *