I am running more than 40 job boards in different countries and languages, and I've been using Google Indexing API since its introduction. I'd like to share some observations I made during the last couple of years when the API stopped working as smoothly as before.
Disclaimer: I am talking only about the legit use of the API, particularly for job pages containing proper jobPosting structured data.
1. When did the first issues begin?
Summer 2023 was the time when Indexing API stopped working as good as before. Some job boards lost a lot of traffic from Google Jobs and others continued to perform well. I had to implement the following:
2. Google Indexing API promotes and demotes websites randomly
Sometimes it just turns on and the website starts to get 100% indexation on the first request. But the other website can be demoted and lose its indexation completely. Usually it happens on the domain level but sometimes even subdomains of the same domain perform differently. For example, I have a job board with 12 subdomains. 10 of them are 100% indexed and 2 others have 0%. These fluctuations usually happen once per 1-2 months. They don't depend on domain authority, content quality or any SEO rankings.
A crazy thing happened in the beginning of 2024 when everything suddenly started to work smoothly on all of the websites. I was really excited. But it lasted only for a couple of days and ended up with usual troubles.
3. Google bans Indexing API accounts
It happened to some Google accounts with very large quota. Re-creating service accounts or keys wouldn't help. Some of them started working again after some waiting, but others are still not working. The API requests are processed as usual, returning latestUpdate->notifyTime, but Googlebot is not coming at all. Once I even requested a higher quota for such a "banned" account, and Google approved it, but it still didn't work.
4, Sending individual requests or size of batch requests don't matter
I tried different batch sizes from 2 to 100, tried sending single requests, and it didn't matter at all. What I've noticed is that for some reason on poorly performing websites the number of pages visited by Googlebot is always a multiple of ten. I have no idea why. It can be 10, 20, max 70 for a specific website.
5. Indexing API starts and stops working at random time
I am re-submitting requests at random time of the day and sometimes it just works. I think it might be because of the API abuse when people use the API for numerous Wordpress websites where pages are not legit. But it concerns only "poorly" performing websites. Those that got a "good" flag from Google continue to get 100% indexed regardless of the time of the day.
6. Google doesn't pessimize accounts for quota overuse
Sometimes I am running out of quota and submit multiple requests that can't be processed. Google just rejects the requests and continues as usual after the daily reset at midnight PST.
7. Resubmitting the same URLs multiple times doesn't matter.
When the API starts working, it accepts all the URLs, even those that were already submitted multiple times.
These are my observations on Google Indexing API. It could probably help some other job boards owners. Also I'd appreciate any comments and ideas. I even started to use indexer services for some of my job boards, which sounds like absurd because in theory they should be easily indexed with Indexing API.
Disclaimer: I am talking only about the legit use of the API, particularly for job pages containing proper jobPosting structured data.
1. When did the first issues begin?
Summer 2023 was the time when Indexing API stopped working as good as before. Some job boards lost a lot of traffic from Google Jobs and others continued to perform well. I had to implement the following:
- Check for Googlebot visits on job pages
- Create a script to re-submit unvisited pages to the API
2. Google Indexing API promotes and demotes websites randomly
Sometimes it just turns on and the website starts to get 100% indexation on the first request. But the other website can be demoted and lose its indexation completely. Usually it happens on the domain level but sometimes even subdomains of the same domain perform differently. For example, I have a job board with 12 subdomains. 10 of them are 100% indexed and 2 others have 0%. These fluctuations usually happen once per 1-2 months. They don't depend on domain authority, content quality or any SEO rankings.
A crazy thing happened in the beginning of 2024 when everything suddenly started to work smoothly on all of the websites. I was really excited. But it lasted only for a couple of days and ended up with usual troubles.
3. Google bans Indexing API accounts
It happened to some Google accounts with very large quota. Re-creating service accounts or keys wouldn't help. Some of them started working again after some waiting, but others are still not working. The API requests are processed as usual, returning latestUpdate->notifyTime, but Googlebot is not coming at all. Once I even requested a higher quota for such a "banned" account, and Google approved it, but it still didn't work.
4, Sending individual requests or size of batch requests don't matter
I tried different batch sizes from 2 to 100, tried sending single requests, and it didn't matter at all. What I've noticed is that for some reason on poorly performing websites the number of pages visited by Googlebot is always a multiple of ten. I have no idea why. It can be 10, 20, max 70 for a specific website.
5. Indexing API starts and stops working at random time
I am re-submitting requests at random time of the day and sometimes it just works. I think it might be because of the API abuse when people use the API for numerous Wordpress websites where pages are not legit. But it concerns only "poorly" performing websites. Those that got a "good" flag from Google continue to get 100% indexed regardless of the time of the day.
6. Google doesn't pessimize accounts for quota overuse
Sometimes I am running out of quota and submit multiple requests that can't be processed. Google just rejects the requests and continues as usual after the daily reset at midnight PST.
7. Resubmitting the same URLs multiple times doesn't matter.
When the API starts working, it accepts all the URLs, even those that were already submitted multiple times.
These are my observations on Google Indexing API. It could probably help some other job boards owners. Also I'd appreciate any comments and ideas. I even started to use indexer services for some of my job boards, which sounds like absurd because in theory they should be easily indexed with Indexing API.
