Programmatic website dropping out of index

SariMcat · Mar 22, 2023

My programmatic website is slowly dropping out of the index.

I started with around 5,000 URLs indexed using rank math API and google is slowly removing them. It seems they are being flagged as "Crawled - Currently not indexed".

Any idea what is happening and how I can get these indexed again?

V · Mar 22, 2023

That's why I avoid indexing the pages using any API/service. Let Google index the pages naturally. Also, how many pages are you creating per day?

SariMcat · Mar 22, 2023

V said:
That's why I avoid indexing the pages using any API/service. Let Google index the pages naturally. Also, how many pages are you creating per day?

I've stopped creating pages. I was using the drip feed method by scheduling 200 posts/day

lbhbochc · Mar 22, 2023

It's because your content is low quality.

"Crawled, currently not indexed" means google has crawled it and processed it but doesn't see a reason to index it.

You need to up your game, programmatic SEO isn't as easy as it used to be.

Steptoe · Mar 22, 2023

SariMcat said:
I started with around 5,000 URLs indexed using rank math API and google is slowly removing them. It seems they are being flagged as "Crawled - Currently not indexed".

The API that rank math is using is only meant for short-lived content such as live streams and job postings, which is why you'll see urls dropping out of the index after a while: https://rankmath.com/wordpress/plugin/instant-indexing/ https://developers.google.com/search/apis/indexing-api/v3/quickstart

Take a look at how unique each of your generated pages are. Do they have a lot of boilerplate per page and not much actual content? Do they have sufficient internal links pointing to them? Any external links?

BroTherm · Mar 22, 2023

This is what happens with thin content programmatic sites. In the end, some stay indexed but the majority goes.
Rework your system to generate more unique text on each page.

SariMcat · Mar 22, 2023

Steptoe said:
The API that rank math is using is only meant for short-lived content such as live streams and job postings, which is why you'll see urls dropping out of the index after a while: https://rankmath.com/wordpress/plugin/instant-indexing/ https://developers.google.com/search/apis/indexing-api/v3/quickstart

Take a look at how unique each of your generated pages are. Do they have a lot of boilerplate per page and not much actual content? Do they have sufficient internal links pointing to them? Any external links?

My pages aren’t any less unique than my main competitor who has thousands of pages indexed and is using a custom internal linking widget similar to inline related posts plugin.

SariMcat · Mar 22, 2023

BroTherm said:
This is what happens with thin content programmatic sites. In the end, some stay indexed but the majority goes.
Rework your system to generate more unique text on each page.

I’ll need to take another approach. There is a way to integrate chatGPT to google sheets in which I can generate unique content based on each cell

BroTherm · Mar 22, 2023

There are hundred of prog sites in most niches. Some are very old and well linked and these are the ones that have the many indexed pages.
Throughout the years most gave developed a way to stuff words in the pages. Many repeat the same info many times in different forms, other spin some text to complete the task.
If your competitor ranks, you with similar content, you will have to create better pages and compete with his link profile.

Programmatic website dropping out of index

SariMcat

Newbie

V

Elite Member

SariMcat

Newbie

lbhbochc

Senior Member

Steptoe

Elite Member

BroTherm

Junior Member

SariMcat

Newbie

SariMcat

Newbie

BroTherm

Junior Member

Main Menu

Marketplace

Making Money

BlackHat World