Pages crawled but not indexed.

[FrozenSpade]

edit title: It should have been Discovered - currently not indexed (instead of crawled and not indexed).

I have 150 pages on my new site. Maybe I pushed it too hard?

US native writers. Imo the articles are really good quality.
Site speed is 99/100. No technical issues with robots.txt or any other noindex tags.
I even have some high DR backlinks (natural DR70+)
Site is 3 months old.
Everything is interlinked.
Fresh domain with no backlinks on date of registering it.

My GSC looks like this (see attached file).

Only 29 pages indexed from 150.
While I understand that it takes time for new sites to get indexed completely, it shouldn't take this long. Also indexing just stopped completely after the first weeks.

Any ideas?

 

[praetserge]

"Pages crawled but not indexed" or discovered and not indexed?
Don't like crawled not indexed as it kinda implies that google crawled the page(s) and didn't bother to index.

PS: I have some of those but they are not indexed for a reason (on an old-ish site) so I either remove or improve them.

 

[Jimoz]

Well googlebot crawl alot of pages daily but doesn't mean it will index them all.

For any site there is no 100% indexation rate of all pages. and sometime google is taking a while to index could takes months too.

 

[FrozenSpade]

"Pages crawled but not indexed" or discovered and not indexed?
Don't like crawled not indexed as it kinda implies that google crawled the page(s) and didn't bother to index.

PS: I have some of those but they are not indexed for a reason (on an old-ish site) so I either remove or improve them.

You are right. It's Discovered - currently not indexed.

@Jimoz for my sites it never took months to index and google was not picky at all in the past. I have tens of other sites and all content pages are indexed.

 

[Jimoz]

You are right. It's Discovered - currently not indexed.

@Jimoz for my sites it never took months to index and google was not picky at all in the past. I have tens of other sites and all content pages are indexed.

I understand, but if you do a simple search on bhw about indexing problem, Lately alot of people facing indexing issues even for high quality content, so may be its a sites to sites case. I suggest you keep submitting at GSC to index to see if it helps.

 

[Furious Man]

This is due to many reasons! You should check the SEO of your webpages, the quality of your content, and the traffic statistics from your Analytics account. Find and fix any issues in your Search Console. Spun content will not rank either. The low quality pages, as well as new blogs, do not get indexed quickly.

 

[FrozenSpade]

I understand, but if you do a simple search on bhw about indexing problem, Lately alot of people facing indexing issues even for high quality content, so may be its a sites to sites case. I suggest you keep submitting at GSC to index to see if it helps.

Yeah I heard about that. It seems to be a problem since December. Honestly I think it's some google glitch or something. I mean, my blog ticks all the boxes for quality. I submitted at least 20-30 pages to GSC about 1 months ago. None indexed.

@Furious: as said the content is top quality and so is the technical side of my site. I haven't used spun content since 2013. The only thing is that my blog is 3 months old but even then, google should at least index the pages even if it doesn't rank them.

 

[CammerCash]

Check your canonicals.
Check duplicated content.
Try to improve internal and external backlinks.

 

[praetserge]

You are right. It's Discovered - currently not indexed.

Then you have NO problem, google found extra posts through sitemap or internal links but hasn't sent the crawlers yet. At times, indexing is a bit slow.
So you could:
1. Update modification date
2. Manually request indexing

 

[Alexion]

This is a problem that goes longer than December.

As of now, nobody can actually pinpoint the issue.

However, a lot of people, myself included, seem to conclude that the issue lies in the content, somehow. Either OVER optimized, under optimized, nothing new, so it might count as duplicate even though it is not. There is a finite way to explain people how to lose weight.

I've managed to get some pages that refused to get indexed by adding images, linking to external sources, and internal as well, and add something unique to the content.

However, I have NOT been able to replicate it on all the pages that refuse to get indexed. But I am 99% sure the issue lies within the content itself.
Being high quality is not the main factor here. I'm sure all of us saw crappy articles ranking in top 3. I trully believe it's the unique touch that makes a difference. Some parahraphs that aren't available on other sources yet.

Google is far more advanced that we chose to believe. I'm confident they have machines who can tell the main concept behind an article. Even with unique wording, and style, if the main idea is the same over and over again, they chose simply to exclude it.

Other webmasters went at it the backlink way, thinking it's because of lack of backlinks. As some truth lies there as well, I simply don't believe this is the main issue. How can new content have backlinks if it's not available. Don't say lack of social signals, as they are irrelevant to indexing. You don't get pages indexed by posting stuff on FB.

I have a few blogspot blogs, created 1 month ago. Each one of them have around 10 unique articles. They are actually unique, BUT the main idea is already covered thousands of times. It simply won't index, even though I am daily requesting manual indexing. Nothing happens.

I am sure there is a solution, and I am also very sure that SEO gurus know about it. But I don't see any of them posting a solution. And it's understandable. In these times, such a solution could mean GOLD.

Arguments like "wait more" or "google doesn't index everything" don't apply to your case, and in a lot of our cases as well. Posts SHOULD get indexed, as googlebot is a BOT, and how it choses to index SOME pages, it should index the others as well.

 

[FrozenSpade]

I've managed to get some pages that refused to get indexed by adding images, linking to external sources, and internal as well, and add something unique to the content.

Good idea. I will try this today.
In fact I will try multiple things on different pages and see what's working.

Thank you Alexion. Best answer yet.

 

[Alexion]

I would love to see other members input on this, as I still experience this problem with posts that refuse to get index. I just shared my opinion, but it's not a solution, sadly.

The more I look into google forums, the more I see google specialists point out to a "potential overload" of the webserver, and in return, google decides to come back another time. However, how could they overload their own servers (i.e. blogspot.com)? Or wordpress for that matter.

 

[FrozenSpade]

I would love to see other members input on this, as I still experience this problem with posts that refuse to get index. I just shared my opinion, but it's not a solution, sadly.

The more I look into google forums, the more I see google specialists point out to a "potential overload" of the webserver, and in return, google decides to come back another time. However, how could they overload their own servers (i.e. blogspot.com)? Or wordpress for that matter.

I never experienced this before December. Heck, even my AI generated content site got indexed 3-4 months ago. All of the pages.
My guess is that with all the AI content that is being added now, google reduced the crawl budget for new sites.

Just think about how much AI content is being added on a daily basis now. Sites start to appear that have 30k pages on page one. This was not the case 1 year ago.

 

[Jimoz]

I never experienced this before December. Heck, even my AI generated content site got indexed 3-4 months ago. All of the pages.
My guess is that with all the AI content that is being added now, google reduced the crawl budget for new sites.

I would agree here with your statement on AI content generated sites. Loads of sites using AI content and Google is using it own AI to filter crappy content and thus not index sites with good content as well but I don't think its about crawl budget.

 

[VibroHeavenUK]

@FrozenSpade I'm having the same problem, when I was drop shipping adult content using the drop shippers photos and descriptions, everything indexed but when I changed my website to more general items, loads of pages won't index even after requesting them individually yet my website average position is moving up everyday! I know website ranking also uses keywords.

 

[Starblazer]

If it says discovered but not indexed, then it's crawl budget issue. You should consistently post content to maintain crawl budget. If it says crawled but not indexed, then it's content issue. You should check for plagiarism and quality.

 

[AliceInSEOLand]

Try this:
https://www.blackhatworld.com/seo/fast-indexing-of-200-pages-via-google-api.1374768/

 

[Alexion]

Try this:
https://www.blackhatworld.com/seo/fast-indexing-of-200-pages-via-google-api.1374768/

This does the same thing as Request Indexing, but it's automatic instead of manually. It's not a solution to the problem of "Discovered currently not indexed".

Some guys on Twitter suggested changing the URL slug AND the publish date. Some report indexing in 1-2 days after this.
I've tried it on 4 posts, will update results.

 

[FrozenSpade]

This does the same thing as Request Indexing, but it's automatic instead of manually. It's not a solution to the problem of "Discovered currently not indexed".

Some guys on Twitter suggested changing the URL slug AND the publish date. Some report indexing in 1-2 days after this.
I've tried it on 4 posts, will update results.

I am also trying everything that was suggested. With different pages I tried this (tried with 3 pages per item):

changed the URL, added new images, added new content and external links.
I will try more things today. Hopefully something will work.

I don't have a post date on the page so I will try adding that as well.

 

[Time Saver]

Here are the top five issues

Crawled – currently not indexed (Quality issue).

Duplicate content.

Discovered – currently not indexed (Crawl budget/quality issue).

Soft 404.

Crawl issue.