How to solve "Crawled - currently not indexed"

T2tkid

T2tkid

Elite Member
Joined
Oct 13, 2010
Messages
5,019
Reaction score
2,479
I was checking my websites and found two sites with these issues;

Crawled - currently not indexed
Discovered – currently not indexed

Focusing on the "Crawled - currently not indexed" one of the websites has /feed in all the URLs. e.g; example.com/secure-the-internet/feed, while the other has no "feed" at the end of the permalink.

Does anyone know what could be the cause? Have you experienced this before? How did you solve it? I have read a few articles that give different opinions, still no solution for 20+ days now.

Note: The two sites have handwritten content and not AI-generated ones.

Thanks.
 
Advertise on BHW
T2tkid said:
I was checking my websites and found two sites with these issues;

Crawled - currently not indexed
Discovered – currently not indexed

Focusing on the "Crawled - currently not indexed" one of the websites has /feed in all the URLs. e.g; example.com/secure-the-internet/feed, while the other has no "feed" at the end of the permalink.

Does anyone know what could be the cause? Have you experienced this before? How did you solve it? I have read a few articles that give different opinions, still no solution for 20+ days now.

Note: The two sites have handwritten content and not AI-generated ones.

Thanks.
Click to expand...
If they are indexable and are not in the index, it's up to Google to decide whether your content is worth indexing.
 
Ranking Expert said:
I can try to index them, if you wish :)
Click to expand...

If I search google for the post with this;
site:https://example.com/ it is there, however, if I remove the site: it is nowhere to be found.

Please, how do you plan to index it? This is a money site :(

Johnik said:
If they are indexable and are not in the index, it's up to Google to decide whether your content is worth indexing.
Click to expand...

True. This is what I have been reading online. However, this writer has been writing for my websites for years. I wonder what change with google.
 
Facing the same issue with loads of pages on my website. :(

The fact that Google decided that particular content isn't worth indexing is really absurd.
I have dozen of copywriters (very experienced ones) working on those pages. They did real investigating work and wrote really good articles on that topics.
Onsite SEO is on point too for entire website.
The domain is clean and we are not doing anything shady there.

I really don't understand what is happening. :(
 
milk_and_cake said:
Facing the same issue with loads of pages on my website. :(

The fact that Google decided that particular content isn't worth indexing is really absurd.
I have dozen of copywriters (very experienced ones) working on those pages. They did real investigating work and wrote really good articles on that topics.
Onsite SEO is on point too for entire website.
The domain is clean and we are not doing anything shady there.

I really don't understand what is happening. :(
Click to expand...

I have been waiting for a few weeks now to see if things will change, I have also deactivated some plugins to see I can figure out the cause. Still no solution yet.

I read somewhere that creating a new sitemap.xml will solve it, I wonder how I can create a new version of sitemap, aside the site.com/sitemap_index.xml (pages & posts included).

Hopefully, someone will come up with an easy solution.
 
T2tkid said:
I have been waiting for a few weeks now to see if things will change, I have also deactivated some plugins to see I can figure out the cause. Still no solution yet.

I read somewhere that creating a new sitemap.xml will solve it, I wonder how I can create a new version of sitemap, aside the site.com/sitemap_index.xml (pages & posts included).

Hopefully, someone will come up with an easy solution.
Click to expand...
Way ahead of you. Tried that too. No luck.
 
T2tkid said:
I have been waiting for a few weeks now to see if things will change, I have also deactivated some plugins to see I can figure out the cause. Still no solution yet.

I read somewhere that creating a new sitemap.xml will solve it, I wonder how I can create a new version of sitemap, aside the site.com/sitemap_index.xml (pages & posts included).

Hopefully, someone will come up with an easy solution.
Click to expand...
I don't think it's a problem on your site. I have the same problem and so do many others. I think it's something on Google's end. Perhaps it's that they don't have enough crawlers for legitimate sites now that people are using AI-generated content to publish millions of pages of half-gibberish/half-legible content per day.
 
I have same problem on 10 pages of my site, some of them aren't indexed from April 2021 :(
 
Go to Twitter and write "Discovered not indexed" in the search bar.

There are a few solutions, especially for the /feed problem someone said above.

However, A LOT of people have this problem. A LOT.

No, google didn't run out of crawlers. It's just their way of saying "fuck you in particular" without saying it.

It seems that for new websites, there is some kind of wait that goes on for 50-70 days before crawlers begin to pickup 1-2 pages every day.

However, for old sites, no one knows.

John Mu says it's quality as a whole. Others say it's just the content, the URL slug, or some kind of keyword possibility that's triggering it.

Some point to YMYL. There's not a lot of discussion about this, but I do see it as something that might be the main culprit.
 
Adding to this, I found Google's bots are more relying on internal linking.. for example a post getting featured on the homepage / sidebar gets a blind eye from search engine bots. Once it is crawled, under referring page it says some other internal URL. Weird!
 
Don't have too much words say about this but if guys following google team on twitter specially john muller than you probably know that google team already indicate about indexing last year.

One of statement i read somewhere that google is considering to reduce indexing budget reduce their carbon footprints.

We are carbon-free, since I don’t even know, 2007 or something, 2009, but it doesn’t mean that we can’t reduce even more our footprint on the environment. And crawling is one of those things that early on, we could chop off some low-hanging fruits.”
Click to expand...

Alexion said:
It seems that for new websites, there is some kind of wait that goes on for 50-70 days before crawlers begin to pickup 1-2 pages every day.
Click to expand...
There are many misconception about google indexing and your spreading one those your statement is absolutely false. Google never developed any programs & algorithm on basis on webpage age it's always depend on quality & value you providing to users no matter you are 10 years old or 10 days
 
Sometimes the cause of this problem may be SSL.

I have faced these issues three months before.

the problem is HTTPS redirect to HTTP.

after fixing the SSL it crawled fine
 
Google crawl bots are slow , these time , Try other indexers and some link pyramids , Would surely help the links to get index !
 
just tell us site:yourdomain.com

how many pages indexed now?
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top