Google indexing pages and categories, should I noindex?

[Scorpion Ghost]

I just did a site:website.com search on Google for one of my Wordpress sites, and I found the first 2 pages of google populated by Pages and Categories and Sub-categories.

For example:

https://website.com/page/7/https://website.com/page/12/
https://website.com/whatever/https://website.com/stuff/
https://website.com/whatever/other/https://website.com/whatever/stuff/
My actual posts start from the end of page 2 and onwards. But even on page 3 and page 4 there are some page/category/subcategory pages indexed.

This looks kinda shitty, I don't know.

Should I add a noindex on all these pages? What do you guys do? What's the 2022 best advice?

 

[Scorpion Ghost]

 

[paraztos]

I personally noindex cat and tag pages, but most ppl say it doesn't matter. I guess they provide some internal link juice if you keep them in the index. Once your site starts to pick up google will prioritize content pages over tag/cat pages.

 

[Starblazer]

I index category & tag archives but noindex pages. I also noindex pagination of all archives (which means page 2, page 3 etc. won't be indexed). But it's a personal preference and there is no thumb rule to decide them.

 

[Beklink]

I always no index tags and category archives

 

[Scorpion Ghost]

I personally noindex cat and tag pages, but most ppl say it doesn't matter. I guess they provide some internal link juice if you keep them in the index. Once your site starts to pick up google will prioritize content pages over tag/cat pages.

I created this website many years ago, back when I was SEO Expert

So I just checked, it seems I have content="noindex,follow" for my TAG pages. So the TAG pages are noindex. Also they don't show in the google search, so that's fine.

I don't have noindex for category pages. But each of my category pages has a unique description on top of the page, so I guess there's some uniqueness there. Plus they take up some more places in the searches, so I guess I'd just leave that be.

But those pagination pages... I don't like them in the index...

I index category & tag archives but noindex pages. I also noindex pagination of all archives (which means page 2, page 3 etc. won't be indexed). But it's a personal preference and there is no thumb rule to decide them.

What are category and tag archives?

You have category pages, you have tag pages. You have an archive page (like a sitemap).

There's no such thing as a category archive or a tag archive. I guess there could be, but that's like pointless to have. Like, here's my archives page which contains all my posts and dates when they were posted. This could be useful to someone.

And then here's my categories archive. That not even a thing. Your categories pages/archive is all in your menu bar on your site. Maybe I'm not understanding you.

I just checked and I also noindex the Pages on the site. So I noindex tags and pages. I do index posts and category pages. Good.

---

This only leaves those pagination pages (homepage -> page 1 -> page 2 -> page 3, etc).

I don't like seeing them in the google search results. And the URL being indexed is like totally non-descriptive, like nonsense -> website.com/page/12/

Stupid...

But I read some article about this earlier, and if I understood correctly, the guy said it's good to have these pages indexed because it makes it more likely for Google to find all of your content (so that Google doesn't only rely on the sitemap). However, having Categories indexed pretty much gets google everything. At least the categories have descriptive URLs for their indexing, it's not just nonsense like the pagination pages.

Still undecided if I should noindex those pagination pages. Let's wait for some more replies...

 

[Scorpion Ghost]

I always no index tags and category archives

OMG you too.

What are category archives? Please explain.

I know of category pages. So this is a category page, and all the posts marked with this category will show if you open this category page.

What is a category archive?

 

[Scorpion Ghost]

If I'm understanding correctly, what you guys call "Category archives" are just what I call "Category Pages". Poteto Potato. Same thing.

I'm cool with noindex the TAG pages because many of them probably just have 1 post under them, at least for me. It's ridiculous to have them indexed.

But the CATEGORY Archives/PAGES are good to index in my opinion, especially if you customize them a bit to have something unique on them. Adds a bit of value, gives you a few more slots in the search engine results.

That's just what I think.

 

[Starblazer]

What are category and tag archives?

Archive simply means a collection. It's just a nomenclature people often use because category pages and tag pages contain a list of posts in that category or having that tag. Websites usually have 4 types of archives: Date archives, Author archives, Category archives, and Tag archives. These are the collections of posts grouped based on their date, author, category, and tag.

 

[Scorpion Ghost]

Archive simply means a collection. It's just a nomenclature people often use because category pages and tag pages contain a list of posts in that category or having that tag. Websites usually have 4 types of archives: Date archives, Author archives, Category archives, and Tag archives. These are the collections of posts grouped based on their date, author, category, and tag.

Look at him with the $5 dollar words. Mr Nomenclature over here. I see that dictionary is serving you well hahaha

So I did a bit of reading and thinking, and I think I'm gonna noindex pagination pages.

I figure, the search engines can find all my posts from my sitemap which I submit to webmaster tools (google, bing, yandex).

They can find all my posts from category pages (archives), which are being indexed.

They can find all my posts from interlinking that I do in my posts.

They can find all my posts from the "Related Posts" sidebar widget and the "Similar Posts" widget under each of my blog posts.

Maybe they can find them from my Archives page (the page where all my blog posts are listed). Even though this page is noindex, a search engine spider could still find its way there maybe. Not sure about this one.

Point is, all of my posts can easily be found by search engines in many ways...

What bothers me is that Google indexes these pagination pages, and then the URL of each of those indexed pages is totally non-descriptive, but google picks a random post from the page and uses that post's title as the title in the search result. And now if a user clicks to open that result because they like the title, they will land on a pagination page instead of on that article. Of course, that article will most likely be on that pagination page, but this is no good forcing people to jump through hoops like that.

And in some cases that article may not even be on that pagination page. Like if I deleted an article or posted a new one, and then the position of my articles in the pagination pages changes, they all move either up or down. And then a user comes to my site to a pagination page through google, but google didn't update that result yet and is still showing the old article on pagination page 6, when in fact that article is on page 5 or page 7 now.

Meh, stupid, I'm gonna noindex pagination pages. Fuck you google

 

[Scorpion Ghost]

https://wordpress.org/support/topic/google-indexing-page-2-page-3-how-to-prevent-it/
According to Google, adding noindex on a page long-term will lead to them not following links on that page. This, you have also rightly mentioned. More recent statements imply that if a page isn’t in their index, the links on it and from it can’t be evaluated at all. So with that, we do not recommend adding noindex to page 2 and further on paginated archives.

---

His recommendation is retarded. It's only valid perhaps if that particular pagination page is the only page where a search engine can find your link.

Unless if he meant that if a page isn't in Google's index, then any links on it and from it can't be evaluated at all, even if they are found on other pages. But I doubt it. That's stupid.

Someone is retarded here. It may be me, or it may be that guy

Stupid fucking advice masquerading as something legit (huh? masquerading? I have some $5 dollar words in my arsenal too @Starblazer )

 

[Starblazer]

Look at him with the $5 dollar words. Mr Nomenclature over here. I see that dictionary is serving you well hahaha

So I did a bit of reading and thinking, and I think I'm gonna noindex pagination pages.

I figure, the search engines can find all my posts from my sitemap which I submit to webmaster tools (google, bing, yandex).

They can find all my posts from category pages (archives), which are being indexed.

They can find all my posts from interlinking that I do in my posts.

They can find all my posts from the "Related Posts" sidebar widget and the "Similar Posts" widget under each of my blog posts.

Maybe they can find them from my Archives page (the page where all my blog posts are listed). Even though this page is noindex, a search engine spider could still find its way there maybe. Not sure about this one.

Point is, all of my posts can easily be found by search engines in many ways...

What bothers me is that Google indexes these pagination pages, and then the URL of each of those indexed pages is totally non-descriptive, but google picks a random post from the page and uses that post's title as the title in the search result. And now if a user clicks to open that result because they like the title, they will land on a pagination page instead of on that article. Of course, that article will most likely be on that pagination page, but this is no good forcing people to jump through hoops like that.

And in some cases that article may not even be on that pagination page. Like if I deleted an article or posted a new one, and then the position of my articles in the pagination pages changes, they all move either up or down. And then a user comes to my site to a pagination page through google, but google didn't update that result yet and is still showing the old article on pagination page 6, when in fact that article is on page 5 or page 7 now.

Meh, stupid, I'm gonna noindex pagination pages. Fuck you google

If you use SEO plugins, you can setup meta titles and descriptions for categories and tags. For example, if you have a weight loss products review site, you can have multiple categories like weight loss diets, weight loss pills, weight loss exercises etc. You can write meta description for that specific title. So, what shows up in SERPs is what you write in meta title & desc.

You can either strictly follow Google and Gurus, or follow BHW tactics.

 

[Scorpion Ghost]

If you use SEO plugins, you can setup meta titles and descriptions for categories and tags. For example, if you have a weight loss products review site, you can have multiple categories like weight loss diets, weight loss pills, weight loss exercises etc. You can write meta description for that specific title. So, what shows up in SERPs is what you write in meta title & desc.

You can either strictly follow Google and Gurus, or follow BHW tactics.

Sorry, I only follow the voices in my head

 

[Scorpion Ghost]

Okay, so I found some people who are not retarded full of retarded advice pretending like they're telling you something.

This guy -> https://techtage.com/yoast-seo-fix/

Well explained, good job.

But his solution doesn't work for me. Asking you to install a plugin. Ridiculous. Fuck that guy!!!

...

So I looked further, and found this -> https://www.intelliwolf.com/noindex-wordpress-subpages/

That worked adding the code in functions.php. That's the proper solution to this problem. You don't go installing a plugin for every little thing you want to do on your site. Plugins slow your site down, and open you up to getting hacked, and just add more confusion to the mess. Less is better!

Anyway I implemented the solution in functions.php, and I did a check.

Homepage = yesindex

Blog posts = yesindex

Category pages and subpages = yesindex

Category pages and subpages pagination (page 2, page 3) = noindex

Pages = noindex

Tag pages = noindex

Tag pages pagination (page 2, page 3) = noindex

The only weird-ish thing is that on Tag pages pagination I have the "noindex" tag twice. This is because one is added by the code I put in functions.php, and one is added by Yoast (for tag pages). I did a quick google search and a guy on some forum said that it shouldn't be a problem. But fuck what that guy said, I can't see any problems with this either. Let the noindex tag be there twice. Whatever...

I think I didn't miss anything. If I did I'm sure you guys will let me know... not

---

So, it seems I'm done here. All is as it should be in the world

 

[Beklink]

OMG you too.

What are category archives? Please explain.

I know of category pages. So this is a category page, and all the posts marked with this category will show if you open this category page.

What is a category archive?

Archives are the collection of posts from categories, tags, author, date. I always no index them(not category pages, I meant tags, author, date archives) because of the thin content that google doesn't like

 

[Scorpion Ghost]

Over 2 months later and google is still showing the pages I set to NOINDEX in their search results.

Stupid assholes...

 

[UpwardDescent]

Over 2 months later and google is still showing the pages I set to NOINDEX in their search results.

Stupid assholes...

That's strange and the

<meta name="robots" content="noindex, nofollow">

Should do the trick.
Maybe check to see if you have another line for "robots" that overwrites that rule?

 

[Scorpion Ghost]

That's strange and the

<meta name="robots" content="noindex, nofollow">

Should do the trick.
Maybe check to see if you have another line for "robots" that overwrites that rule?

The only thing I see if I check the page source of these pages is this:

<meta name='robots' content='noindex, follow'/>

That should be it.

I checked my .htaccess file and, while I'm no expert, I don't see anything that would override this. I mean, why would there be something in the .htaccess file to override it anyway? I don't think so.

Anyway this SEO/Google/Ranking/Indexed/Not indexed crap is cancer. I'm glad I early on in my internet "career" decided not to dick around with SEO. I try to follow best practices, but this shit is like a house of cards, unstable, unpredictable, unreliable, un... unwelcome!

 

[Scorpion Ghost]

3 months later and google is still showing the pages I set to NOINDEX in their search results.

Stupid assholes...

 

[Scorpion Ghost]

Okay, so I found some people who are not retarded full of retarded advice pretending like they're telling you something.

This guy -> https://techtage.com/yoast-seo-fix/

Well explained, good job.

But his solution doesn't work for me. Asking you to install a plugin. Ridiculous. Fuck that guy!!!

...

So I looked further, and found this -> https://www.intelliwolf.com/noindex-wordpress-subpages/

That worked adding the code in functions.php. That's the proper solution to this problem. You don't go installing a plugin for every little thing you want to do on your site. Plugins slow your site down, and open you up to getting hacked, and just add more confusion to the mess. Less is better!

Anyway I implemented the solution in functions.php, and I did a check.

Homepage = yesindex

Blog posts = yesindex

Category pages and subpages = yesindex

Category pages and subpages pagination (page 2, page 3) = noindex

Pages = noindex

Tag pages = noindex

Tag pages pagination (page 2, page 3) = noindex

The only weird-ish thing is that on Tag pages pagination I have the "noindex" tag twice. This is because one is added by the code I put in functions.php, and one is added by Yoast (for tag pages). I did a quick google search and a guy on some forum said that it shouldn't be a problem. But fuck what that guy said, I can't see any problems with this either. Let the noindex tag be there twice. Whatever...

I think I didn't miss anything. If I did I'm sure you guys will let me know... not

---

So, it seems I'm done here. All is as it should be in the world

So I decided to take a second look at the situation.

It seems that the Category pages and subpages pagination (page 2, page 3) are indexed, and the noindex tag is set by me (this is also for homepage pagination)

The Pages are not indexed, and the noindex tag is set by Yoast plugin

The Tag pages are not indexed, and the noindex tag is set by Yoast plugin

The Tag pages pagination are not indexed, and the noindex tag is set by Yoast plugin (but these pages also have the noindex tag that was set by me)

---

This leads me to think I fucked up with the noindex tag. However, the only difference between what I added and what the Yoast plugin adds are quotes.

Yoast:

<meta name="robots" content="noindex,follow"/>

Me:

<meta name='robots' content='noindex, follow'/>

I got the idea from here where a guy complained his noindex was not working, and then someone told him it's not working because he has curly quotes instead of regular quotes. Well, I have single quotes instead of double quotes, so maybe that's my issue - https://moz.com/community/q/topic/65472/why-is-our-noindex-tag-not-working/4

I also ran a page from my site that only has my noindex tag here, and it shows that Meta robots: noindex, follow, as it should be - https://www.seoreviewtools.com/bulk-meta-robots-checker/

But even though this tools says so, google still indexes those pages.

---

So I'll edit my tag to include the double quotes, and then wait a few weeks to see if google finally deindexes those pages. Fingers crossed.

#SeoSucks