1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Too many pages indexed, how to reduce the amount?

Discussion in 'Black Hat SEO' started by lazlow, Nov 3, 2011.

  1. lazlow

    lazlow Regular Member

    Joined:
    Mar 22, 2009
    Messages:
    434
    Likes Received:
    965
    Occupation:
    Webmaster
    Location:
    This Place
    Hi,

    my wordpress blog has 10k posts, I installed SEO SearchTerms Tagging 2 and my traffic went sky high but also my indexed page count went up to 40k, found out about it only after a while.

    So then my traffic started to go down and so did my position in SERP.
    Now I have 40k indexed pages and shitty rank position.

    I've remade my robots.txt to this

    Code:
    User-agent: *
    
    Disallow: /cgi-bin
    Disallow: /wp-*
    Disallow: */feed*
    Disallow: /comments
    Disallow: /author
    Disallow: /tag
    Disallow: /archives
    Disallow: /20*
    Disallow: /iframes
    Disallow: /privacy-policy.html
    Disallow: /web-site-agreement.html
    Disallow: /category/*/*
    Disallow: */trackback*
    
    User-agent: Googlebot
    Disallow: *.php
    Disallow: /*.js$
    Disallow: /*.inc$
    Disallow: /*.css$
    Disallow: /*.gz$
    Disallow: /*.wmv$
    Disallow: /*.cgi$
    Disallow: /*.xhtml$
    Disallow: /*.xlsx $
    Disallow: /*.doc$
    Disallow: /*.pdf$
    Disallow: /*.zip$
    
    User-agent: *
    Allow: /images
    
    Sitemap: http://yourdomain.com/sitemap.xml.gz
    Any suggestions on how to minimize indexed page count and speed the process up? Will that benefit my SERP?

    Thanks!
     
  2. Em][n3m

    Em][n3m Power Member

    Joined:
    Dec 8, 2010
    Messages:
    558
    Likes Received:
    145
    Occupation:
    Student
    Location:
    City of Lost Heaven
    LOOOL

    It is very good having 40k page indexed in google

    why you want them de-indexed?
     
  3. moonpromo

    moonpromo Newbie

    Joined:
    Aug 26, 2010
    Messages:
    30
    Likes Received:
    12
    Location:
    Ukraine
    Same thing happened to me. I recommend you to delete all unnecessary pages and set 410 error, not 404. This will work much quicker than 404
     
  4. phpbuilt

    phpbuilt Jr. VIP Jr. VIP

    Joined:
    May 16, 2011
    Messages:
    1,650
    Likes Received:
    5,208
    Occupation:
    $ from websites I own.
    Location:
    putting monkeys in paypal
    Your traffic might also be going down because the tags plugin injected a whole ton of linked keywords onto all your pages and Google considers it keyword stuffing.

    Instead of tags, consider something like YARPP (yet another related posts plugin), reducing your internal backlinking to only categories and the 5 most related posts works great for me. I do have some sites where tags make sense for visitors, on those sites I follow,noindex.

    If you noindex your tags, be prepared to wait about 6 months for Google to get rid of them all out of the index. They like to drag their feet on that.