Help Me Google index some weird urls

Discussion in 'Black Hat SEO' started by lmaestro, Jan 18, 2012.

  1. lmaestro

    lmaestro Power Member

    Joined:
    Jan 28, 2011
    Messages:
    667
    Likes Received:
    227
    its something like this

    mydomain/wp-includes/js/comment-reply.js?ver=20090102
    mydomain/wp-includes/js/jquery/jquery.js?ver=1.7.1

    how to deindex these links?
     
  2. SEOWhizz

    SEOWhizz Power Member

    Joined:
    Oct 22, 2011
    Messages:
    608
    Likes Received:
    435
    Location:
    Lat: 38N 43' 11.298" Long: 27W 12' 7.733"
    Remove the pages from the search results using Google Webmaster Tools:

    If the page still exists after 90 days:

    http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1663419
     
    • Thanks Thanks x 1
    Last edited: Jan 18, 2012
  3. Knoxgates

    Knoxgates Supreme Member

    Joined:
    Aug 9, 2008
    Messages:
    1,266
    Likes Received:
    919
    Put this in your Robots.txt file
    Code:
    Sitemap: http://www.yourdomain.com/sitemap.xml
    User-agent: *
    Disallow: /wp-content/
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Disallow: /wp-
    Disallow: /feed/
    Disallow: /trackback/
    Disallow: /cgi-bin/
    User-agent: Googlebot
    Disallow: /*.php$
    Disallow: /*.js$
    Disallow: /*.cgi$
    Disallow: /*.xhtml$
    Disallow: /*.php*
    Disallow: */trackback*
    Disallow: /*?*
    Disallow: /z/
    Disallow: /wp-*
    Disallow: /*.inc$
    Disallow: /*.css$
    Disallow: /*.txt$
     
    • Thanks Thanks x 1
  4. lmaestro

    lmaestro Power Member

    Joined:
    Jan 28, 2011
    Messages:
    667
    Likes Received:
    227
    actually what are those urls? javascript? is it ok if i just leave them? if i alter the Robots.txt file will it mess up my theme or site
     
  5. capripio

    capripio Regular Member

    Joined:
    Dec 25, 2010
    Messages:
    249
    Likes Received:
    157
  6. SEOWhizz

    SEOWhizz Power Member

    Joined:
    Oct 22, 2011
    Messages:
    608
    Likes Received:
    435
    Location:
    Lat: 38N 43' 11.298" Long: 27W 12' 7.733"
    This line will exclude the search engines from crawling all files and folders which start with "wp-includes":
    So, the following URL's will not be crawled:

    Example Wordpress robots.txt:

    (Obviously replace the sitemap with your sites)

    Code:
    http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html
     
    • Thanks Thanks x 1
    Last edited: Jan 18, 2012
  7. lmaestro

    lmaestro Power Member

    Joined:
    Jan 28, 2011
    Messages:
    667
    Likes Received:
    227
    im using xml sitemap plugin.how do i do that?
     
  8. Knoxgates

    Knoxgates Supreme Member

    Joined:
    Aug 9, 2008
    Messages:
    1,266
    Likes Received:
    919
    if you are using sitemap plugin,just replace the yourdomain.com with your actual domains in the first line of Robots.txt file and upload it to your webhost.
    Code:
    Sitemap: http://www.yourdomain.com/sitemap.xml