1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Help Me Google index some weird urls

Discussion in 'Black Hat SEO' started by lmaestro, Jan 18, 2012.

  1. lmaestro

    lmaestro Power Member

    Joined:
    Jan 28, 2011
    Messages:
    666
    Likes Received:
    227
    its something like this

    mydomain/wp-includes/js/comment-reply.js?ver=20090102
    mydomain/wp-includes/js/jquery/jquery.js?ver=1.7.1

    how to deindex these links?
     
  2. SEOWhizz

    SEOWhizz Power Member

    Joined:
    Oct 22, 2011
    Messages:
    606
    Likes Received:
    432
    Location:
    Lat: 38N 43' 11.298" Long: 27W 12' 7.733"
    Remove the pages from the search results using Google Webmaster Tools:

    If the page still exists after 90 days:

    http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1663419
     
    • Thanks Thanks x 1
    Last edited: Jan 18, 2012
  3. Knoxgates

    Knoxgates Supreme Member

    Joined:
    Aug 9, 2008
    Messages:
    1,266
    Likes Received:
    918
    Put this in your Robots.txt file
    Code:
    Sitemap: http://www.yourdomain.com/sitemap.xml
    User-agent: *
    Disallow: /wp-content/
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Disallow: /wp-
    Disallow: /feed/
    Disallow: /trackback/
    Disallow: /cgi-bin/
    User-agent: Googlebot
    Disallow: /*.php$
    Disallow: /*.js$
    Disallow: /*.cgi$
    Disallow: /*.xhtml$
    Disallow: /*.php*
    Disallow: */trackback*
    Disallow: /*?*
    Disallow: /z/
    Disallow: /wp-*
    Disallow: /*.inc$
    Disallow: /*.css$
    Disallow: /*.txt$
     
    • Thanks Thanks x 1
  4. lmaestro

    lmaestro Power Member

    Joined:
    Jan 28, 2011
    Messages:
    666
    Likes Received:
    227
    actually what are those urls? javascript? is it ok if i just leave them? if i alter the Robots.txt file will it mess up my theme or site
     
  5. capripio

    capripio Regular Member

    Joined:
    Dec 25, 2010
    Messages:
    248
    Likes Received:
    157
    Occupation:
    Programmer
    Location:
    127.0.0.1
  6. SEOWhizz

    SEOWhizz Power Member

    Joined:
    Oct 22, 2011
    Messages:
    606
    Likes Received:
    432
    Location:
    Lat: 38N 43' 11.298" Long: 27W 12' 7.733"
    This line will exclude the search engines from crawling all files and folders which start with "wp-includes":
    So, the following URL's will not be crawled:

    Example Wordpress robots.txt:

    (Obviously replace the sitemap with your sites)

    Code:
    http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html
     
    • Thanks Thanks x 1
    Last edited: Jan 18, 2012
  7. lmaestro

    lmaestro Power Member

    Joined:
    Jan 28, 2011
    Messages:
    666
    Likes Received:
    227
    im using xml sitemap plugin.how do i do that?
     
  8. Knoxgates

    Knoxgates Supreme Member

    Joined:
    Aug 9, 2008
    Messages:
    1,266
    Likes Received:
    918
    if you are using sitemap plugin,just replace the yourdomain.com with your actual domains in the first line of Robots.txt file and upload it to your webhost.
    Code:
    Sitemap: http://www.yourdomain.com/sitemap.xml