1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Can you check this Robots.txt plz

Discussion in 'White Hat SEO' started by misteryou., Feb 15, 2012.

  1. misteryou.

    misteryou. Power Member

    Joined:
    Feb 1, 2012
    Messages:
    573
    Likes Received:
    109
    HI there

    Can you check this robots.txt files and tell me if its correct please
    and what correction is needed if

    thanks

    User-Agent: *
    Disallow: /cgi-bin/
    Disallow: /admin/
    Disallow: /wp-content/plugins/
    Disallow: /wp-content/cache/
    Disallow: /wp-content/themes/
    Disallow: /wp-images/
    Disallow: /author/
    Disallow: /wp-includes/
    Disallow: /wp-admin/
    Disallow: /img/
    Disallow: /icon-blog/
    Disallow: /trackback/
    Disallow: /wp-login.php/
    Disallow: /wp-register.php/
    Disallow: /xmlrpc.php/
    Disallow /go/
    Disallow: /images/
    Disallow: /backup/
    Disallow: /banners/
    Disallow: /tags /
    Disallow: /archives/
     
  2. mrpasta

    mrpasta Newbie

    Joined:
    Apr 29, 2011
    Messages:
    25
    Likes Received:
    19
    You're missing a colon :)) on this line:
    Disallow /go/

    Other than that it looks good. The way you have it set up will block all bots from indexing any content in those folders you've specified. Is that what you want?
     
  3. misteryou.

    misteryou. Power Member

    Joined:
    Feb 1, 2012
    Messages:
    573
    Likes Received:
    109
    In fact i am in doubt with the banner,img,image :(