1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

URGENT help robots.txt

Discussion in 'Black Hat SEO' started by MariosElGreco, Oct 7, 2016.

  1. MariosElGreco

    MariosElGreco Regular Member

    Joined:
    May 31, 2014
    Messages:
    261
    Likes Received:
    8
    Gender:
    Male
    Occupation:
    http://eas-seo.com
    Location:
    Greece
    Home Page:
    I have the site eas-seo.com that i like SE visit it ,
    I have the sub site eas-seo.com/examples that i dont like SE visit it.
    If i use robots.txt on eas-seo.com/example those lines it means that SE would not visit any page on eas-seo.com too , or only the eas-seo.com/examples ?
    User-agent: *
    Disallow: /
    Disallow: /cgi-bin/
    Sitemap: http://eas-seo.com/examples/sitemap.xml
     
  2. gsy159

    gsy159 Power Member

    Joined:
    Apr 29, 2011
    Messages:
    667
    Likes Received:
    159
    Add to your robots.txt:
    Disallow: /example/

    and robots wont visit site.com/example but will visit site.com and other links
     
  3. MariosElGreco

    MariosElGreco Regular Member

    Joined:
    May 31, 2014
    Messages:
    261
    Likes Received:
    8
    Gender:
    Male
    Occupation:
    http://eas-seo.com
    Location:
    Greece
    Home Page:
    tyy