1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Noobie help with Robot.exe

Discussion in 'White Hat SEO' started by bpenn34, Jan 4, 2012.

  1. bpenn34

    bpenn34 Junior Member

    Joined:
    Mar 28, 2010
    Messages:
    104
    Likes Received:
    11
    How do you add specific pages to your robot.exe file? I cannot figure it out after searching :/
     
  2. TZ2011

    TZ2011 Senior Member

    Joined:
    Jun 26, 2011
    Messages:
    832
    Likes Received:
    864
    Occupation:
    Cleaning servers
    You mean on robots.txt ? :) look here

    Code:
    http://www.robotstxt.org/robotstxt.html
     
  3. NoHoneyWithoutMoney

    NoHoneyWithoutMoney Regular Member

    Joined:
    Dec 5, 2011
    Messages:
    365
    Likes Received:
    81
    Occupation:
    Killer
    Location:
    * Locating ... Server Down 404 *******************
    Last edited: Jan 5, 2012
  4. GangsterProfit

    GangsterProfit Junior Member

    Joined:
    Aug 26, 2011
    Messages:
    193
    Likes Received:
    69
    Home Page:
    • Thanks Thanks x 1
  5. NoHoneyWithoutMoney

    NoHoneyWithoutMoney Regular Member

    Joined:
    Dec 5, 2011
    Messages:
    365
    Likes Received:
    81
    Occupation:
    Killer
    Location:
    * Locating ... Server Down 404 *******************