Noobie help with Robot.exe

Discussion in 'White Hat SEO' started by bpenn34, Jan 4, 2012.

  1. bpenn34

    bpenn34 BANNED BANNED

    Joined:
    Mar 28, 2010
    Messages:
    107
    Likes Received:
    12
    How do you add specific pages to your robot.exe file? I cannot figure it out after searching :/
     
  2. TZ2011

    TZ2011 Senior Member

    Joined:
    Jun 26, 2011
    Messages:
    834
    Likes Received:
    866
    You mean on robots.txt ? :) look here

    Code:
    http://www.robotstxt.org/robotstxt.html
     
  3. NoHoneyWithoutMoney

    NoHoneyWithoutMoney Regular Member

    Joined:
    Dec 5, 2011
    Messages:
    365
    Likes Received:
    82
    Occupation:
    Killer
    Location:
    * Locating ... Server Down 404 *******************
    Last edited: Jan 5, 2012
  4. GangsterProfit

    GangsterProfit Junior Member

    Joined:
    Aug 26, 2011
    Messages:
    193
    Likes Received:
    70
    • Thanks Thanks x 1
  5. NoHoneyWithoutMoney

    NoHoneyWithoutMoney Regular Member

    Joined:
    Dec 5, 2011
    Messages:
    365
    Likes Received:
    82
    Occupation:
    Killer
    Location:
    * Locating ... Server Down 404 *******************