1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Question: So where do I put r0b0t.txt to avoid blackhatters?

Discussion in 'BlackHat Lounge' started by loophole, Feb 28, 2009.

  1. loophole

    loophole Registered Member

    Joined:
    Apr 7, 2008
    Messages:
    91
    Likes Received:
    82
    Hello blackhatters..

    So as we all know there's this trick of checking domain.com/r0b0ts.txt for hidden pages ... like product download pages.

    So where can I place the r0bots.txt file such that it can be found by search engines but NOT by blackhatters :)

    Or alternatively is there a way to block search engines from indexing certain pages without using the r0b0ts.txt file?

    Cheers
     
    Last edited: Feb 28, 2009
  2. mr4army

    mr4army Regular Member

    Joined:
    Nov 23, 2008
    Messages:
    459
    Likes Received:
    80
    Home Page:
    just put it in the normaly place and password protect it simples
     
    • Thanks Thanks x 1