robots.txt

Discussion in 'Black Hat SEO' started by theseodude, Aug 9, 2012.

  1. theseodude

    theseodude Regular Member

    Joined:
    Jun 25, 2012
    Messages:
    303
    Likes Received:
    88
    Hi guys
    I am new to this concept of robots.txt
    I dont want search engines to visits one of the pages on my site. let's say this page is http:/www.mysite.com/foo.html

    I need to make sure the following is correct:
    Code:
    User-agent: *
    Disallow: /foo.html
    
    Is this code correct?

    So I just copy/paste the above 2 lines into a text file and save the text file, naming it robots?
    and then upload it to public_html?
     
    Last edited: Aug 9, 2012
  2. pallyD

    pallyD Junior Member

    Joined:
    Aug 6, 2010
    Messages:
    137
    Likes Received:
    95
    Location:
    MW
    That's it in a nutshell.


    Sent from my YP-G1