What Is Robots.txt?

Discussion in 'Black Hat SEO' started by Samlko, Mar 6, 2012.

  1. Samlko

    Samlko Newbie

    Joined:
    May 28, 2009
    Messages:
    11
    Likes Received:
    0
    Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
     
  2. Gradimir Stankovic

    Gradimir Stankovic Power Member

    Joined:
    Jan 10, 2010
    Messages:
    737
    Likes Received:
    845
    Location:
    404 not found
    Yes it is? Are you asking question or explaining what robots.txt is?
     
  3. calebmann

    calebmann Power Member Premium Member

    Joined:
    Sep 12, 2010
    Messages:
    541
    Likes Received:
    299
    Occupation:
    FULL TIME INTERNET
    Location:
    Digitopia
    that is exactly what it is! nice job answering your own question buddy!!
     
  4. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    25,565
    Likes Received:
    11,108
    Gender:
    Male
    Occupation:
    PROXIES .
    Location:
    UK - ALWAYS ON BHW
    Home Page:
    techly reported again for replying to threads from 1-5 years ago
     
  5. Sonia Sharma

    Sonia Sharma BANNED BANNED

    Joined:
    Oct 16, 2017
    Messages:
    277
    Likes Received:
    59
    Stupid thread
     
    • Thanks Thanks x 1