1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

question to those with private networks

Discussion in 'Black Hat SEO' started by swi7ch, Oct 15, 2012.

  1. swi7ch

    swi7ch Power Member

    Joined:
    May 13, 2012
    Messages:
    528
    Likes Received:
    144
    Do you block robots/crawlers/spiders from sites like OpenSiteExplorer, MajesticSEO, Ahrefs, etc. in your robots.txt? Part of me thinks that I should so that my competitors can't spy on me and report me. But I also think that by blocking these spiders I'm telling Google that I am up to something.

    Thoughts please?
     
  2. MaxTucker

    MaxTucker Regular Member

    Joined:
    Aug 22, 2010
    Messages:
    322
    Likes Received:
    192
    Occupation:
    Run authority sites.
    Location:
    UK
    I don't block them. If somebody really wants to find the information they will do so regardless of the robots you block.
     
  3. RankGeek

    RankGeek Jr. VIP Jr. VIP

    Joined:
    Jul 11, 2012
    Messages:
    958
    Likes Received:
    124
    Occupation:
    Entrepreneur
    Location:
    Near the Sea
    Home Page:
    Hi Swi7ch,

    That is precisely what Google would not want you to do. Blocking any identifiable information only serves to tell search engines that you have something to hide! And competition spying is just a side effect of that - you will obviously do that to your competitors anyway, right?

    Go clean mate, go long term!
     
  4. formosa

    formosa Regular Member

    Joined:
    Apr 30, 2008
    Messages:
    309
    Likes Received:
    27
    Home Page:
    thats a nice thought ... but wonder if there is any side effect
     
  5. bargz

    bargz Newbie

    Joined:
    Jul 15, 2012
    Messages:
    15
    Likes Received:
    2
    Cant hurt you really... i wouldnt worry about it
     
  6. swi7ch

    swi7ch Power Member

    Joined:
    May 13, 2012
    Messages:
    528
    Likes Received:
    144
    The side effect is that you're hiding something... why is that? Remember that with private networks, you want to make them to look like as legit as possible. But if you block spiders from MajesticSEO, Ahrefs, etc. then you're doing something out of the ordinary and that could cause some concern (instead of avoiding them).