question to those with private networks

highl4nder · Oct 15, 2012

Do you block robots/crawlers/spiders from sites like OpenSiteExplorer, MajesticSEO, Ahrefs, etc. in your robots.txt? Part of me thinks that I should so that my competitors can't spy on me and report me. But I also think that by blocking these spiders I'm telling Google that I am up to something.

Thoughts please?

MaxTucker · Oct 15, 2012

I don't block them. If somebody really wants to find the information they will do so regardless of the robots you block.

RankGeek · Oct 15, 2012

Hi Swi7ch,

That is precisely what Google would not want you to do. Blocking any identifiable information only serves to tell search engines that you have something to hide! And competition spying is just a side effect of that - you will obviously do that to your competitors anyway, right?

Go clean mate, go long term!

formosa · Oct 15, 2012

thats a nice thought ... but wonder if there is any side effect

bargz · Oct 15, 2012

Cant hurt you really... i wouldnt worry about it

highl4nder · Oct 15, 2012

formosa said:
thats a nice thought ... but wonder if there is any side effect

The side effect is that you're hiding something... why is that? Remember that with private networks, you want to make them to look like as legit as possible. But if you block spiders from MajesticSEO, Ahrefs, etc. then you're doing something out of the ordinary and that could cause some concern (instead of avoiding them).

question to those with private networks

highl4nder

Power Member

MaxTucker

BANNED

RankGeek

Senior Member

formosa

Regular Member

bargz

Newbie

highl4nder

Power Member

Main Menu

Marketplace

Making Money

BlackHat World