bgallotte
Junior Member
- Jul 29, 2009
- 101
- 42
I am starting to build some autoblogs and I am a bit concerned about how to protect myself from big G. crawls and robots.. I know this crawls and robots check everything in your page, including your plugins subdirectory.
Whats the best way to protect myself from this check? Because once the spider get to my plugins they will know through the names that it is an autoblog...
Does robots.txt works? I mean, will big G. respect it? or I should change each plugins subdirectories name and respectives php files?
I dont know if I am being paranoid just would like some suggestions pls.
Whats the best way to protect myself from this check? Because once the spider get to my plugins they will know through the names that it is an autoblog...
Does robots.txt works? I mean, will big G. respect it? or I should change each plugins subdirectories name and respectives php files?
I dont know if I am being paranoid just would like some suggestions pls.