Discussion in 'Black Hat SEO' started by Holla1, Nov 18, 2014.
does anyone have one of these? i want to keep competitors from viewing site info.
cant be done via robots.txt brother...thats only for the good cops.
You need to learn crash cours on .htaccess tutorial..is only way.
you can implement a whitelist firewall or blacklist block list only through .htaccess.
Mind you if you are using wordpress ..some of the all in one security plug ins have a block by ip list..if you knew whom was crawling or scraping your site..you caould get there ip or ip range and paste it in and they would be 403,ed.
Those files are only SUGGESTIONS to robots. Nothing will prevent a robot to crawl your site if it wants to disregard. Google obeys, and other reputable engines too... but spying sites have little reason to bother honestly identifying themselves or not peeking. IMHO all this does is puts up a red flag for possible blackhat shenanigans on your site. Unless you have an IP you want to block it's a fool's errand.
Case is difficult. Your competitors always find the way. I doubt, that can be done, in order to work.
Separate names with a comma.