How to stop Facebook bot from crawling my site?

usong725 · Mar 10, 2014

As the title says.

The facebook bot is crashing my shared hosting. It is just 10th of the month and my bandwidth allocation is already gone.

Need help badly. Searched the net and the forum but the results are either very old topics or dont work anymore.

Thanks in advance.

TheeAriGrande · Mar 10, 2014

Try adding this to your .htaccess

Code:

RewriteCond %{HTTP_USER_AGENT} !facebookexternalhit/[0-9] 
RewriteRule .* - [R=403,L]

usong725 · Mar 10, 2014

Let me try that.

Thanks TheeAriGrande.

edit: I removed the "!" to make it work. No more visit from facebook crawlers since adding it in my .htaccess file.

blueakoi · Mar 20, 2014

TheeAriGrande said:
Try adding this to your .htaccess

Code:

RewriteCond %{HTTP_USER_AGENT} !facebookexternalhit/[0-9] RewriteRule .* - [R=403,L]

Sir can you or other explain to me? Does this work similar like cloaking?

mnpenchev · Mar 20, 2014

do you have to apply it for new site and if, apply it on current site will you expect improvement of the reach in FB with this ?

How to stop Facebook bot from crawling my site?

usong725

Regular Member

TheeAriGrande

Regular Member

usong725

Regular Member

blueakoi

Junior Member

mnpenchev

Newbie

Main Menu

Marketplace

Making Money

BlackHat World