1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How to stop Facebook bot from crawling my site?

Discussion in 'FaceBook' started by usong725, Mar 10, 2014.

  1. usong725

    usong725 Regular Member

    Joined:
    Nov 7, 2011
    Messages:
    234
    Likes Received:
    35
    Location:
    Philippines
    Home Page:
    As the title says.

    The facebook bot is crashing my shared hosting. It is just 10th of the month and my bandwidth allocation is already gone.

    Need help badly. Searched the net and the forum but the results are either very old topics or dont work anymore.

    Thanks in advance.
     
  2. TheeAriGrande

    TheeAriGrande Regular Member

    Joined:
    Jul 14, 2013
    Messages:
    270
    Likes Received:
    151
    Location:
    Candlestick Park
    Try adding this to your .htaccess

    Code:
    
    RewriteCond %{HTTP_USER_AGENT} !facebookexternalhit/[0-9] 
    RewriteRule .* - [R=403,L]
     
    • Thanks Thanks x 1
  3. usong725

    usong725 Regular Member

    Joined:
    Nov 7, 2011
    Messages:
    234
    Likes Received:
    35
    Location:
    Philippines
    Home Page:
    Let me try that.

    Thanks TheeAriGrande.

    edit: I removed the "!" to make it work. No more visit from facebook crawlers since adding it in my .htaccess file.
     
    Last edited: Mar 11, 2014
  4. blueakoi

    blueakoi Registered Member

    Joined:
    Jul 29, 2013
    Messages:
    87
    Likes Received:
    6
    Occupation:
    Full Time Internet Marketer
    Location:
    Good place to live.
    Sir can you or other explain to me? Does this work similar like cloaking?
     
  5. mnpenchev

    mnpenchev Newbie

    Joined:
    Mar 18, 2014
    Messages:
    21
    Likes Received:
    2
    do you have to apply it for new site and if, apply it on current site will you expect improvement of the reach in FB with this ?