1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

What should my .htaccess file look like?

Discussion in 'Black Hat SEO' started by MySeoEmail, Apr 2, 2016.

  1. MySeoEmail

    MySeoEmail Regular Member

    Joined:
    Mar 15, 2015
    Messages:
    225
    Likes Received:
    22
    Occupation:
    Business Owner
    Location:
    Denver, CO
    If i wanted to block crawlers on my PBNs what should my .htaccess file look like? I see so many threads saying diff things just wondering what works for you guys and what it should look like when I edit my .htaccess file?
     
  2. kbklash

    kbklash Senior Member

    Joined:
    Jun 2, 2012
    Messages:
    1,140
    Likes Received:
    449
    Location:
    in my BMW to my BHW
    Home Page:
    RewriteEngine On


    RewriteCond %{HTTP_USER_AGENT} (googlebot|bingbot|Baiduspider) [NC]
    RewriteRule .* - [R=403,L]

    This is what most of us do.You might need to add more crawlers ,if you want.
     
  3. Furious Man

    Furious Man Jr. VIP Jr. VIP

    Joined:
    Aug 4, 2015
    Messages:
    1,571
    Likes Received:
    242
  4. Sristy

    Sristy Jr. VIP Jr. VIP Premium Member

    Joined:
    Aug 17, 2010
    Messages:
    1,800
    Likes Received:
    478
    Gender:
    Female
    Location:
    In My Blog Network
    Home Page:
  5. noobfriendly

    noobfriendly Newbie

    Joined:
    Feb 26, 2016
    Messages:
    29
    Likes Received:
    1
    Isn't the purpose google to index your pbn network? If you block 'googlebot' how google will index it? Or i am missing something.
     
  6. 512 Designs

    512 Designs Junior Member

    Joined:
    Jun 10, 2015
    Messages:
    134
    Likes Received:
    9
    Gender:
    Female
    its right i think RewriteCond %{HTTP_USER_AGENT} (googlebot|bingbot|Baiduspider) [NC] RewriteRule .* - [R=403,L]
     
  7. Ste Fishkin

    Ste Fishkin Jr. VIP Jr. VIP Premium Member

    Joined:
    May 14, 2011
    Messages:
    2,047
    Likes Received:
    10,422
    Mostly white, with some black characters on it.

    Depends what program you open it in.
     
  8. agent-x

    agent-x Junior Member

    Joined:
    Apr 4, 2012
    Messages:
    173
    Likes Received:
    34
    Gender:
    Male
    Try that and let us know how it works out for you!