How do I block ahrefs from visiting a forum?

[RuthSam]

Does anyone know how I can block all Ahrefs crawlers to visiting my clients forum?

I know how to use htaccess, I just need to know what I need to blog to be 99% sure!

 

[ShahidMian]

I am not sure why you want this?
Can you explain your question a bit so can be ans easily??

 

[RuthSam]

Well, your new to BHW, welcome to this awesome forum. Read a little more around and you will know it

 

[laur.laurix]

The safest way is the htaccess way. After intensive testing, u know hidding PBNs, i concluded that blocking ahrefs with htaccess is the best way.

 

[moanon]

just add rel="nofollow" to your links.
Or set up a redirect page on your site like /nofollowlink?url=[the_link_url], change the links and block /nofollowlink in your robots.txt

just google "nofollow" and you'll get plenty of information on that topic

 

[Jobdollarr]

Ip blocking them. I do not know how. But there must be a way for it

 

[laur.laurix]

) shut up Meg. Op is talking about ahrefs not G.

just add rel="nofollow" to your links.
Or set up a redirect page on your site like /nofollowlink?url=[the_link_url], change the links and block /nofollowlink in your robots.txt

just google "nofollow" and you'll get plenty of information on that topic

 

[RuthSam]

Did some research and found this thread http://www.blackhatworld.com/blackhat-seo/black-hat-seo/584470-anybody-code-block-majestic-ahrefs-all-others-crawling-site.html it explains it all.. sorry to ask again.

 

[moanon]

) shut up Meg. Op is talking about ahrefs not G.

I guess Ahrefs uses an web crawler just like google and should follow the robots.txt rules when crawling the site.. but yeah could be that I misunterstood the question.

By the way neither is my name Meg nor do I like to be told to shut up but I just guess it's an reference to Family Guy.. yeah good one.. because both names start with M or what? I don't really get it but I'm sure people not that stupid like me or Meg will get it

Sorry if I misunderstood the question, my answer was useless and I wasted your time. Just wanted to help..

 

[RuthSam]

I guess Ahrefs uses an web crawler just like google and should follow the robots.txt rules when crawling the site.. but yeah could be that I misunterstood the question.

By the way neither is my name Meg nor do I like to be told to shut up but I just guess it's an reference to Family Guy.. yeah good one.. because both names start with M or what? I don't really get it but I'm sure people not that stupid like me or Meg will get it

Sorry if I misunderstood the question, my answer was useless and I wasted your time. Just wanted to help..

:bsflag:you're trolling and don't even read the thread!

 

[moanon]

:bsflag:you're trolling and don't even read the thread!

Ok sorry if you think that.. Just saying that if you make a redirect page and block it with robots.txt it should have the effect you wanted because web crawlers (google ahrefs majestics whatever) should follow the rules in robots.txt and if you just link to "/nofollowlink?url=" ( [url] maybe encoded ) there is ...r time (and mine even more, genius) whatever

 

[watsonovedades]

Check this out
maybe helpful
http://spyderspanker.com/

cheers

 

[sergey007]

You can either block bots using .htacess (more info here: http://www.blackhatworld.com/blackh...seo-opensiteexplorer-other-backlink-bots.html) or, as watsonovedades recommended, use Spyder Spanker plugin. personally, I have been using both ways to block majesticseo and ahrefs bots.

 

[chickensalad]

Ahrefs are persistent - their bread and butter is crawling sites. I've never found a simple htaccess block ahrefs ua pls thx to work - they're possibly renaming their bot/IP ranges and sending them back out.

Haven't heard of SpyderSpanker - but if they're constantly updating banned IP ranges, it's worth checking out.

 

[rogerke]

Ahrefs are persistent - their bread and butter is crawling sites. I've never found a simple htaccess block ahrefs ua pls thx to work - they're possibly renaming their bot/IP ranges and sending them back out.

No, they aren't ;-)

A simple .htaccess block/redirect code including ahrefsbot works just fine. It might be that you're using CCarter's code and although he's a straight genius, it's a wrong code for this purpose because it only blocks them from the homepage.

 

[chickensalad]

No, they aren't ;-)

A simple .htaccess block/redirect code including ahrefsbot works just fine. It might be that you're using CCarter's code and although he's a straight genius, it's a wrong code for this purpose because it only blocks them from the homepage.

Lol, no way. What's the correct one?

 

[darkoltree]

:bsflag:you're trolling and don't even read the thread!

I know right. A lot of people these days commenting on the forums don't. You can see it for yourself. I'm disgusted.

 

[RuthSam]

No, they aren't ;-)

A simple .htaccess block/redirect code including ahrefsbot works just fine. It might be that you're using CCarter's code and although he's a straight genius, it's a wrong code for this purpose because it only blocks them from the homepage.

Yeah, what is the correct code please?

 

[Silverbak]

Jesus H. Christ, is this turning into the SEO section on Warrior Forum (where the blind leads the blind)? He's asking about blocking on a FORUM. So unless Ruth is running BBPress, SpyderSpanker aint gonna work cause it's a WORDPRESS plugin.

 

[RuthSam]

It's a Vbulletin Forum ! sp there is no wordpress or anything like that