Block SEMrush

choppa2choppa

Registered Member
Joined
Feb 22, 2016
Messages
96
Reaction score
15
I am currently adding some adwords ad but I know that SEMrush will find them soon and allow competitors to copy my ads and keyword etc. Is there a way to block SEMrush from spying all my ads out and allowing competitors to copy?

Also does the SEMrush bot go on to websites and follow the backlink to the ad or does the bot monitor the ads that show up on google search?
 
Check here on how to block the semrush bot: https://www.semrush.com/bot/

If you're not confident with that you can block it with .htaccess file:

BrowserMatchNoCase SemrushBot bad_bot
Order Deny,Allow
Deny from env=bad_bot

An other option is to block it with PHP:
PHP:
<?php
if(preg_match('/SemrushBot/i',$_SERVER['HTTP_USER_AGENT']))
    {
    header('HTTP/1.0 403 Forbidden');
    echo "Yippie ka yee motherf#cker";
    exit();
    }
?>
 
I know you can block semrush bot with a plugin called linkprivacy. All it does is stop tools from crawling your website. Not sure if this will help you with adwords or not.
 
Thanks for the help guys I have blocked it using robots.txt which looks like this below would this be ok?

User-Agent: *

User-agent: SemrushBot
Disallow: /
 
I know you can block semrush bot with a plugin called linkprivacy. All it does is stop tools from crawling your website. Not sure if this will help you with adwords or not.

It won't, and this issue is a huge one for those of us in the lead generation industry.

To be honest, if you're using Adwords right now and are looking to stop competitors clicking adverts and discovering how you earn your money follow these rules:

1. Block your site completely from Google by using robots.txt + meta tags - This is especially true if certain pages / text on your site could identify what affiliate / lead gen network you're using like custom privacy pages or terms and conditions that contain certain statements required by the merchant.

2. Try to steer clear of head terms - Head terms convert terribly and are typically the first port of call for click happy newbies and "SEO professionals" looking to scope the competition.

3. Use a separate website for testing - Your testing website will almost always be bidding on broad match keywords, hence it's often the one that will be picked up by SEO experts, brand managers and other affiliates. It's also the one people will probably test Spyfu against to try and figure out how your getting your conversions. By only testing using this domain you'll always be throwing people off the scent by bidding on useless keywords. When you've found a profitable subset of keywords, add them to your negative list for the testing site and create a new campaign for your main conversion site - and always try to either use phrase match, exact match or try to bid just enough on a broad match to make a profit. You want to minimize your exposure to having the main sales website discovered, especially if you've worked hard to find the perfect design to promote your conversion rate.

4. If you're in a really competitive niche, switch up the sales site frequently - This is especially true to certain financial and compensation niches, competitors especially other affiliates will use Spyfu to try and figure out what keywords you're bidding on. By changing your sites every 3 months you'll keep them off the scent.

5. Private whois - Always use a private whois. Competitors will try to look for the links between brands, websites and company names.
 
Last edited:
Those robot files are only a REQUEST to not be crawled by a bot. If SemRush wanted to ignore that request and crawl it anyway it could. And they could also run, un-branded bots. They may not do this... but I don't see why not.
 
I have used robots and going to check in 2-3 weeks to see if the site is still listed in SEMrush. Also i'm guessing they would need to play by the rules if they have posted this https://www.semrush.com/bot/ if they ignore this would this be bordering illegal for a big legitimate company like them to be ignoring the bot disallow?
 
Hello,
I have a question, in php code what is correct?
1) if(preg_match('/Semrush/is
2) if(preg_match('/Semrush/i

is with "i" only or be with "is"
because in this link said be only "i" .

PLEASE CONFIRM.
 
Even if you manage to block SEMRush, what will you do about something like SpyFu? It takes screenshots of the display ads and shit.
 
Back
Top