530inc
Regular Member
- Dec 18, 2013
- 201
- 77
For the last couple of months, I'm spending a lot more time that I would have wanted on a client's site, and this because of a nasty bounce rate attack that no matter what I do won't subside. And by subside I mean blocking the sources of bounce so that the bounce rate gets to where it previously was. Problem is that I get tons of 100% bounce attacks from direct, refferal and organic sources - and these mofos make rankings fluctuate a bit too much for my taste - aka I dropped from page #1 on some positions and that makes me mad. Oh, and it's funny since tons of people, incl Google say that they don't take under account analytics data. True, because they get their data no matter if you have analytics or not on the site. So, don't believe everything you read
What I did so far it can be considered by most insane work, but here's what I ended up doing.
Step 1.
Download site logs
Step 2.
Download GA data report.
I download a custom report in the following format: Audiences - Network - Secondary dimension "hourofday".
This allows me to better pin-point single 100% bounce visits duringa a 1h timeframe.
Step3.
Merge data in excel to find which IPs caused the bounce rate (shit ton of editing, text to columns, find ISP ASN from GA and IP ASN from Logs, v-lookup's and such; now I loathe the fact that I gave up programming as some VBA would sure come in handy).
Step4.
Block IPs that give 100% bounce.
Nevertheless, even if I have blocked a metric ton of IPs I still get a 40-50% bounce rate (initial bounce rate before the attack was 10%). If I apply a filter in GA to view only non-bounce visits bounce rate is normal.
To protect the site (wordpress site) from attacks, I used the following with questionable success:
Sucuri - just in case
Couldflare - but the firewall was blocking all traffic even from whitelisted IPs and countries so I gave that UP; additionally, I suspected that due to the CDN GA code was loading even if the user would see a 403 page after parsing (but i might just be crazy)
IpGEOblock - works well blocking most of the countries that the toughest bounces came from, works well blocking IPs too, but doesn't block all - still using it
PVB (aka Proxy and VPN blocker) - which works with proxycheck.io API (paid for the 10k queries per day) but questionable results - most notably due to the API
Of course, .htaccess Order allow,deny, deny all ips that create bounce + block SB proxies from daily scrape. I stopped using htaccess direct block for the last week as the server response time was getting over 2s, but still, I saw no difference in response time as the list of IPs to be parsed is in IpGeoBlock so it's the same thing apparently. So I will start with it again, even if right now there's 50k proxies and vpns blocked and the list grows daily by anywhere from 100 to 2k IPs to be blocked.
In htaccess I also tried the following with no visible improvement:
<ifModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP:VIA} !^$ [OR]
RewriteCond %{HTTP:FORWARDED} !^$ [OR]
RewriteCond %{HTTP:USERAGENT_VIA} !^$ [OR]
RewriteCond %{HTTP:X_FORWARDED_FOR} !^$ [OR]
RewriteCond %{HTTP
ROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:XPROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:HTTP_PC_REMOTE_ADDR} !^$ [OR]
RewriteCond %{HTTP:HTTP_CLIENT_IP} !^$
RewriteRule .* - [F]
</ifModule>
The above should block all proxies (aka the ones that meet the above criteria) but that doesn't work. Plus tends to also block some of Google's services (lighthouse one of them) in the process which is bad in my case as we're talking about a money site. I tried to add an exception with a new rewritecond rule, but aparently lighthouse uses some shady connections when analysing the page speed, so, just to be sure that the same didn't happen for other google services, I gave up using this method.
Additionally, I also used the "block all bots except google" piece of code from @MatthewWoodward + list of bad bots found online so basically, on the site only Googlebot enters and aparently, shit ton of proxies and VPNs too.
How my day started to look like (and it's driving me insane as there are no visible results):
- wake up, light up a cigarrette, put a cup of coffe and get down in front of the computer
- download logs from the previous day
- download analytics data from the previous day
- bang your head against the desk, smoke 5 cigs
- merge data together and make the list of ips that give 100% bounce
- bang your head against the desk, smoke 10 cigs in doing the above as it's tedious work
- check ips from logs against proxycheck.io and see which ones I missed (aka additional proxies and vpns)
- open VPS, run a Scrapebox Proxy harvesting session, test proxies, download proxies that passted the anonimity test, add them to list to be blocked that day, add them in proxycheck.io blacklist, add them to ipgeoblock blaclist
- bang your head against the desk, smoke a couple of cigs
- sit back thinking you did good, check a few hours later to see the results
- see 40%-50% bounce rate, bang you head against the wall, smoke a couple of cigs.
Now, in case somebody is as insane as I am, I can share the following:
- my list of steps to make the link between analytics data and IPs (incl all the functions used in Excel) - if there is somebody that knows how to code and is willing to share with the community i think it would be a great addition
- macro to merge multiple excels into one (found online, works great)
- macro to merge multiple sheets into one (useful when you check IP geo location needed for ISP name and ASN, also found online, works great)
- list of differences between ISP name from https://www.infobyip.com/ipbulklookup.php and Analytics ISP name (i spit on analytics and their isp names
)
- my custom scrapebox harvesting sources
So, any ideas?
I suspect the attacker uses BearPro or a similar bot, which heavily relies on proxies. The 100% bounce rate hourly reccurance is 3-4 visits per hour.
I might be checking the Log IPs vs Analytics wrong (currenly I do a merge of ASN + DateAndHourOfDay columns into one and vlookup against analytics data prepared into the same format). So basically I do not check by ISP/ASN and hour in separate columns as Vlookup returns the first value found, and I'd miss some visits. Merging two cells into one creates a custom unique strings containing hour and ASN and I only do a Vlookup to check in the Analytics data (which I would have already prepped into a same format). Of course, I adjusted date and time from analytics to date and time from logs so that there are no discrepancies.
Still, with the above I noticed something strange. I get IPs that give a 403 error (aka they were blocked by htaccess/ipgeoblock/pvb) that show up in the analytics bounce rate report. My 403 page does not load the freakin' analytics code! So, basically, ips blocked with 403 error still show up in analytics as 0 time spent and 100% bounce rate!
What I suspect:
Attacker changes proxies or vpns several times per day which makes it hard to block.
If I would get rid of all defenses, probably I'd end up with 100% bounce or a very nice DDoS attack.
I'm considering checking all proxy and VPN IPs that I found that were not in my scrapebox lists and send and abuse email to ISP but I doubt that would help since this is no DDoS Attack.
What would you do?
Any tools (free or paid) you know that I can use? Any suggestions?
Feel like i'm losing my freaking mind here, so any help or suggestion would be appreciated.
What I did so far it can be considered by most insane work, but here's what I ended up doing.
Step 1.
Download site logs
Step 2.
Download GA data report.
I download a custom report in the following format: Audiences - Network - Secondary dimension "hourofday".
This allows me to better pin-point single 100% bounce visits duringa a 1h timeframe.
Step3.
Merge data in excel to find which IPs caused the bounce rate (shit ton of editing, text to columns, find ISP ASN from GA and IP ASN from Logs, v-lookup's and such; now I loathe the fact that I gave up programming as some VBA would sure come in handy).
Step4.
Block IPs that give 100% bounce.
Nevertheless, even if I have blocked a metric ton of IPs I still get a 40-50% bounce rate (initial bounce rate before the attack was 10%). If I apply a filter in GA to view only non-bounce visits bounce rate is normal.
To protect the site (wordpress site) from attacks, I used the following with questionable success:
Sucuri - just in case
Couldflare - but the firewall was blocking all traffic even from whitelisted IPs and countries so I gave that UP; additionally, I suspected that due to the CDN GA code was loading even if the user would see a 403 page after parsing (but i might just be crazy)
IpGEOblock - works well blocking most of the countries that the toughest bounces came from, works well blocking IPs too, but doesn't block all - still using it
PVB (aka Proxy and VPN blocker) - which works with proxycheck.io API (paid for the 10k queries per day) but questionable results - most notably due to the API
Of course, .htaccess Order allow,deny, deny all ips that create bounce + block SB proxies from daily scrape. I stopped using htaccess direct block for the last week as the server response time was getting over 2s, but still, I saw no difference in response time as the list of IPs to be parsed is in IpGeoBlock so it's the same thing apparently. So I will start with it again, even if right now there's 50k proxies and vpns blocked and the list grows daily by anywhere from 100 to 2k IPs to be blocked.
In htaccess I also tried the following with no visible improvement:
<ifModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP:VIA} !^$ [OR]
RewriteCond %{HTTP:FORWARDED} !^$ [OR]
RewriteCond %{HTTP:USERAGENT_VIA} !^$ [OR]
RewriteCond %{HTTP:X_FORWARDED_FOR} !^$ [OR]
RewriteCond %{HTTP
RewriteCond %{HTTP:XPROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:HTTP_PC_REMOTE_ADDR} !^$ [OR]
RewriteCond %{HTTP:HTTP_CLIENT_IP} !^$
RewriteRule .* - [F]
</ifModule>
The above should block all proxies (aka the ones that meet the above criteria) but that doesn't work. Plus tends to also block some of Google's services (lighthouse one of them) in the process which is bad in my case as we're talking about a money site. I tried to add an exception with a new rewritecond rule, but aparently lighthouse uses some shady connections when analysing the page speed, so, just to be sure that the same didn't happen for other google services, I gave up using this method.
Additionally, I also used the "block all bots except google" piece of code from @MatthewWoodward + list of bad bots found online so basically, on the site only Googlebot enters and aparently, shit ton of proxies and VPNs too.
How my day started to look like (and it's driving me insane as there are no visible results):
- wake up, light up a cigarrette, put a cup of coffe and get down in front of the computer
- download logs from the previous day
- download analytics data from the previous day
- bang your head against the desk, smoke 5 cigs
- merge data together and make the list of ips that give 100% bounce
- bang your head against the desk, smoke 10 cigs in doing the above as it's tedious work
- check ips from logs against proxycheck.io and see which ones I missed (aka additional proxies and vpns)
- open VPS, run a Scrapebox Proxy harvesting session, test proxies, download proxies that passted the anonimity test, add them to list to be blocked that day, add them in proxycheck.io blacklist, add them to ipgeoblock blaclist
- bang your head against the desk, smoke a couple of cigs
- sit back thinking you did good, check a few hours later to see the results
- see 40%-50% bounce rate, bang you head against the wall, smoke a couple of cigs.
Now, in case somebody is as insane as I am, I can share the following:
- my list of steps to make the link between analytics data and IPs (incl all the functions used in Excel) - if there is somebody that knows how to code and is willing to share with the community i think it would be a great addition
- macro to merge multiple excels into one (found online, works great)
- macro to merge multiple sheets into one (useful when you check IP geo location needed for ISP name and ASN, also found online, works great)
- list of differences between ISP name from https://www.infobyip.com/ipbulklookup.php and Analytics ISP name (i spit on analytics and their isp names
- my custom scrapebox harvesting sources
So, any ideas?
I suspect the attacker uses BearPro or a similar bot, which heavily relies on proxies. The 100% bounce rate hourly reccurance is 3-4 visits per hour.
I might be checking the Log IPs vs Analytics wrong (currenly I do a merge of ASN + DateAndHourOfDay columns into one and vlookup against analytics data prepared into the same format). So basically I do not check by ISP/ASN and hour in separate columns as Vlookup returns the first value found, and I'd miss some visits. Merging two cells into one creates a custom unique strings containing hour and ASN and I only do a Vlookup to check in the Analytics data (which I would have already prepped into a same format). Of course, I adjusted date and time from analytics to date and time from logs so that there are no discrepancies.
Still, with the above I noticed something strange. I get IPs that give a 403 error (aka they were blocked by htaccess/ipgeoblock/pvb) that show up in the analytics bounce rate report. My 403 page does not load the freakin' analytics code! So, basically, ips blocked with 403 error still show up in analytics as 0 time spent and 100% bounce rate!
What I suspect:
Attacker changes proxies or vpns several times per day which makes it hard to block.
If I would get rid of all defenses, probably I'd end up with 100% bounce or a very nice DDoS attack.
I'm considering checking all proxy and VPN IPs that I found that were not in my scrapebox lists and send and abuse email to ISP but I doubt that would help since this is no DDoS Attack.
What would you do?
Any tools (free or paid) you know that I can use? Any suggestions?
Feel like i'm losing my freaking mind here, so any help or suggestion would be appreciated.