Amazon is checking every page of my site

T

testdrive

Regular Member
Joined
Sep 18, 2008
Messages
444
Reaction score
174
Yep, IP address from amazon and amazon technologies is browsing every page of my site at this moment.. it's been more than 20 minutes now and it's still there.. IP address change every visit but all from amazon.

It visits every page, comment including sitemap. probably looking for cookie stuff or something, aside from that, I can't think for any reason why they are scanning my site.

Any thoughts?
 
Advertise on BHW
just block their IPs :P

If they ask you about it, say you detected unusual bot activity and thought a competitor might be trying to scrape your site.
 
I dont have the list of their ip range to block.. it seems that they got a massive ip addresses.

here some:
184.73.151.118
107.20.34.112
54.242.28.17
23.20.85.53
23.22.98.198
23.20.26.105
23.20.10.165
23.23.4.209
107.20.23.163
107.20.50.83
54.242.147.12

anyway, I'll look around for the amazon ip range and decide if I'll block them or not. for the meantime, I'll just wait for the scan to finish.
 
Those are EC2 dynamic IPs, they're not usually used by Amazon for anything, just their customers. IP management, SEO analytics companies and others spider sites from EC2 nodes.
 
Use this service (free trial) http://www.distil.it/ or (free plan available) https://www.cloudflare.com/ you can set page view limits and bot detection. You'll be able to bring them to a grinding halt within 1hr. :P
 
hmmm. my site is down can't access the whole site, just blank white page. it seems like it is an attack or something. I can't even access wp-admin. I emailed my hosting provider to look into it. anything I can do at the moment?
 
Last time I checked almost 50% of all Cloud hosting was on the Amazon EC2 platform, so its hardly surprising that some spiders/crawlers will be accessing your website from EC2 IP ranges. If you are nervous about it, use your "robots.txt" site to block certain crawlers - they should abide by the rules - just be sure not to block Google. :)
 
I always blocked all IP ranges belong to amazon:
Code: 
deny from 174.129.0.0/16deny from 122.248.224.0/19deny from 184.72.0.0/15deny from 204.236.128.0/17deny from 50.16.0.0/14deny from 72.44.32.0/19deny from 75.101.128.0/17deny from 176.34.240.0/21deny from 107.20.0.0/14deny from 67.202.0.0/18deny from 23.20.0.0/14deny from 54.240.0.0/12deny from 184.169.128.0/17deny from 50.112.0.0/16
 
hotnguyen said:
I always blocked all IP ranges belong to amazon:
Code: 
deny from 174.129.0.0/16deny from 122.248.224.0/19deny from 184.72.0.0/15deny from 204.236.128.0/17deny from 50.16.0.0/14deny from 72.44.32.0/19deny from 75.101.128.0/17deny from 176.34.240.0/21deny from 107.20.0.0/14deny from 67.202.0.0/18deny from 23.20.0.0/14deny from 54.240.0.0/12deny from 184.169.128.0/17deny from 50.112.0.0/16
Click to expand...

That's pretty harsh! Plenty of Social / Web 2.0's won't be able to access your sites (e.g. Pinterest) and so will be less likely to trust/accept your content or posts!!
 
^^^ True! But I'm getting tired of those IPs that keep flooding my site, stealing my content, and more... When I blocked 1 single IP, them kept changing to another. So I just simply shut the whole range down for good. Sleep better.
 
Block them by host, through .htaccess. Simply blocking host amazonaws will block them all. Go to perishablepress or corz.org or just search "how to block bad bots with .htaccess"
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top