1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Amazon is checking every page of my site

Discussion in 'Black Hat SEO' started by testdrive, Nov 27, 2012.

  1. testdrive

    testdrive Regular Member

    Joined:
    Sep 18, 2008
    Messages:
    443
    Likes Received:
    173
    Yep, IP address from amazon and amazon technologies is browsing every page of my site at this moment.. it's been more than 20 minutes now and it's still there.. IP address change every visit but all from amazon.

    It visits every page, comment including sitemap. probably looking for cookie stuff or something, aside from that, I can't think for any reason why they are scanning my site.

    Any thoughts?
     
  2. the_demon

    the_demon Jr. Executive VIP

    Joined:
    Nov 23, 2008
    Messages:
    3,177
    Likes Received:
    1,563
    Occupation:
    Search Engine Marketing
    Location:
    The Internet
    just block their IPs :p

    If they ask you about it, say you detected unusual bot activity and thought a competitor might be trying to scrape your site.
     
    • Thanks Thanks x 2
  3. testdrive

    testdrive Regular Member

    Joined:
    Sep 18, 2008
    Messages:
    443
    Likes Received:
    173
    I dont have the list of their ip range to block.. it seems that they got a massive ip addresses.

    here some:
    184.73.151.118
    107.20.34.112
    54.242.28.17
    23.20.85.53
    23.22.98.198
    23.20.26.105
    23.20.10.165
    23.23.4.209
    107.20.23.163
    107.20.50.83
    54.242.147.12

    anyway, I'll look around for the amazon ip range and decide if I'll block them or not. for the meantime, I'll just wait for the scan to finish.
     
  4. thejake

    thejake Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 13, 2009
    Messages:
    685
    Likes Received:
    828
    Those are EC2 dynamic IPs, they're not usually used by Amazon for anything, just their customers. IP management, SEO analytics companies and others spider sites from EC2 nodes.
     
    • Thanks Thanks x 2
  5. the_demon

    the_demon Jr. Executive VIP

    Joined:
    Nov 23, 2008
    Messages:
    3,177
    Likes Received:
    1,563
    Occupation:
    Search Engine Marketing
    Location:
    The Internet
    • Thanks Thanks x 1
  6. VinBed

    VinBed Regular Member

    Joined:
    Apr 24, 2012
    Messages:
    230
    Likes Received:
    67
    Occupation:
    Bedford Slims Affiliate Network Director
    Home Page:
    Creepy, for sure. I doubt they're what you think they are, though.
     
    • Thanks Thanks x 1
  7. testdrive

    testdrive Regular Member

    Joined:
    Sep 18, 2008
    Messages:
    443
    Likes Received:
    173
    hmmm. my site is down can't access the whole site, just blank white page. it seems like it is an attack or something. I can't even access wp-admin. I emailed my hosting provider to look into it. anything I can do at the moment?
     
  8. ikstob

    ikstob Junior Member

    Joined:
    Nov 12, 2012
    Messages:
    147
    Likes Received:
    129
    Location:
    ikstob.com
    Home Page:
    Last time I checked almost 50% of all Cloud hosting was on the Amazon EC2 platform, so its hardly surprising that some spiders/crawlers will be accessing your website from EC2 IP ranges. If you are nervous about it, use your "robots.txt" site to block certain crawlers - they should abide by the rules - just be sure not to block Google. :)
     
  9. hotnguyen

    hotnguyen Registered Member

    Joined:
    May 17, 2010
    Messages:
    73
    Likes Received:
    16
    I always blocked all IP ranges belong to amazon:
    Code:
    deny from 174.129.0.0/16deny from 122.248.224.0/19deny from 184.72.0.0/15deny from 204.236.128.0/17deny from 50.16.0.0/14deny from 72.44.32.0/19deny from 75.101.128.0/17deny from 176.34.240.0/21deny from 107.20.0.0/14deny from 67.202.0.0/18deny from 23.20.0.0/14deny from 54.240.0.0/12deny from 184.169.128.0/17deny from 50.112.0.0/16
     
  10. ikstob

    ikstob Junior Member

    Joined:
    Nov 12, 2012
    Messages:
    147
    Likes Received:
    129
    Location:
    ikstob.com
    Home Page:
    That's pretty harsh! Plenty of Social / Web 2.0's won't be able to access your sites (e.g. Pinterest) and so will be less likely to trust/accept your content or posts!!
     
  11. hotnguyen

    hotnguyen Registered Member

    Joined:
    May 17, 2010
    Messages:
    73
    Likes Received:
    16
    ^^^ True! But I'm getting tired of those IPs that keep flooding my site, stealing my content, and more... When I blocked 1 single IP, them kept changing to another. So I just simply shut the whole range down for good. Sleep better.
     
  12. TZ2011

    TZ2011 Senior Member

    Joined:
    Jun 26, 2011
    Messages:
    832
    Likes Received:
    863
    Occupation:
    Cleaning servers
    Block them by host, through .htaccess. Simply blocking host amazonaws will block them all. Go to perishablepress or corz.org or just search "how to block bad bots with .htaccess"