1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

spiders ip list

Discussion in 'Black Hat SEO' started by nickk, Feb 22, 2009.

  1. nickk

    nickk BANNED BANNED

    Joined:
    Feb 1, 2009
    Messages:
    616
    Likes Received:
    262
    can anyone upload or let me know how i can find an updated spider ip list?
    Thank you
     
  2. Grizzy

    Grizzy Senior Member

    Joined:
    Nov 11, 2008
    Messages:
    919
    Likes Received:
    999
    I used to use
    Code:
    hxxp^^w3*iplists*com
    but it looks like it hasn't been updated in a while... I've found it's hard to find and keep good updated lists unless your constantly lurking on webmaster forums.

    I think the problem with spider ip lists, is that the big SEs like G are constantly changing their spider's IP ranges. So it's often better to identify spiders by user agent (this is even what big G recommends).

    For that I have been using
    Code:
    hxxp^^spiderlist*codeforgers*com/list.php
    and
    Code:
    hxxp^^w3*user-agents*org