spiders ip list

Discussion in 'Black Hat SEO' started by nickk, Feb 22, 2009.

  1. nickk

    nickk BANNED BANNED

    Joined:
    Feb 1, 2009
    Messages:
    616
    Likes Received:
    262
    can anyone upload or let me know how i can find an updated spider ip list?
    Thank you
     
  2. Grizzy

    Grizzy Senior Member

    Joined:
    Nov 11, 2008
    Messages:
    919
    Likes Received:
    1,001
    I used to use
    Code:
    hxxp^^w3*iplists*com
    but it looks like it hasn't been updated in a while... I've found it's hard to find and keep good updated lists unless your constantly lurking on webmaster forums.

    I think the problem with spider ip lists, is that the big SEs like G are constantly changing their spider's IP ranges. So it's often better to identify spiders by user agent (this is even what big G recommends).

    For that I have been using
    Code:
    hxxp^^spiderlist*codeforgers*com/list.php
    and
    Code:
    hxxp^^w3*user-agents*org