1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Where can I get all IP ranges of G,Bing,Yahoo?I want to disable all engines except these

Discussion in 'Black Hat SEO' started by 123again, Jul 3, 2012.

  1. 123again

    123again Registered Member

    Joined:
    Apr 13, 2010
    Messages:
    64
    Likes Received:
    7
    I am thinking the possibility of blocking all robots engines except GG,Bing and Yahoo, because it really waste too many server sources for some bad bots. and it is not good idea to just disable from robots.txt because the bad bots ignore this file.

    Anyone can tell me the IP ranges for these three? and if possible with existing script doing this job would be more helpful.
     
  2. HatIsBlacked

    HatIsBlacked Regular Member

    Joined:
    Dec 30, 2010
    Messages:
    224
    Likes Received:
    57
    what does buy you? so what if you know the top search engines blocks of ip addresses.

    how are you going to determine real visitors?

    you need the ip addresses of the search engines you want to deny access.
    for starters, use robots.txt and if they continue to crawl, look at your logs and deny by ip address
     
  3. 123again

    123again Registered Member

    Joined:
    Apr 13, 2010
    Messages:
    64
    Likes Received:
    7
    As I said above, the robots.txt is only useful to such as g,bing,yahoo, it is no use for those bad bots,they don't care what you write on robots.txt.

    I think is not hard to identify real visitors, for example, real visitor will visit javascript codes but bots not. Of course it is little hard for me to finish it so it would be better if any existing scripts there.