dre2027
Newbie
- Jul 21, 2008
- 15
- 3
I've been looking for a good whois scraper/harvester.
For a start, I need something that can check 10k specific domains w/o getting blocked every 8-15 records. Or at least can change its ip address after every 5-10 records pulled.
hidemyass.c0m/vpn/ can change my ip every xx minutes. That's as close as i've come to finding a way to do this. But still, xx minutes isn't good enough. I still get blocked. I need to change more frequently than by xx minutes. I need to change by the # of records or cycle through multiple whois systems(?).
I may be going about this all wrong.
But if not, do you have any idea of how to go about doing whois lookups w/o getting banned/blocked every handful of search results? Or know of an all-in-one bh tool that can do the harvesting for me en masse?
Oh. Eventually I will need to lookup 250k+ specific domains. This initial 10k test group will determine if is worth the effort or not to continue with the idea.
Thanks
For a start, I need something that can check 10k specific domains w/o getting blocked every 8-15 records. Or at least can change its ip address after every 5-10 records pulled.
hidemyass.c0m/vpn/ can change my ip every xx minutes. That's as close as i've come to finding a way to do this. But still, xx minutes isn't good enough. I still get blocked. I need to change more frequently than by xx minutes. I need to change by the # of records or cycle through multiple whois systems(?).
I may be going about this all wrong.
But if not, do you have any idea of how to go about doing whois lookups w/o getting banned/blocked every handful of search results? Or know of an all-in-one bh tool that can do the harvesting for me en masse?
Oh. Eventually I will need to lookup 250k+ specific domains. This initial 10k test group will determine if is worth the effort or not to continue with the idea.
Thanks
