skywalker55
Junior Member
- Jan 29, 2010
- 123
- 110
Hi everyone....I could really do with some help...
I have built an app for a client of mine in php which is scraping an api to build a large database. It currently has over 2 million domain records.
Some background to the app....its using cURL, its multi threaded and is running with 50 private proxies.
He now wants a new feature that will get the email contact details from whois and add it to the database.
I have been playing around with scrapebox's whois addon and that works....but problem is that its not done within my php application so although it works as a hack....its not an ongoing solution as the main app is adding new domains all the time.
I want to write some php that will run on a cron job to query whois rather than using scrapebox.
My problem is that I cannot find how to do this. To be honest I dont know much about whois...I cant even work out a single url to scrape the data from....
On certain whois searches I found the email info I need but it appears to be storing the information on the page as an image ???
Any information or help towards a solution would be really appreciates...
I have built an app for a client of mine in php which is scraping an api to build a large database. It currently has over 2 million domain records.
Some background to the app....its using cURL, its multi threaded and is running with 50 private proxies.
He now wants a new feature that will get the email contact details from whois and add it to the database.
I have been playing around with scrapebox's whois addon and that works....but problem is that its not done within my php application so although it works as a hack....its not an ongoing solution as the main app is adding new domains all the time.
I want to write some php that will run on a cron job to query whois rather than using scrapebox.
My problem is that I cannot find how to do this. To be honest I dont know much about whois...I cant even work out a single url to scrape the data from....
On certain whois searches I found the email info I need but it appears to be storing the information on the page as an image ???
Any information or help towards a solution would be really appreciates...