Discussion in 'Cloaking and Content Generators' started by huzah, Apr 19, 2012.
As long as there is data, scraping is profitable if you can get creative with it. The power is in the ideas. For example, look at SpyFu. They built a huge business upon screen scraping google results.
You can PM me if you want to talk about scraping, because I'm currently undertaking quite a few scraping projects.
That tool looks great, how do you integrate the proxies? Are you using any special hosting? Or just a shared one?
Well the website runs on a dedicated server, but that's not really important. The proxy is *selected* from a mysql database, based on how often it has been used (I allow a maximum usage of 25 pages/proxy/hour/website but also no more than 4 pages/proxy/minute/website). It is then sent along with the scraping request to the C# Master server, which then selects a child server for the actual scraping. It is inside that C# Child server that the proxy is actually used. They're just squid proxies running on cheap VPS's.
By the way, it's funny how many of the test searches are sex related, lmao. (Don't worry, no IP's are being logged).
what kind of program do you plan on writing for promoting the content !?
sex is the only thing on the internet isnt it?
You're probably a couple of years too late - would have been good commercially as a wordpress autoblog plugin
I'd look more to "data mining" - e.g. finding stuff for researchers e.g. case law for lawyers, medical research for doctors, etc. not as something for webmasters
Separate names with a comma.