dannyhw
Senior Member
- Jul 16, 2008
- 979
- 476
Even after an enormous amount of queries?
Google Keyword Tool Data Scraped - how to market this?
- Thread starter rnc505
- Start date
Even after an enormous amount of queries?
how to buy this Date?
BlackHatSoda
Regular Member
- Oct 6, 2009
- 393
- 329
It's not in a form designed for retail. The interface is minimal and it requires SQL Server for the database.
I toyed with the idea of offering it as a hosted service to a limited set of people but in other forums where I asked people if they would be interested in something like this, I didn't get much of a response.
It would be a lot of work to make it into a service others could use and would require dedicated hosting, maintenance and customer support so unless I had enough interest, it wouldn't be worth the effort.
Also, Google changed their keyword tool a couple of months ago so that CPC data is no longer shown unless you log in.
I modified my tool to log into an AdWords account and scrape with that but you simply can't get the same volume of queries that way. The only way now is to create a bunch of AdWords accounts that you rotate through via proxies.
Or you can pay the API fees which gets very expensive at the keyword volumes that I use.
phpbuilt
Account compromised December 2025
- May 16, 2011
- 1,864
- 5,369
Basically Google changed the way the Keyword tool works. It no longer just sends a simple HTTP request with data in the query string and post header.
It now uses a mess of compressed JavaScript code which does XML-RPC calls. The parameters of the request are encoded somehow and unless you unravel the whole mess of JS code to figure out how the request parameters are encoded and what order they go in, you can't do this with a simple cURL request.
And no, you can't use Fiddler or some other tool to intercept the calls and simply change the keywords in the XML-RPC data being sent in the HTTP request. There appear to be checksum values in the call data. I was able to take the call data for a single keyword and replace it with any other single keyword and make it work, but trying to pass in more than one keyword at a time didn't work.
So I created a tool which scrapes the keyword tool which uses the IE browser control under the hood to get around the whole mess. Works just fine under multiple threads. The only issue is that you can't run multiple browser controls, each with their own proxy settings as setting the proxy for the IE control sets it for all IE browsers.
But my tool works just fine in scraping the tool under 3-4 threads until the current proxy IP gets a temporary ban. Then I just rotate to the next proxy in the list. I'm able to suck in an average of about 1200 keywords per minute or about 1.7 million per day per computer that I run it on
Hey man, fire up several VMWare instances and you can run as many instances of your program as you can run vmware sessions. A decent computer could probably handle 20 or more.
BlackHatSoda
Regular Member
- Oct 6, 2009
- 393
- 329
MyOtherPersona
Newbie
- Sep 13, 2011
- 11
- 2
I would be very interested in testing and potentially using your solution.
You are saying it is not ready for retail, but perhaps you could install it on a hosted windows server?
Let me know what you have in mind (be reasonable please ) and let me know how I could test it.
Matthias
You are saying it is not ready for retail, but perhaps you could install it on a hosted windows server?
Let me know what you have in mind (be reasonable please
) and let me know how I could test it.Matthias
ethanhunt0257
Newbie
- Sep 13, 2011
- 10
- 0
Hey, guys need information on SENUKE software, can't find the download links.
