twmas52712
Junior Member
- Jun 17, 2009
- 140
- 27
Hey guys,
I started working today on scraping the new google adwords tool, and I've run into some serious issues. Mainly, the js is fcking complicated! In fact, I've spent the last three hours trying to track down where the POST parameters are being defined to no avail.
This is the general HTTP Request Flow:
1) Break Captcha - Done
2) Post to https://adwords.google.com/o/Targeting/g?__u=1000000000&__c=1000000000
The post data looks something like this:
|1|54|https://adwords.google.com/o/Target...909C258|_|invoke|3|15s|19y|15n|15o|g|TiAction (Search.SearchInput.KEYWORD_IDEAS.RelatedToKeyword) Batch #2|p0XfWsCzHMYz4hDJDEOmec9UkDc:1276859569935|19s|10g|15v|15x|c|h|i|106|17t|110|en_US|ul|12g|11f|17k|11i|KEYWORD|GLOBAL_MONTHLY_SEARCHES|ESTIMATED_COST_PER_CLICK|IDEA_TYPE|AVERAGE_TARGETED_MONTHLY_SEARCHES|AD_SHARE|COMPETITION|EXTRACTED_FROM_WEBPAGE|SEARCH_SHARE|TARGETED_MONTHLY_SEARCHES|16y|12x|19z|hh|US|13a|hu|en|139|17q|bm|13h|16x|17o|bl|'.$keyword.'|1|2|3|4|1|5|5|6|3141861564|16782206751997952|8|7|1|8|0|9|0|0|10|2|1000000000|0|1000000000|0|11|12|13|1|14|15|16|-4|0|0|0|17|5|0|0|0|0|0|0|0|0|0|18|0|19|1|0|0|0|0|0|0|0|0|0|7|0|0|0|20|21|235|22|0|23|24|50|0|25|0|0|26|0|27|10|28|29|28|30|28|31|28|32|28|33|28|34|28|35|28|36|28|37|28|38|39|27|4|40|41|1|42|43|0|44|41|1|45|46|47|48|49|2|50|51|52|53|-42|54|0|0|0|
Where the hell all that data comes from I have no clue. If anybody is up for a challenge and wants to bounce ideas around, I'm always in the IRC channel or you can PM me or keep it on the forum. I'm down to share the results, of course.
I started working today on scraping the new google adwords tool, and I've run into some serious issues. Mainly, the js is fcking complicated! In fact, I've spent the last three hours trying to track down where the POST parameters are being defined to no avail.
This is the general HTTP Request Flow:
1) Break Captcha - Done
2) Post to https://adwords.google.com/o/Targeting/g?__u=1000000000&__c=1000000000
The post data looks something like this:
|1|54|https://adwords.google.com/o/Target...909C258|_|invoke|3|15s|19y|15n|15o|g|TiAction (Search.SearchInput.KEYWORD_IDEAS.RelatedToKeyword) Batch #2|p0XfWsCzHMYz4hDJDEOmec9UkDc:1276859569935|19s|10g|15v|15x|c|h|i|106|17t|110|en_US|ul|12g|11f|17k|11i|KEYWORD|GLOBAL_MONTHLY_SEARCHES|ESTIMATED_COST_PER_CLICK|IDEA_TYPE|AVERAGE_TARGETED_MONTHLY_SEARCHES|AD_SHARE|COMPETITION|EXTRACTED_FROM_WEBPAGE|SEARCH_SHARE|TARGETED_MONTHLY_SEARCHES|16y|12x|19z|hh|US|13a|hu|en|139|17q|bm|13h|16x|17o|bl|'.$keyword.'|1|2|3|4|1|5|5|6|3141861564|16782206751997952|8|7|1|8|0|9|0|0|10|2|1000000000|0|1000000000|0|11|12|13|1|14|15|16|-4|0|0|0|17|5|0|0|0|0|0|0|0|0|0|18|0|19|1|0|0|0|0|0|0|0|0|0|7|0|0|0|20|21|235|22|0|23|24|50|0|25|0|0|26|0|27|10|28|29|28|30|28|31|28|32|28|33|28|34|28|35|28|36|28|37|28|38|39|27|4|40|41|1|42|43|0|44|41|1|45|46|47|48|49|2|50|51|52|53|-42|54|0|0|0|
Where the hell all that data comes from I have no clue. If anybody is up for a challenge and wants to bounce ideas around, I'm always in the IRC channel or you can PM me or keep it on the forum. I'm down to share the results, of course.