1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scraping New Google Adwords Tool

Discussion in 'General Scripting Chat' started by twmas52712, Sep 9, 2010.

  1. twmas52712

    twmas52712 Junior Member

    Joined:
    Jun 17, 2009
    Messages:
    130
    Likes Received:
    27
    Hey guys,

    I started working today on scraping the new google adwords tool, and I've run into some serious issues. Mainly, the js is fcking complicated! In fact, I've spent the last three hours trying to track down where the POST parameters are being defined to no avail.

    This is the general HTTP Request Flow:
    1) Break Captcha - Done
    2) Post to https://adwords.google.com/o/Targeting/g?__u=1000000000&__c=1000000000

    The post data looks something like this:
    |1|54|https://adwords.google.com/o/Target...909C258|_|invoke|3|15s|19y|15n|15o|g|TiAction (Search.SearchInput.KEYWORD_IDEAS.RelatedToKeyword) Batch #2|p0XfWsCzHMYz4hDJDEOmec9UkDc:1276859569935|19s|10g|15v|15x|c|h|i|106|17t|110|en_US|ul|12g|11f|17k|11i|KEYWORD|GLOBAL_MONTHLY_SEARCHES|ESTIMATED_COST_PER_CLICK|IDEA_TYPE|AVERAGE_TARGETED_MONTHLY_SEARCHES|AD_SHARE|COMPETITION|EXTRACTED_FROM_WEBPAGE|SEARCH_SHARE|TARGETED_MONTHLY_SEARCHES|16y|12x|19z|hh|US|13a|hu|en|139|17q|bm|13h|16x|17o|bl|'.$keyword.'|1|2|3|4|1|5|5|6|3141861564|16782206751997952|8|7|1|8|0|9|0|0|10|2|1000000000|0|1000000000|0|11|12|13|1|14|15|16|-4|0|0|0|17|5|0|0|0|0|0|0|0|0|0|18|0|19|1|0|0|0|0|0|0|0|0|0|7|0|0|0|20|21|235|22|0|23|24|50|0|25|0|0|26|0|27|10|28|29|28|30|28|31|28|32|28|33|28|34|28|35|28|36|28|37|28|38|39|27|4|40|41|1|42|43|0|44|41|1|45|46|47|48|49|2|50|51|52|53|-42|54|0|0|0|


    Where the hell all that data comes from I have no clue. If anybody is up for a challenge and wants to bounce ideas around, I'm always in the IRC channel or you can PM me or keep it on the forum. I'm down to share the results, of course.
     
  2. twmas52712

    twmas52712 Junior Member

    Joined:
    Jun 17, 2009
    Messages:
    130
    Likes Received:
    27
    just wanted to give this a bump. i'd love to hear from anybody who's successfully scraped the new adwords tool.
     
  3. truckin

    truckin Newbie

    Joined:
    Sep 3, 2010
    Messages:
    15
    Likes Received:
    0
    I am wondering this too... people have told me it is impossible.

    how is it going twmas?