1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How can I automate a google query in bulk

Discussion in 'Black Hat SEO Tools' started by Fathom, Aug 18, 2011.

  1. Fathom

    Fathom Power Member

    Joined:
    Jul 1, 2011
    Messages:
    518
    Likes Received:
    282
    Location:
    Hertfordshire
    I want an automated way to scrape Google for several commands such as intitle:keyword

    For example I have a list of 500 potential keywords and I want to see how many results are in Google for each particular command.

    I need to find a way to automate this as I'm spending too much time doing it manually.
     
  2. William Rufino

    William Rufino Newbie

    Joined:
    Aug 3, 2011
    Messages:
    32
    Likes Received:
    2
    Well you can always code it :)

    its so easy......... just look at the html and parse it...

    php you can use DOM
    python BeautifulSoup
    C# htmlagility something...
     
  3. Fathom

    Fathom Power Member

    Joined:
    Jul 1, 2011
    Messages:
    518
    Likes Received:
    282
    Location:
    Hertfordshire
    anyone capable of doing this for me? what's your price?
     
  4. dromero

    dromero Newbie

    Joined:
    Nov 19, 2010
    Messages:
    35
    Likes Received:
    18
    Php using curl, i coded a tool to retrieve how many adwords advertisers of every keyword from a list, using a delay you wont need proxies but it will be slow as hell
     
  5. webblackart

    webblackart Registered Member

    Joined:
    May 13, 2009
    Messages:
    60
    Likes Received:
    9
    do it on your own!
    here some example autoit code, all you need to do is load your keywordfile and loop through it using the code i have written for you to query qoogle for the count of results.

    Code:
    Func httpget($url)
        $oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
        $oHTTP.Open("GET" , $url, False)
        $oHTTP.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 4.0.20506)")
        $oHTTP.SetRequestHeader("Referrer", $url)
        $oHTTP.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded")
        $oHTTP.Send()
        Return $oHTTP.ResponseText
    EndFunc ;httpget($url)
    
    $xurl = "http://www.google.de/search?q=intitle%3Akeyword"
    $s_html = httpget($xurl)
    $p = "(?i)resultStats>\S+ ([\d\.]+) "
    $res = StringRegExp($s_html,$p,3)
    consolewrite($res[0] & @crlf)
    
     
    • Thanks Thanks x 1
    Last edited: Aug 19, 2011
  6. wu1239

    wu1239 Newbie

    Joined:
    Jun 4, 2011
    Messages:
    16
    Likes Received:
    0
    I could finish your job with a price of $20. If you are interested, PM me.
    And I will give you the source code, modify the code is easy and I can teach you everything you need.
    BTW: It's quite easy.
     
  7. scarab

    scarab Newbie

    Joined:
    Dec 11, 2010
    Messages:
    38
    Likes Received:
    11
    I want to do something simliar, but won't google block the IP where the queries are comeing from?

    For example,
    Let say I have a website that I manage to get a lot of people to use. They run a search of my site, and let's say in this particular instance, only a few results that are on my site, or maybe no results, come up. At that point, I want to display Google search results, as if they were part of my own search results, but shown below the results that are on my site (if any).

    I think Google would eventually block my sites IP because if I get a lot of traffic and they are doing searches (which ultimately query Google), google will think its some kind of automated thing and block it.


    similar real world example:
    PicClick does queries of Craigslist, eBay, etsy. Apparantly, CL blocked PicClick at one point. I can't post links but there is a post on thesundaytimesmarket about this.

    So, is thier some solution to this that won't slow down the searches?
    I think using proxies might slow it down.

    Does Google offer a plan so I can pay them to be "powered by Google"?
     
    Last edited: Sep 6, 2011
  8. Fathom

    Fathom Power Member

    Joined:
    Jul 1, 2011
    Messages:
    518
    Likes Received:
    282
    Location:
    Hertfordshire
  9. musicalflava

    musicalflava Newbie

    Joined:
    Jun 7, 2011
    Messages:
    43
    Likes Received:
    3
    Use the tool allinscraper
     
  10. machines-in-motion

    machines-in-motion Junior Member

    Joined:
    Aug 11, 2010
    Messages:
    129
    Likes Received:
    46
    Occupation:
    Engineer
    Location:
    Here and there
    If you want to be clever about the queries so that you don't get bounced, do two things:
    First, spoof your tool to say that the queries are from Firefox or chrome.
    Second, use the IP range 74.125.0.1 to 74.125.0.57 instead of Google.com. That is all of their servers (as far as I know) and they keep track of repeat queries separately. Going to Google.com simply auto-assigns one, which is inefficient for mass queries.
     
    • Thanks Thanks x 1
  11. TheMatrix

    TheMatrix BANNED BANNED

    Joined:
    Dec 20, 2008
    Messages:
    3,444
    Likes Received:
    7,280
    ScrapeBox can do this.
     
  12. fizz707

    fizz707 Jr. VIP Jr. VIP Premium Member

    Joined:
    Jul 11, 2011
    Messages:
    267
    Likes Received:
    57
    Occupation:
    SEO, IM, IT dev
    Location:
    Eastern Europe/US - back and forth
    Home Page:
    Rank Tracker can do this, too ;)
    Besides, you can use such commands as allintitle, allinurl, allinanchor, allintext, inanchor, intext, inurl, and some sequences of these commands.
     
  13. mcbain2279

    mcbain2279 Newbie

    Joined:
    Mar 22, 2011
    Messages:
    49
    Likes Received:
    34
    market samurai is free, and it does this too. It works super well.
     
  14. MyGirlfriendIsaPBN

    MyGirlfriendIsaPBN Jr. VIP Jr. VIP

    Joined:
    Mar 22, 2016
    Messages:
    318
    Likes Received:
    44
    bumping this old ass thread. Looking for a tool that does exactly what the OP has described but im struggeling to find anything good.