1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

SB:Google Operators (intitle,inurl etc) and how to harvest a large list

Discussion in 'Black Hat SEO' started by jb2008, Nov 17, 2010.

  1. jb2008

    jb2008 Senior Member

    Joined:
    Jul 15, 2010
    Messages:
    1,158
    Likes Received:
    972
    Occupation:
    Scraping, Harvesting in the Corn Fields
    Location:
    On my VPS servers
    When I go on Google (normal search in my browser), if I use too many operators like intitle, inurl etc. in a short amount of time, they basically block me from searching. If I search a lot WITHOUT operators it's fine.

    Now I'm thinking about my Scrapebox harvests, where I have had big keyword lists, say 100,000 , and a few hundred proxies, yet a lot of the keywords have come back red, i.e. incomplete.

    When I'm in my browser, Google allows me about 20 searches with operators, before it shuts me off. Therefore to keep harvesting ok I would have to have 5000 proxies assuming they all were live and none of them died in the process.

    My question is, how do you get around these limits? And if it is not possible for large keyword lists, do you just use Yahoo instead? How much heavy usage do they allow, compared to Google? I heard some people don't even bother using proxies when harvesting from Yahoo but I'd like to get your input on it.
     
  2. SleepieGirl

    SleepieGirl Regular Member

    Joined:
    Mar 7, 2009
    Messages:
    439
    Likes Received:
    290
    Occupation:
    Need Custom Programs? Message Me!!!
    Location:
    TenDollarBlog.com
    slow down & use proxies?
     
  3. Cindy

    Cindy Power Member

    Joined:
    Apr 7, 2008
    Messages:
    632
    Likes Received:
    88
    Yeah I was wondering that about yahoo too.
     
  4. vetter80

    vetter80 Newbie

    Joined:
    Nov 4, 2010
    Messages:
    18
    Likes Received:
    5
    Setting up a random delay of between 1 and 60 seconds between searches and using a lot of different browser useragents is helpful in my experience.
     
    • Thanks Thanks x 2
  5. Bartholomew

    Bartholomew Regular Member

    Joined:
    Dec 31, 2009
    Messages:
    290
    Likes Received:
    103
    Home Page:
    Use proxies, try to not use google operators. Say, for phpbb forum you might think about combination of the following
    Code:
    In total there are users online  registered  hidden and guests based on users active over the past  minutes
    Most users ever online was 
    
     
  6. jagreen78

    jagreen78 Regular Member

    Joined:
    Sep 28, 2009
    Messages:
    498
    Likes Received:
    115
    Location:
    Houston
    dude this just happened to me as well while I was doing some keyword research....this is horse balls
     
  7. bilbo

    bilbo Power Member

    Joined:
    Jan 26, 2009
    Messages:
    644
    Likes Received:
    1,134
    Occupation:
    an actor on wizard of oz - the 3rd munchkin
    Location:
    middle earth
  8. jb2008

    jb2008 Senior Member

    Joined:
    Jul 15, 2010
    Messages:
    1,158
    Likes Received:
    972
    Occupation:
    Scraping, Harvesting in the Corn Fields
    Location:
    On my VPS servers
    I can't not use operators as I've got some golden footprints which require the inurl: operator each time.