1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scraping google - clicks vs direct links question

Discussion in 'Black Hat SEO Tools' started by amailman, Aug 6, 2014.

  1. amailman

    amailman Newbie

    Joined:
    Jul 20, 2014
    Messages:
    20
    Likes Received:
    4
    So I am attempting to scrape some information from google. I know after 50ish urls i get a CAPTCHA. I am curious if this will show up less if I am actually writing a script that navigates through their website via clicking their links, like clicking the "next" button instead of using the url for the next page. Does anyone have suggestions or experience with this?
     
  2. YourMoneyButton

    YourMoneyButton Jr. VIP Jr. VIP Premium Member

    Joined:
    Dec 1, 2013
    Messages:
    957
    Likes Received:
    207
    Location:
    Automating The Web
    IP issue is 99% the culprit.

    Are you using public proxies or a VPN of some kind?
     
  3. djw1606

    djw1606 Regular Member

    Joined:
    Jan 24, 2014
    Messages:
    381
    Likes Received:
    187
    Use a VPN. I get this when I am using keyword tools. When I reach the first captcha, I change the VPN location and then carry on.