1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scrapebox optimal proxies?

Discussion in 'Proxies' started by Roi07, May 6, 2015.

  1. Roi07

    Roi07 Registered Member

    Joined:
    Apr 29, 2015
    Messages:
    50
    Likes Received:
    1
    I'm using 12 dedicated private Ip's with scrpaebox. after setting the software on the most optimal settings
    I was able to scrape 20K URL's with 30KW's before my list died (google ban). what setup will help me run smoother and scrape more url's?
    Also, what other engine should I look into that'll give me good results and is less secured against bots?

    Scrapebox settings-
    Harvester connections : 1
    Results : 500
    Delay: 10sec
    Platform: Wordpress
    Search Engine: Google
     
  2. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,875
    Likes Received:
    2,058
    Gender:
    Male
    Home Page:
    You could increase the delay if you want to avoid a ban, its simple really, bans happen because your going to fast. Go slower by adding more proxies or increasing the timeout.

    Bing and yahoo are both pretty good, deeperweb is excellent and gets its results from google.

    I would just load up all the engines and look at the total count scraped when its done and use the ones that meet your criteria. Glad that you were able to get more results then the 2500 or 4000 or whatever it was before though. :) Your headed in the right direction.
     
  3. Roi07

    Roi07 Registered Member

    Joined:
    Apr 29, 2015
    Messages:
    50
    Likes Received:
    1
    Great, that's good to know.
    I was also able to scrape from bing, deeperweb and start page with pretty good results, it gave me about a 100K url list with 30KW's.

    I have another question, I read a method that explains how to get AA lists. It basically says that I should find 1 AA blog spammed to death, find a comment in there that links to another site, scrape it's backlink profile and then just post directly to it.
    Problem is... with free Majestic and Mozscape it's almost impossible to scrape so much (for example 400K backlink profile).
    With MarketSamurai I was able to scrape 1000 backlinks at once but it's too slow. Are there ways to optimize and make the proccess faster?

    Thank you for your patience and your willing to help!
     
  4. RuthSam

    RuthSam Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 19, 2010
    Messages:
    3,814
    Likes Received:
    979
    Gender:
    Male
    Home Page:
    Thanks for this info http://deeperweb.com/ - I didn't know of them.. :)


     
  5. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,875
    Likes Received:
    2,058
    Gender:
    Male
    Home Page:
    Moz seems to work fine for me, but either way, yes that is an effective method.