1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scrapebox Harvester Not Harvesting

Discussion in 'Black Hat SEO Tools' started by Blueprint, May 31, 2012.

  1. Blueprint

    Blueprint Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 10, 2009
    Messages:
    284
    Likes Received:
    117
    Location:
    Online
    I'm having some issues with the Scrapebox harvester, as of the date of this post. It doesn't seem to be scraping every time. I'm testing this using Google passed proxys. Private Proxys and also Proxy Go proxys, so the proxys themselves are not an issue. I think it's more to do with the software.

    I've seen others with this issue by running a search on this forum and also in Google, but not found a solution.

    I've tweaked the settings and tried on multiple devices (I have several VPS and SB locally, all licensed copies)

    I am getting some odd results, for example it will run scraping, and then just say (flash) "Harvesting" and simply not harvest anything.
    Sometimes after clicking "start harvesting" several times it'll run, but only to a maximum of around 200 or so results (the term has millions of listings, so it should be maxing 1000)
    I just ran it now and only scraped 95 results.

    I'm a little lost as to why this is happening because it doesn't seem to be consistent.

    Finally when I deselect proxys, Scrapebox pulls everything I need, but I'm not about to burn all my VPS and local IPs.

    Again it cannot be proxys as I'm using several top quality paid services. Is this a bug in the latest release?
     
    Last edited: May 31, 2012
  2. alaltaierii

    alaltaierii Supreme Member

    Joined:
    Jun 11, 2010
    Messages:
    1,408
    Likes Received:
    349
    What kind of private proxies do you use?
    I'm using the last scrapebox release(1.15.47) and is harvesting without a problem. For every VPS I have about 60 private proxies(from buyproxy) and a low number of connections(4-5). Here are my results:


    [​IMG]
     
  3. Blueprint

    Blueprint Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 10, 2009
    Messages:
    284
    Likes Received:
    117
    Location:
    Online
    Wow. What results! I'm using a 1GB/s vps, just been playing with the results.
    So do you use the private proxys for scraping/harvesting? I just managed to get 376 results for an "intitle" kw search, but in comparison to yours that's terrible.

    I used public proxys, but these are harvested for me via a service.
     
  4. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,262
    Likes Received:
    8,710
    key word results will rely on the keywords used
    1 person search on a certain keyword will yield
    different results to another key word thought ide
    share that part. i no my service works ok granted
    port scanned public proxies will not beat 24/7 private
    proxies, i have never claimed that, i only offer an alternative
    for thoes who want a cheap service with no recurring monthly
    fees, if you bare that in mind, it works for its purpose with 600
    subs and 0 refunds on my service, but yes if u purely want to scrape
    24/7 with millions or thousands of keywords private proxies might be
    better, just remember not everyone searches high volume. there will
    always be a market for public / private proxies, if you use them for what
    there suited best for , they can still work well

    ps ive had issues scraping with latest version
    so i use 138 to scrape instead, works fine
     
  5. alaltaierii

    alaltaierii Supreme Member

    Joined:
    Jun 11, 2010
    Messages:
    1,408
    Likes Received:
    349
    Yes, I'm always using private proxies when scraping. Just use a low number of connections and you will be fine. The results from the screen above are after 24 hours of harvesting. At the end I will probably get about 8 millions urls.
     
  6. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,262
    Likes Received:
    8,710
    well i no from a chat on skype with 1 of my subs, he scraped 2.5 million
    over night last week, baring in mind hes using port scanned public proxies
    to do it > not scraped public proxies which suck <
    i think 2.5 million over night on port scanned proxies is very good
    yes private proxies will always beat public scraped proxies and public
    port scanned proxies, so you weigh up a monthly cost with a recuring fee
    against a 1 time lifetime fee with slightly lower results
    theres always a place for both
     
  7. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,262
    Likes Received:
    8,710
    or put another way, if given the choice of paying say 50$ every month to
    get 8 million urls a day thats 600$ a year at 224 million results a month
    and u still keep paying for thoes private proxies

    weigh up against

    2/3 million urls a day - 1 flat fee 130$ to use a service as long as you
    want. in the end ille beat you by a mile on price but fall short on urls
    and thats what u get from port scanned public proxies with me

    in short 224 million results a month at 8 million a day at 50$ a month
    or
    77 million results a month 1 fee 130$ and use it as long as you want.

    ile be beaten only on url quantity but by price and longevity just think
    people who payed me in 2010 are still using it in 2012 and paying nothing.

    pros and cons.
     
    Last edited: May 31, 2012
  8. Blueprint

    Blueprint Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 10, 2009
    Messages:
    284
    Likes Received:
    117
    Location:
    Online
    Yes... anyway.

    It's down to the settings as I already diagnosed, as the harvester seems to be now kicking off at about 95 ish URLS from a intitle: KW search. Surprised I can't get any more. This is a Google.com scrape and I've limited the maximum harvester connections to 5, even though I'm on a 1GB/s connection.

    Any other tips on settings from people out there? As I said, I've got the proxy resources so this isn't the issue for me, and I would prefer to discuss other solutions via settings, or tweaks I can do to scrapebox, because regardless of what I'm using the harvester finishes really quickly, which is yielding me only a few URLs I'd hope for 1000's
     
    Last edited: May 31, 2012
  9. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,262
    Likes Received:
    8,710
  10. VIC SEO

    VIC SEO Elite Member

    Joined:
    Feb 19, 2010
    Messages:
    2,156
    Likes Received:
    363
    Gender:
    Male
    Occupation:
    SEO Specialist
    Location:
    iSynergyMedia
    Home Page:
    i boght vps and some proxies .I always used private proxies and free proxies ,never used vps along with private proxies.Have to see what type of results i get