Scrapebox Harvesting at 2 URLs/s....

B

blitz206

Newbie
Joined
Sep 24, 2013
Messages
47
Reaction score
8
Hi guys,

I have been having problems with SB harvesting Google URLs. I have 60 paid proxies (semi-private), and a dedicated VPS. My connections are up to 10 for G harvesting, and my timeouts are set to max.

I can harvest the first 2000 URLs quickly, but after that they seem to burn out for about a full day. I have tried everything on my end possible.

This is definitely a Google-specific issue, because I can harvest Bing/Yahoo with no problem. Has anyone had this problem? Does anyone have any suggestions? Thanks.
 
Advertise on BHW
I have had the same problem for weeks. Sent email to support but got a load of bullshit from them so I gave up and haven't scraped with scrapebox since. Its all to do with their new update which has messed a lot of things up in scrapebox. I guess the reason people are not complaining is that a lot of people don't consider scrapebox that important anymore so they just move on to other things and other software. What a shame...I used to swear by scrapebox.
 
Can you recommend any alternative software?
 
1. 60 semi private proxies = you dont know what other users of these proxies are doing. Maybe they are also harvesting google so your proxies can be banned by google.

2. Yahoo and bing dont ban proxies as fast as google - thats why you can scrape bing&yahoo but not google. Your problems are not because of scrapebox but because of google updates. Few months ago it was possible to scrape google with low number of proxies. Now proxies are banned much faster.

Solution for proxies:
Only private ones - not shared or tons of public proxies (for example proxygo service)

Solution for scraping:
Gscraper is very hast harvester, you can harvest few times faster than in scrapebox and gscraper is more stable than scrapebox, but you need good proxy source - private ones or shitload of public proxies. Dont buy gscraper paid proxy service - its not worth single penny now.

Scraping yahoo/bing is also not best way to go. I was scraping yahoo for few days and get tons of irrelevant results. Only way to go when scraping google is good scraper + good proxies. With gscraper i can scrape 50urls per second with public proxies that cost less than $10 per month (its not known provider here and his proxies are not used by many members so dont expect same results with other $10 per month services).
 
Thanks for the infor, satyr85. How many proxies do you use with Gscraper?
 
Great call on GScraper, I am harvesting 250 a second on 10 proxies!
 
I believe it is a SB issue now, given I harvested at 250/sec with GScraper.
 
blitz206 said:
I believe it is a SB issue now, given I harvested at 250/sec with GScraper.
Click to expand...

Have you restarted Scrapebox and your VPS? Just a small suggestion. I would recommend contacting their support, but I don't know how active they are with it.

Did you update to their latest version? 1.6?
 
I haven't tried their support, which I should. I have always found the forum community to be more punctual and knowledgeable, however!

I am on V1.16.
 
Did you update to their latest version? 1.6?
Click to expand...

I was using 1.6 and comparing it to gscraper. 1.6 is have some bugs, thats all. For example when i take shitload of keywords (390k - only english), 10 footprints every time i exit scrape sb crash. Custom scraper sometimes was working, sometimes not with my proxies (mostly not)

@OP
250/sec or 250/minute? Gscraper tell how many you scrape per minute not per second. 250 per second = 15k per minute. It would kill 60 private proxies very fast.
 
I was getting 16K a minute. Full list in about 20 secs.
 
That being said, it's not working now. Could have burned them out.
 
I have never had luck with URL harvesting consistently. How many should I use? What is the secret that I am missing?
 
Why don't you harvest proxies? Go to google, type in proxy list and combinations of that. Then copy one of the IP strings and type that into google and hit search. You'll find all of the proxy websites that posted that proxy along with new ones. Then run them through the proxy checker in scrapebox. Remove the duplicates, check for alive or dead, copy alive to clipboard and post them back in the proxy field. Then use those to harvest URLs.
 
I have tried both Scrapebox and GScraper, with three different proxy providers, on both my machine and my VPS (located in Poland). Still, nothing. Is anyone else having an issue with Google URL harvesting? This CAN'T be only me..
 
PM me if you're still having a proxy problem for scraping. I'll let you try out some for free that should remove your issue entirely if it's proxy related.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top