Scrapebox - Low results for Google?

[luckskywalker]

So,

I'm building my first list for GSA. I'm running Scrapebox with 30 private proxies in a dedicated, but I'm get very low results for Google and a lot for other engines. This is normal?

Take a look:


Ah, sometimes I get Error 999 on Yahoo Search Engine only, why? - Edit: I set Yahoo connections to 15, let's see.

Thank you.

 

[divok]

Don't use private proxies for scraping . How many connections have you set for google ?

 

[luckskywalker]

Don't use private proxies for scraping . How many connections have you set for google ?

How do you mean don't use private proxies for scraping? Everybody told me to use good proxies for that, I bought the semi-dedicated from buyproxies.

40 Connections.

 

[satyr85]

To harvest like a boss you need:
- tons of public proxies, private are good for posting but not for harvesting
- fast harvester - Gscraper or Hrefer, SB is slooow
- good footprints
- good keywords
Read about harvesting, footprints, how fast and why google ban proxies.

 

[Kralj187]

Private proxies are good for harvesting as long as you have enough of them...

 

[alaltaierii]

With 40 connections your 30 semi dedicated proxies will burn in minutes. Wait 24 hours and try again with only 2-3 connections.
In this time you can use scrapebox to scrape public proxies and start harvesting with those.

 

[Rua999]

Private proxies are the bomb for scraping, but you have to have a lot of them to do any serious scraping on google, or reduce your connections - and especially when scraping with advanced operators (inurl: etc).

You can take my word for it or not, but trust me i've been frustrated so many times just like you, been back and forward with scapebox support, and i can tell you that if im scraping google with 100 semi dedi proxies with advanced operators then il only scrape with 5 connections or in other words at a 20:1 ratio.

If im not using advanced operators i scrape at 8 or 10 connections, or a 10:1 ratio with 100 semi dedi proxies.

Your private proxies are burnt at the minute, you'll have to lay off scraping google with them for a day or a day and a half or so until google unban them. It's not that they're banned from scraping everything, or performing page rank checks or anything, but they are defo banned from scraping advanced operators.

Go to settings >> use multi threaded harvester - and uncheck this. Then try and harvest, there will be a status column. What errors are you getting in the status column?
Here is a video by loopline that goes over how to do this and other troubleshooting for the harvester that might be helpful, and will defo show you that some or most of your proxies are currently banned when you use the single thread url harverter.

https://www.youtube.com/watch?v=2QaLWgTXsRo

999 from yahoo means the IP is blocked. 302 from google means the IP is blocked. Are you scraping for advanced operators? If so this video by loopline will be helpful:

>
> In general google has really cracked down lately on banning IPs much faster. There is no rhyme or reason to it either, proxies can pass the proxy test and then if you immediately retest them they can show failed.

Since google is such a bitch with their advanced operators, and since yahoo doesn't allow you to lose a lot of them but does allow you to scrape at a high number of connections, a good tip is to use a tool like "Footprint Factory" and scrape just a few say inurl: results and then analyze them with Footprint Factory. Then use the footprints that generates to scrape with in Yahoo.

 

[luckskywalker]

Private proxies are the bomb for scraping, but you have to have a lot of them to do any serious scraping on google, or reduce your connections - and especially when scraping with advanced operators (inurl: etc).

You can take my word for it or not, but trust me i've been frustrated so many times just like you, been back and forward with scapebox support, and i can tell you that if im scraping google with 100 semi dedi proxies with advanced operators then il only scrape with 5 connections or in other words at a 20:1 ratio.

If im not using advanced operators i scrape at 8 or 10 connections, or a 10:1 ratio with 100 semi dedi proxies.

Your private proxies are burnt at the minute, you'll have to lay off scraping google with them for a day or a day and a half or so until google unban them. It's not that they're banned from scraping everything, or performing page rank checks or anything, but they are defo banned from scraping advanced operators.

Go to settings >> use multi threaded harvester - and uncheck this. Then try and harvest, there will be a status column. What errors are you getting in the status column?
Here is a video by loopline that goes over how to do this and other troubleshooting for the harvester that might be helpful, and will defo show you that some or most of your proxies are currently banned when you use the single thread url harverter.

999 from yahoo means the IP is blocked. 302 from google means the IP is blocked. Are you scraping for advanced operators? If so this video by loopline will be helpful:

>
> In general google has really cracked down lately on banning IPs much faster. There is no rhyme or reason to it either, proxies can pass the proxy test and then if you immediately retest them they can show failed.

Since google is such a bitch with their advanced operators, and since yahoo doesn't allow you to lose a lot of them but does allow you to scrape at a high number of connections, a good tip is to use a tool like "Footprint Factory" and scrape just a few say inurl: results and then analyze them with Footprint Factory. Then use the footprints that generates to scrape with in Yahoo.

I'm very sad with scrapebox, I should buy GScraper, since is more investment but also more fast with unlimited proxies...

First, my proxies are fresh, it's the third time that I ask for buyproxies to new proxies (lucky me buyproxies is a good provider). Now I set up 30 semi dedi proxies with just 5 connections. Results? Error 999

Second, every time I Stop Harvesting, Bing doesn't stop, so the software stay harvesting forever, the only way to stop is closing on Task Manager but I lose all my setting and my harvested URL.

I hope I get a refund on this...

And please, don't say to use public proxies, if this takes 2 days to scrape with private proxies, how about public proxies, LOL

 

[Rua999]

Use yahoo to scrape like i said.. not bing, it's buggy as hell and yes your scrapebox will constantly hang like you said. I wouldn't go pinning your hopes on gscraper either. I had it and sold the license to someone else cuz i wasn't happy with it. The results you see people talking about are based on it skipping a keyword when the proxy fails and moving onto the next one - not good enough for me if i need every keyword to be scraped, and also it's sooooo slow if you scrape with advanced operators too :-/

I have no problem scraping with scrapebox now using the ratios i said above and private proxies from buyproxies.

 

[luckskywalker]

I have only 30 semi proxies, and yes I'm using advanced operators...

Ok, I have a few questions.

Should I use 2 connections for Yahoo and Google with 30 semi proxies? LOL

And yes, Im getting 302 and 999 errors. I have to wait one day now? Now I can't use them with GSA too right? ):

Also, can you explain what Trim to Root does?

If I already have a list or URL, can I start harvesting again and merge this lists?

Thanks.

 

[satyr85]

I wouldn't go pinning your hopes on gscraper either. I had it and sold the license to someone else cuz i wasn't happy with it.

Gscraper is fastest and best Google scraper on market. You use private proxies? And how many links you harvest per day ? I harvest 100-200+ milion links with public proxies - its not doable with private proxies without spending high XXX or low 1XXX a month.

Harvesting and proxies for harvesting are my bread & butter. Trust me private ones are not way to go. Want to scrape like a boss? Use good public proxies service (thats most important) and Gscraper. With good public proxies you can get results like this:




P.S. No i dont use any proxies service from BHW.

 

[Rua999]

Trim to root does exactly as it sound like. If you have a list urls like domain.com/long-url-here it will time them to domain.com.

Ye, you need to give your proxies a rest from scraping for a day or a day and half to be on the safe side and start scraping with less threads then.

With 30 proxies i would probably scrape at 2 connections with advanced operators on google, and on yahoo without advanced operators at about 6 or 7 connections.

They might sound like a low number of connections, but they scrape a whole lot quicker than public proxies do.

You can save the whole scrapebox environment every so often as well by going to projects > save application environment.. so if scrapebox crashes you can just load it back to where you had it last saved and your proxies, keywords, and scraped list will all be where you left them

 

[luckskywalker]

Bah.

Just waste my entire time today then

Here my settings now:



Recommend anything?

 

[proudgoodmkt]

When using private proxies for scraping, use fewer threads. Like 1:5, a single thread per 5 private proxies would last you long enough but it would mean having to wait much longer as well.

 

[Sweetfunny]

Use yahoo to scrape like i said.. not bing, it's buggy as hell and yes your scrapebox will constantly hang like you said. I wouldn't go pinning your hopes on gscraper either. I had it and sold the license to someone else cuz i wasn't happy with it. The results you see people talking about are based on it skipping a keyword when the proxy fails and moving onto the next one - not good enough for me if i need every keyword to be scraped, and also it's sooooo slow if you scrape with advanced operators too :-/

I have no problem scraping with scrapebox now using the ratios i said above and private proxies from buyproxies.

The new harvester in ScrapeBox v2 is a lot better, here's bing for example:

That's over 100,000 urls per minute at only 150 connections, using free internal ScrapeBox proxies which i didn't even have to test first. But you will still be able to harvest with keyword stats, do retries etc like you want.

Alot of people want to do "finesse" scraping and gather say the top 10 business urls for specific keywords and care about accuracy and metrics, others just want to tear down 100 million urls as fast as possible. Well the new harvester will have 2 modes and accommodate both, also with a native 64 Bit version so memory is no issue.

 

[ibraheemproductions]

How do you mean don't use private proxies for scraping? Everybody told me to use good proxies for that, I bought the semi-dedicated from buyproxies.

40 Connections.

sub to me on youtube, you can help me with seo. ibraheemproductions

 

[luckskywalker]

The new harvester in ScrapeBox v2 is a lot better, here's bing for example:

That's over 100,000 urls per minute at only 150 connections, using free internal ScrapeBox proxies which i didn't even have to test first. But you will still be able to harvest with keyword stats, do retries etc like you want.

Alot of people want to do "finesse" scraping and gather say the top 10 business urls for specific keywords and care about accuracy and metrics, others just want to tear down 100 million urls as fast as possible. Well the new harvester will have 2 modes and accommodate both, also with a native 64 Bit version so memory is no issue.

You didn't test the proxies? Because takes sooooo long to test it.

And holy shit, this is a lot of URL for 4min. You can't get this for Google right?

 

[Sweetfunny]

You didn't test the proxies? Because takes sooooo long to test it.

No because with v2 the proxies are already tested/filtered on the ScrapeBox servers, and when you start harvesting they are downloaded and ready to use instantly without doing anything.

 

[luckskywalker]

No because with v2 the proxies are already tested/filtered on the ScrapeBox servers, and when you start harvesting they are downloaded and ready to use instantly without doing anything.

From where did you get the V2? My scrapebox is already updated and just see V1.

 

[kristur]

No because with v2 the proxies are already tested/filtered on the ScrapeBox servers, and when you start harvesting they are downloaded and ready to use instantly without doing anything.

The v2 sounds amazing. Have you set a release date or?