Gscraper performance

Hi all

Gscraper get slow since a few days.
I hardly get 1500 urls harvested per minute... using gscraper proxies service

How much you guys doing ?

fmartin said:

Hi all

Gscraper get slow since a few days.
I hardly get 1500 urls harvested per minute... using gscraper proxies service

How much you guys doing ?

Click to expand...

I do not know... I use proxies filtered and scraped with GSA Proxy Scraper, and autoloaded into GScraper once per hour. With only thirty threads I vary between 500 and 1000 URL's/min. depending on the keyword contents. I use so few threads because I am also running Search Engine Ranker (100 T), Proxy Scraper (35 T) , SEO Indexer (100 T), and two instances of ScrapeBox (150 T each).

Like any scraper, the key to fast scraping is working proxies. GSA attempts a search with a proxy, then drops the proxy for a new search, even if the same page and keyword; and rotates through the loaded proxies. If the proxys are burnt out, as happens with GScrapers service, then your scrape speed will slow. It also depends on whether you are scraping Google, Yahoo, or Bing. Google bans proxies much faster than the other two engines. The speed of proxies being banned also depends on the use of the advanced SE operators.

What you have to remember about GScraper, is that you are not the only one using the proxy pool.

Try a new proxy source and see if that helps.

kn0wl3dg3 said:

Try a new proxy source and see if that helps.

Click to expand...

Precisely! Just what I wanted to say.

Just to erk the OP, and get the point across :



This is from scraped proxies.

Now at 13 minutes in and I am at just under 3700 URL's/min and rising