Has Google gone hard on proxies?

M

mlkykhan

Newbie
Joined
Jan 19, 2012
Messages
43
Reaction score
7
I am using 20 private proxies in Scrapebox and had my max connections set to 10% of proxies i.e 2 connections. But after harvesting 40k links google bans the proxies. I tried to harvest with 5% max connections i.e 1 connection. It completed a harvest of 140k links.

Is it just me, my proxies or everyone else is facing difficulty with selecting 10% of proxies as Max connections?
 
Advertise on BHW
i usually use 3-5k public proxies for scraping,and i change them every few hours,so i get millions of urls.20 proxies is a very low number for scraping.
 
I too face proxy ban most of the time. Even it happens frequently.
 
Last edited:
SerpEvil said:
i usually use 3-5k public proxies for scraping,and i change them every few hours,so i get millions of urls.20 proxies is a very low number for scraping.
Click to expand...

SB skips a public proxy if it dies during scraping or it doesn't return results for the keyword on which the proxy dies?

I am asking because I heard a lot that scraping with public proxies is a bad idea.
 
SerpEvil said:
i usually use 3-5k public proxies for scraping,and i change them every few hours,so i get millions of urls.20 proxies is a very low number for scraping.
Click to expand...

What proxy service do you use?
 
Don't use scrapebox if you are doing heavy scrapping. It's great for the other functions but it's not really good when you need to scrape urls in the millions.

You are better off with Gscraper of Hrefer
 
Maybe someting wrong with your proxies?
are they private or semi-private? if they are semi-private you can be sharing the same proxies with heavy spammers marketers and they banned the proxies for all the rest of you (semi.private are shared between 2 or more users)
 
With so few proxies you should avoid scraping with advanced search operators like inurl, etc in order to avoid proxy bans.
 
It's not rocket science man. You need wayyy more proxies.

I usually scrape around 1000 then replace them every 4 hours or so.
 
For scraping you are better off using public proxy list that are Google passed.
 
watsonovedades said:
Maybe someting wrong with your proxies?
are they private or semi-private? if they are semi-private you can be sharing the same proxies with heavy spammers marketers and they banned the proxies for all the rest of you (semi.private are shared between 2 or more users)
Click to expand...

Proxies are private. From NewIpNow.org. May be I should contact support.
 
gu3sswh0 said:
For scraping you are better off using public proxy list that are Google passed.
Click to expand...

Which public proxy service do you recommend? I tried scraping lots but only a few were google passed and died very quickly.
 
Dont waste ur money on public proxies they burn fast. Buy 50 private proxies set sb to 2 connections rnd delay min 35 sec max 60 sec ( very important) and fire it up. As we speak my sb is harvesting 60-80 urls/ sec with 2 connections. Pros: the harvesting session is stable, u can scrape for days like this without stopping.
Cons: u wont get 15k urls/s this way:)
 
laur.laurix said:
Dont waste ur money on public proxies they burn fast. Buy 50 private proxies set sb to 2 connections rnd delay min 35 sec max 60 sec ( very important) and fire it up. As we speak my sb is harvesting 60-80 urls/ sec with 2 connections. Pros: the harvesting session is stable, u can scrape for days like this without stopping.
Cons: u wont get 15k urls/s this way:)
Click to expand...

Thank you. That is why I started with buying 20 private proxies. I guess I will buy some more now. I will test mixing private proxies with share-private proxies too and share result :)
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top