Recent content by viliche

I agree, it is just that puneetas3 said he scraped at speed of 2 page/second without getting banned on google using this setup. My guess is that using thousands of IPv6 equals to use a single IPv4. On my side, running a query every 15sec looks optimum when using a single IP.

I wanted to scrape Google using a wide IPv6 range. I didn?t tested it with 10K or 25K but "only" 500 random addresses. I?ve started scrapping Google (3 firsts serps, 1 query for each) using specific useragent and headers (well, simulating a browser as much I can with Curl). Despite using a...