Web Scraping Google Shopping | i need your help


Aug 26, 2016
Im in need of a setup for web crawling

BHW Community | I need your help


this is a sample link of a random product from google shopping.

Its a pricing table which is showing data about the competition. For my project i need to get info about the best price (lowest endprice with shipping costs). Furthermore i want to scrape thousands of these google shopping price tables, so i need a tool which is able to work multi-threaded and with proxies. On my journey i couldnt find any tool which is able to do this. therefore im asking this community to help me out. Would be great if there is any program which is able to parse the information directly into an exel or csv, etc. file.

im aware that google proxies will may cost and that free proxies may be blocked. So in the first step im interested in a working tool that can output the information i need in a fast and reliable way. It should check if the price table is working -> block check or some way of checking if data is fetchable.

I tried to setup scrapy or other python based programs but all of them where not able to work for me or werent able to work multi threaded or have proxy support...

thanks in advance for your help!
AdBlock Detected

We get it, advertisements are annoying!

Sure, ad-blocking software does a great job at blocking ads, but it also blocks useful features and essential functions on BlackHatWorld and other forums. These functions are unrelated to ads, such as internal links and images. For the best site experience please disable your AdBlocker.

I've Disabled AdBlock