webscraper

My background is in growth hacking. A year ago, my colleague introduced me to this website, and I have occasionally lurked there. Currently, I am building a Chrome extension so you can web scrape and automate directly in Chrome. If you can do that in the Chrome JavaScript console, you should...

Hey friends, I've been scraping data by automating Google Dorks. Since Google is a public form of information this data is legal to used as audiences. I got a first client who uses those leads for cold calling, mostly doctors and lawyer data and whatever the cold callers generate revenue I get...

I've been using SerpApi to scrape emails from Google based on hashtags and all social medias. For each query I can only get around 300 results before it chokes out. So I need to rotate a ton of variables to make new results show up on the first 30 pages (3 pages of 100 results to reduce API...

Seeking help from you guys Hi, I am new to BHW and proficient in web scraping. Seeking help from you guys to utilize the opportunities provided by this platform.

Hello, I have a custom PHP based solution for scraping Amazon (EU locales). Im looking for a developer with proper knowledge and experience with Amazon: The software works with backconnect proxies and can handle/sync up to 500.000 ASINs per day. Actually, we are facing an issue with...

Just wanted to say hello to everyone here. I'm a newbie here, but not on IM world. So I decided to join this great community after reading posts on BHW for about a year, and i thought I could make new IM friends and work together with someone. My Background I have a degree in Math and I...

Hello, I really need some medium project for python portfolio so I decided for bot web scrapper (it's not medium, so I have to push it to the limit). Should generate some money as well. Twitter should be go on, as all data are public - full of information. But I don't really know what kind of...

I am currently web-scraping a bunch of similar pages in Python. Here is my Setup: - Using Requests sessions to get data, no Selenium or JavaScript involved - Using Threads to make concurrent requests, not Asyncio or Aiohttp (Aiohttp is too buggy) - 55 Threads - Maintaining a pool of Request...

I am web scraping developer. I used luminati few months back and read a post saying they share real IP with google(also real IP detected by amazon I think). I am a freelancer and I am looking for something like pay as you go plan or I need many proxies and pay for bandwidth. Also if you know any...

I need help to find the bottleneck and fine tuning my scrapy/python based scraper. Im scraping products from Amazon (Italy at the moment) but struggling with overall requests throughput. Using backconnect rotating proxies: StormProxies (50 threads plan) + Proxyrotator (100 threads) + TOR but...

Hey there if anybody looking for a tool scraping medias, media likes, account followers, followings, comments and other metadata like email I created one in python: https://github.com/SergioWagenleitner/instagram-scraper It also has comment and like functionality. If you are interested I would...

In case anybody is looking for a tool, scaping user followers, followings, medias, likes and other metadata like email, I created one in Python : https://github.com/SergioWagenleitner/instagram-scraper it also has comment and like functionality if anybody is interested I would love to hear...

Hello all, I used to manually search google for pages where I could find public whatsapp groups related to my niche. It was difficult and discouraging.. I got some few hundred groups however, but I know there are tons more out there. How to fish them out using "chat.whatsapp.com" as one of my...