I’m attempting to scrape data from a website using ChromeDriver and Selenium.

V

vittorianicolosi1994

Regular Member
Joined
Jan 17, 2024
Messages
279
Reaction score
81
I’m attempting to scrape data from a website using ChromeDriver and Selenium, but It has anti-bot detection .I successfully scrape it once on my PC, but after that, I cannot scrape anymore due tò anti-bot ; even with a 4G mobile IP. I tried on a new PC and 4g, and it worked, but After that... anti-bot ...again. so i tryed VPS online, I still cannot scarpe due tò anti-bot . Should I import cookies into Chrome on the VPS? Any advice
 
Advertise on BHW
How faster are you scraping, are you at least making it look like a human?
 
Set up user agent, + connect proxy. Try in headless mode, see what happens, at what stage the error occurs, maybe it does not find CSS, xpath
 
You may want to try Firecrawl? Its anew service that uses AI to help you setup scraping. Ive had a great experience so far, albeit very limited campaigns.
 
You need to be using a version of selenium that is undetectable. YouTube it, there's plenty of tutorials
 
I personally feel Selenium and Mozilla work like an absolute charm... Your beautiful soup in the mix makes everything nice and compact
 
the thing is, chromedriver and selenium have some red flags that are easily detected by anti-bot systems. there are special chromedriver builds for anti-anti-bot parsing, like undetected-chromedriver.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top