Setup a .htaccess redirect for any of the sites that are sending to spam to you. ( redirect based on referral url)
Then setup a filter in google analytics to filter out any of the bad traffic.
Clickease is also a good tool for stopping bots getting in via google ads.
There is a large market already for them, and you'd need to be a pretty experienced developer with time on your hands to build these products.
You would need to have good understanding of sessions, proxies and scraping just to name a few.
The best way to get started would be to create a bot...
How much data are you trying to get from YP?
I've written a custom scraper for the Au Yellow Pages, gets phone/email without needing to load additional pages ( just the search result pages).
Happy to help out if you need!
HTTrack is used for cloning websites, but I don't think that's what you're looking for. To save/download streams I'd try out youtube-dl it works for a lot more than just youtube.
What information are you after?
Binance have an API, so you might not need to scrape the data at all. That would still require some custom coding, I've worked with Binance's API and also written custom scrapers in python.
PM me if you want any more info :)
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.