Recent content by Gazo

It's not illegal. There is a protocol called robots.txt that literally tells your crawler which sites it can crawl on the website. Even Blackhat World has a robots.txt file (https://www.blackhatworld.com/robots.txt) It only becomes illegal when you STEAL information from other websites. For...

I did not describe it in the post, so good question. What I normally do in these cases is use the PhantomJS web driver in Selenium. PhantomJS is a lightweight headless browser that renders Javascript, so you get the best of both worlds. A light weight scraper, and the power of Selenium.

With a bit of tweaking this can do that, but not out of the box.

Did you set the PROXY_ORBIT_TOKEN environment variable?

When I have to check proxies myself without a checker I setup a web server and make a request through the proxy to the web server. The I then check the headers of the incoming request. Some proxy servers have a header called X-Forwarded-For which contains the original IP address of the request...

Thank you for the kind works, much appreciated.

Overview Most Python web crawling/scraping tutorials use some kind of crawling library. This is great if you want to get things done quickly, but if you do not understand how scraping works under the hood then when problems arise it will be difficult to know how to fix them. In this tutorial I...

When it comes to these kinds of questions it comes down to what you personally are more drawn to. Both languages will do exactly what you need, and there are plenty of jobs that would hire you for knowing either language too, if you'd want to be a developer at some point. If you have never...

Not a problem, any time you need some Python help just hit me up.

I'd start looking into some tutorials about writing Python code. It is a fairly easy language to learn and has some very powerful features. When you feel like you have the basics look into a library called Selenium. Selenium allows you to automate your browser, like a kind of advanced iMacros...

I think I understand what you are trying to do here, and there is probably an easier way to do it, but trying to go about it the way you did I came up with this: from multiprocessing import Pool, Manager def test(obj): counter, item = obj counter_val = counter.get()...

You have to work hard. Maybe try to get into a University in Canada, network with people that can get you closer. No one is going to hand something like that to you. The only way you will reach your dream is by working hard to get there. Set goals for yourself. Make a timeline. Map out what you...