Phantomjs Google scraper using antigate

Javardo69 said:

the captcha its just a text code or its that new Recaptcha where you have to select pictures?

I've only used phantom js to browser automation (because it is headless), but if possible i try to avoid, send me a pm message with the link and i'll tell you if i can do something or not

Click to expand...

terebi said:

Captcha is standard re-captcha (not selecting pictures).

Code should be based on this:
https://github.com/PavelPolyakov/parsing-with-php/blob/master/5_phantomjs_antigate/antigate.js

Remember the page being visited is a Google url

Click to expand...

Javardo69 said:

That is a code to send a captcha to antigate and get the captcha solution text, antigate has 11 languages examples to use their API as you can see here https://anti-captcha.com/apidoc if you skip intro and go for code examples, here's for python for instance https://github.com/gotlium/antigate.

I've made a small script to scrape the first 20 images of google images on python without the need of automating a browser just replicating the calls that the browser does, thats why i was asking for the google page that you want to scrape and what you want to be scraped. I asked if the re-captcha was that one of selecting pictures because i have no solution for that but if it is the standard with a image and answer with a text antigate service should be enough to tackle the problem.

Click to expand...

MrBlue said:

Interesting but I've never been hit rate limiting when scraping Google using headless browsers like PhantomJS or CasperJS. How many requests are you making per second? Are you using proxies? Have you correctly set all your headers?

Click to expand...