1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scraping software - which is the easiest/fastest with JS support?

Discussion in 'Black Hat SEO' started by dinkerdok, Jan 30, 2017.

  1. dinkerdok

    dinkerdok Newbie

    Joined:
    Feb 4, 2008
    Messages:
    34
    Likes Received:
    4
    Hi,

    I'm a developer looking into scraping software that is Fast, Easy to deploy and can read Javascript (sorry I've been away a few years and a lot has changed in the time I last looked).

    Now we have selenium, python (beautifulsoup) - this seems like a nice combination - but... SLOW - selenium is slow.

    Is there a better way to scrape nowadays.

    For instance, there seem so many tools its just overwhelming like here for instance - github.com/adw0rd/awesome-web-scraping

    I've so far seen the use of NodeJS and a headless browser - which seems pretty simple - anyone using this?

    From other research it seems also that Selenium has moved further forward now and has great dom capabilities so beautifulsoup may no longer needed? Any way to speed up selenium if this is the case?
     
  2. SEOMadHatter

    SEOMadHatter Junior Member

    Joined:
    Aug 15, 2015
    Messages:
    161
    Likes Received:
    30
    You posted in the SEO section you'd maybe get a better response here.
     
  3. dinkerdok

    dinkerdok Newbie

    Joined:
    Feb 4, 2008
    Messages:
    34
    Likes Received:
    4
    You're right thanks :) Ill post it in there, hope it can be deleted here
     
  4. extremeboy

    extremeboy Jr. VIP Jr. VIP

    Joined:
    Jul 8, 2010
    Messages:
    3,185
    Likes Received:
    667
    Occupation:
    World Best RANK Tracker SERPCloud.com
    Home Page:
    use phantom js and casper js might help you it's faster and JS framework for browser simulation faster than Selenium hope it helps.
     
  5. dinkerdok

    dinkerdok Newbie

    Joined:
    Feb 4, 2008
    Messages:
    34
    Likes Received:
    4
    It is indeed big helpings thank you :)