Javascrip scrapping

Discussion in 'General Programming Chat' started by ultra.marine, Feb 1, 2013.

  1. ultra.marine

    ultra.marine Registered Member

    Joined:
    Oct 5, 2012
    Messages:
    80
    Likes Received:
    105
    Location:
    Macedonia
    Hello guys, anyone know of good api that can scrape javascript pages[either .net or java] that is fast enough for scrapping tasks/loading the pages with scrips ? Thanks!
     
    Last edited: Feb 1, 2013
  2. MrBlue

    MrBlue Senior Member

    Joined:
    Dec 18, 2009
    Messages:
    975
    Likes Received:
    682
    Occupation:
    Web/Bot Developer
    PhantomJS
    Code:
    http://phantomjs.org/
    +
    CasperJS
    Code:
    http://casperjs.org/
     
    • Thanks Thanks x 1
  3. ultra.marine

    ultra.marine Registered Member

    Joined:
    Oct 5, 2012
    Messages:
    80
    Likes Received:
    105
    Location:
    Macedonia

    Thanks man, I'll try them during this day :)
     
  4. fastslip

    fastslip Newbie

    Joined:
    Jun 7, 2012
    Messages:
    8
    Likes Received:
    0
    NodeJS
     
  5. s0ap

    s0ap Executive VIP Jr. VIP Premium Member

    Joined:
    Sep 23, 2008
    Messages:
    230
    Likes Received:
    831
    Occupation:
    :] guess
    Location:
    Congo/DRC
    It is SCRAPING. One 'p'. Every time someone posts this it sounds like they are looking for a street fight behind the bar downtown with a programming language.
     
    • Thanks Thanks x 3
  6. theMagicNumber

    theMagicNumber Regular Member

    Joined:
    May 13, 2010
    Messages:
    347
    Likes Received:
    195
    These libraries are coded in JS for web automation and testing. The OP wanted something that will execute/emulate the JS in the html pages using .NET or JAVA.

    I don't know for .NET but you can use http://htmlunit.sourceforge.net/ for Java
    I don't use it myself, but heard good things about the library.
     
    • Thanks Thanks x 1
    Last edited: Feb 6, 2013
  7. ultra.marine

    ultra.marine Registered Member

    Joined:
    Oct 5, 2012
    Messages:
    80
    Likes Received:
    105
    Location:
    Macedonia

    Thanks, that was exactly what I was looking for!
     
  8. botrockets

    botrockets Regular Member

    Joined:
    Mar 16, 2013
    Messages:
    373
    Likes Received:
    570
    Occupation:
    Software Developer & Entrepreneur
    Location:
    .
    for .net : selenium webdriver, inbuilt browser control