1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

trial of web scrapers?

Discussion in 'Black Hat SEO' started by [mcr], Jul 10, 2013.

  1. [mcr]

    [mcr] Junior Member

    Joined:
    Sep 19, 2009
    Messages:
    114
    Likes Received:
    0
    What are the best web scrapers, and are there any that offer a trial before buying?

    Thanks.
     
  2. echizen

    echizen Newbie

    Joined:
    May 26, 2012
    Messages:
    45
    Likes Received:
    13
    Home Page:
    what web and what data you want to scrap?
     
  3. [mcr]

    [mcr] Junior Member

    Joined:
    Sep 19, 2009
    Messages:
    114
    Likes Received:
    0
    There's a website I want to scrape, it requires login. But when I look at the source the info displayed on the page isn't there. It uses javascript to display the info. So the scraper would have to be able to parse js.

    What are the best out there? scrapebox? What else?
     
  4. [mcr]

    [mcr] Junior Member

    Joined:
    Sep 19, 2009
    Messages:
    114
    Likes Received:
    0
    Anyone with any insight on this? Thanks!
     
  5. dummydecoy

    dummydecoy Junior Member

    Joined:
    Jul 4, 2010
    Messages:
    154
    Likes Received:
    39
    zennoposter or ubot

    or if you're a programmer , try PhantomJS
     
    • Thanks Thanks x 1
  6. [mcr]

    [mcr] Junior Member

    Joined:
    Sep 19, 2009
    Messages:
    114
    Likes Received:
    0
    Good stuff, thank you. ubot seems solid.

    Has anyone used scrapebox and ubot and zennoposter and can compare the three

    I want to buy one, I just need to be sure I'm buying the right one. This would be so much easier if they provided trial versions
     
  7. Banner Jackson

    Banner Jackson Newbie

    Joined:
    Jul 29, 2013
    Messages:
    1
    Likes Received:
    0
    I would recommend phantomjs. It is free. It is command line based, but loads complete webpages because it is a web browser. You can write javascript based scripts to get data from the dom which can sometimes easier than writing regular expressions. the only shortcoming I have found in the newerversions is that they do not support plugins meaning they will not load Flash content. I think i saw somebody had created a fork of phantomjs though, adding back in that functionality.

    It supports clicking through multiple pages, submitting forms, etc. Very versatile for scraping. Hit me up if you need help with any scripts.
     
  8. keval007

    keval007 Junior Member

    Joined:
    Jun 12, 2012
    Messages:
    145
    Likes Received:
    26
    Occupation:
    Web Scraper & PHP Developer
  9. paulblack2025

    paulblack2025 Newbie

    Joined:
    Jul 8, 2016
    Messages:
    15
    Likes Received:
    1
    Gender:
    Male
    Hi,

    If you are a programmer, I would like to recommend phantomjs.
    If you are a non programmer, some web scrapers
    like :http://www.octoparse.com/ or http://www.uipath.com/ seem fine.
    I used the former and works well for scraping data behind login and dropdown menu.

    Plus Thereā€˜s no way collecting data from a site that requires a login and password using import.io.
    I used Import.io before I switch to OP. We ended up writing our own script when using import.io.

    You may look into web scraper like these if u want.
     
  10. arc323

    arc323 Junior Member

    Joined:
    Sep 23, 2015
    Messages:
    100
    Likes Received:
    25
    Location:
    Denver, CO
    Home Page:
    uBot lets you build a custom scraper. I suggest getting the pro version and skipping over the entry level version. You get some some really important features with the pro version that you don't get with the basic version. If you have some basic programing knowledge you will be very successful with uBot. Hell, I didn't have any programming knowledge when I started with uBot and I've already made some really great bots that save me a ton of time.

    I use Scrapebox every day. I don't think it will be useful to you for this as you have to be logged into the website to get the data.

    Never used Zennoposter and I'm not a programmer so I can't comment on PhantomJS.

    Seems like uBot might be the answer for you if you're not a programmer since it can login to sites and scrape data. Good luck!