1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Which is the best content scraper youve used?

Discussion in 'Blogging' started by waitier, Apr 14, 2016.

  1. waitier

    waitier Power Member

    Joined:
    Sep 5, 2013
    Messages:
    738
    Likes Received:
    202
    I have a need to scrape articles from custom sites whereby I just input the URL, it will then scrape all the links with the title+content and then output into a format that i can just upload to wordpress. If you want to call it autoblogging, then that's what it is.

    I plan to re post sources found in places like techrunch or any other major sites as a post in my own site.

    I found a bunch of them online and was unsure which ones does the job with ease, something like a one click button?

    Webharvy
    WP robot
    Cyberseo
    ..And a bunch of wp plugins found in official repo..

    I've tried mostly the free ones and their limitation and bugs is just way too annoying. Some of them managed to take the RSS feed and extract them to full text, while some kept it as a freemium feature which i hate the most.

    RSS feed to post type of plugins worked to a certain extend, but in the text sometimes ends up jumbled up and left garbage text in there which i have to manually clean it up defeating the purpose.
     
  2. Chiefjop

    Chiefjop Jr. VIP Jr. VIP

    Joined:
    Mar 25, 2013
    Messages:
    570
    Likes Received:
    104
    If you know a little coding; scrapy.org is the way to go :)
     
  3. SEO FOX

    SEO FOX Jr. VIP Jr. VIP

    Joined:
    Apr 27, 2015
    Messages:
    3,220
    Likes Received:
    656
    Gender:
    Male
    Location:
    Infront Of U!!
    Home Page:
    Import.io is a browser based web scraping tool, give it a try.
     
  4. waitier

    waitier Power Member

    Joined:
    Sep 5, 2013
    Messages:
    738
    Likes Received:
    202
    Oh i forgot to mention that one as well. These visual web scraper they are one thing, it says it's easy, but it's so complicated at the same time.

    Technically the template i needed just isn't there. It seems easy, or at least it's the easiest thing and most common thing any article scrapper can spit out.

    I've tried to add the elements using import.io, point and click, but umm they don't click into the links to extract the body content. Can't find that function either.. am i missing something?
     
  5. xpression50

    xpression50 Jr. VIP Jr. VIP

    Joined:
    Sep 17, 2010
    Messages:
    170
    Likes Received:
    80
    Occupation:
    Web Design, Internet Marketing, SEO
    Home Page:
    Content foundry is my favourite, It allows you to scrape from loads of websites and has built in proxy support. Only downside is it has a monthly fee.
     
  6. igvlad

    igvlad Junior Member

    Joined:
    Nov 24, 2015
    Messages:
    169
    Likes Received:
    10
    Import.io will be good , they hav also free package
     
  7. Crazy Monkey

    Crazy Monkey Jr. VIP Jr. VIP

    Joined:
    Aug 4, 2015
    Messages:
    1,951
    Likes Received:
    239
    Gender:
    Male
    Location:
    In Jungle
    I suggested tool work will be good
     
  8. Devil Rider

    Devil Rider BANNED BANNED

    Joined:
    Jul 24, 2015
    Messages:
    554
    Likes Received:
    59
    You can select a specific data point, a price, a rating etc and then use your browser menu: click Scrape Similar and you will get multiple options to export or copy your data to Excel or Google Docs. This plugin is really basic but does the job it is built for fast and easy screen scraping.
     
  9. Ninjaseotools

    Ninjaseotools Junior Member

    Joined:
    Apr 3, 2016
    Messages:
    101
    Likes Received:
    9
    Gender:
    Male
    Occupation:
    Running a IM Company
    Location:
    India
    Home Page:
    Yea Content foundry is my also Best