1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

(Free) tool to scrape articles from a specific site

Discussion in 'Black Hat SEO Tools' started by iliketurtles, Sep 27, 2013.

  1. iliketurtles

    iliketurtles Regular Member

    Joined:
    Dec 10, 2008
    Messages:
    283
    Likes Received:
    10
    Home Page:
    I've found a good site I would like to scrape, and then translate the articles to my language. There are hundreds of articles so it'll take hours and hours to copy and paste. So do you know of a tool I can use to scrape this one site? Wouldn't mind if it's free, cuz I will most likely use it like once a year or something, if that.
     
  2. zacatictac

    zacatictac Power Member

    Joined:
    May 2, 2010
    Messages:
    584
    Likes Received:
    751
    Occupation:
    SEO
    Location:
    Metaverse
    As a programmer I can say it all depends on the site. To parse data out of a site you need to use a regex to pick out the article from the html. I won't bore you with the details but if it's a rather unheard of site chances are there is no tool for it.
     
  3. iliketurtles

    iliketurtles Regular Member

    Joined:
    Dec 10, 2008
    Messages:
    283
    Likes Received:
    10
    Home Page:
    Crap.. One article a time it is, then. :prof10:
     
  4. phil7346

    phil7346 Registered Member

    Joined:
    Jun 19, 2011
    Messages:
    75
    Likes Received:
    26
    been looking into this a little, but haven't got far yet :). I was looking at going into wordpress. My plan was to use httrack to download the pages and then use 'html import 2' plugin to bring it in. The plugin needs you to be able to specify the div tag that the content is in.
     
  5. zacatictac

    zacatictac Power Member

    Joined:
    May 2, 2010
    Messages:
    584
    Likes Received:
    751
    Occupation:
    SEO
    Location:
    Metaverse
    If you need a custom scraper for a site let me know.
     
  6. iliketurtles

    iliketurtles Regular Member

    Joined:
    Dec 10, 2008
    Messages:
    283
    Likes Received:
    10
    Home Page:
    Do you make custom scrapers? How much will that cost?
     
  7. zacatictac

    zacatictac Power Member

    Joined:
    May 2, 2010
    Messages:
    584
    Likes Received:
    751
    Occupation:
    SEO
    Location:
    Metaverse
    yea i do. it would cost what you can afford really. i enjoy programming scrapers. you can get ahold of me at arsynseo.com if your interested
     
  8. leet0r

    leet0r Newbie

    Joined:
    Sep 27, 2013
    Messages:
    11
    Likes Received:
    1
    just use Imacros for simple steps like this ;)
     
  9. iliketurtles

    iliketurtles Regular Member

    Joined:
    Dec 10, 2008
    Messages:
    283
    Likes Received:
    10
    Home Page:
    How you do that?
     
  10. leet0r

    leet0r Newbie

    Joined:
    Sep 27, 2013
    Messages:
    11
    Likes Received:
    1
    download the Imacros plugin for ff browser and do the steps..
     
    • Thanks Thanks x 1
  11. iliketurtles

    iliketurtles Regular Member

    Joined:
    Dec 10, 2008
    Messages:
    283
    Likes Received:
    10
    Home Page:
    The plugin let me download the content on a certain page, but i still gotta do it one page at the time, and all the formatting is lost.
     
  12. leet0r

    leet0r Newbie

    Joined:
    Sep 27, 2013
    Messages:
    11
    Likes Received:
    1
    Yeah sure it´s a free plugin .. to do this multithreaded (more pages at same time) you Need a custom bot. Don´t know why the formatting is gone..
    If you still Need a bot I could do it ;)