Scrape content off list of urls? Software that does this?

Discussion in 'Black Hat SEO' started by Bostoncab, Aug 8, 2013.

  1. Bostoncab

    Bostoncab Elite Member

    Joined:
    Dec 31, 2009
    Messages:
    2,256
    Likes Received:
    514
    Occupation:
    pain in the ass cabbie
    Location:
    Boston,Ma.
    I want to generate lists of urls with scrapebox an then feed them into another tool that visits the urls scrapes all the content and puts them into a text file. I know scrapebox does metadata but not the actual words on the page.
     
  2. dinkish

    dinkish Power Member

    Joined:
    Apr 19, 2013
    Messages:
    689
    Likes Received:
    159
    Look into the Article Scraper plugin for scrapebox. Might be worth a shot, may just be worth shit.
     
    • Thanks Thanks x 1
  3. royserpa

    royserpa Jr. VIP Jr. VIP Premium Member

    Joined:
    Sep 28, 2011
    Messages:
    5,103
    Likes Received:
    3,857
    Gender:
    Male
    Occupation:
    Negative Options aka Rebills!
    Location:
    Exploiting Loopholes!
    Home Page:
    I have done this on php.
    Pm me if you are willing to pay.
     
  4. Bostoncab

    Bostoncab Elite Member

    Joined:
    Dec 31, 2009
    Messages:
    2,256
    Likes Received:
    514
    Occupation:
    pain in the ass cabbie
    Location:
    Boston,Ma.
    Depends how much and how effective it is.
     
  5. satyr85

    satyr85 Jr. VIP Jr. VIP

    Joined:
    Aug 7, 2011
    Messages:
    682
    Likes Received:
    518
    Location:
    Poland
    Zennoposter.
    Doable with demo version but it will be slow. With paid versions you can do it in no time.
     
  6. Bostoncab

    Bostoncab Elite Member

    Joined:
    Dec 31, 2009
    Messages:
    2,256
    Likes Received:
    514
    Occupation:
    pain in the ass cabbie
    Location:
    Boston,Ma.
    scrapebox article scraper scrapes from predefined sources not from list of url
     
  7. royserpa

    royserpa Jr. VIP Jr. VIP Premium Member

    Joined:
    Sep 28, 2011
    Messages:
    5,103
    Likes Received:
    3,857
    Gender:
    Male
    Occupation:
    Negative Options aka Rebills!
    Location:
    Exploiting Loopholes!
    Home Page:
    It is really easy. B ut also depends on the content. If we are talking about images only, will be another entirely different game.
     
  8. dinkish

    dinkish Power Member

    Joined:
    Apr 19, 2013
    Messages:
    689
    Likes Received:
    159
    fgets and curl functions would probably do the bulk of the work in PHP. I would run a php server locally, or find a VPS with it installed if you were going that route though. Server load could be substantial.
     
  9. Bostoncab

    Bostoncab Elite Member

    Joined:
    Dec 31, 2009
    Messages:
    2,256
    Likes Received:
    514
    Occupation:
    pain in the ass cabbie
    Location:
    Boston,Ma.
    I wound up buying a custom vb bot that does it.. works pretty damn good.
     
  10. lmxftw

    lmxftw BANNED BANNED

    Joined:
    Mar 11, 2013
    Messages:
    1,010
    Likes Received:
    1,179
    care2share ?

     
  11. gullsinn

    gullsinn Elite Member

    Joined:
    Dec 24, 2009
    Messages:
    2,442
    Likes Received:
    2,263
    Gender:
    Male
    Occupation:
    Jobless :D
    Location:
    Graveyard
    You can also get Get Article Lite bot for articles scraping.
     
  12. qlwik

    qlwik Newbie

    Joined:
    Aug 18, 2011
    Messages:
    48
    Likes Received:
    9
    Zennoposter will do it easy, I can even do it for You.
     
    • Thanks Thanks x 1
    Last edited: Aug 13, 2013