1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scrape content off list of urls? Software that does this?

Discussion in 'Black Hat SEO' started by Bostoncab, Aug 8, 2013.

  1. Bostoncab

    Bostoncab Elite Member

    Joined:
    Dec 31, 2009
    Messages:
    2,255
    Likes Received:
    514
    Occupation:
    pain in the ass cabbie
    Location:
    Boston,Ma.
    Home Page:
    I want to generate lists of urls with scrapebox an then feed them into another tool that visits the urls scrapes all the content and puts them into a text file. I know scrapebox does metadata but not the actual words on the page.
     
  2. dinkish

    dinkish Power Member

    Joined:
    Apr 19, 2013
    Messages:
    689
    Likes Received:
    159
    Look into the Article Scraper plugin for scrapebox. Might be worth a shot, may just be worth shit.
     
    • Thanks Thanks x 1
  3. royserpa

    royserpa Jr. VIP Jr. VIP Premium Member

    Joined:
    Sep 28, 2011
    Messages:
    4,657
    Likes Received:
    3,498
    Gender:
    Male
    Occupation:
    Negative Options aka Rebills!
    Location:
    Royserpa
    Home Page:
    I have done this on php.
    Pm me if you are willing to pay.
     
  4. Bostoncab

    Bostoncab Elite Member

    Joined:
    Dec 31, 2009
    Messages:
    2,255
    Likes Received:
    514
    Occupation:
    pain in the ass cabbie
    Location:
    Boston,Ma.
    Home Page:
    Depends how much and how effective it is.
     
  5. satyr85

    satyr85 Power Member

    Joined:
    Aug 7, 2011
    Messages:
    580
    Likes Received:
    445
    Location:
    Poland
    Zennoposter.
    Doable with demo version but it will be slow. With paid versions you can do it in no time.
     
  6. Bostoncab

    Bostoncab Elite Member

    Joined:
    Dec 31, 2009
    Messages:
    2,255
    Likes Received:
    514
    Occupation:
    pain in the ass cabbie
    Location:
    Boston,Ma.
    Home Page:
    scrapebox article scraper scrapes from predefined sources not from list of url
     
  7. royserpa

    royserpa Jr. VIP Jr. VIP Premium Member

    Joined:
    Sep 28, 2011
    Messages:
    4,657
    Likes Received:
    3,498
    Gender:
    Male
    Occupation:
    Negative Options aka Rebills!
    Location:
    Royserpa
    Home Page:
    It is really easy. B ut also depends on the content. If we are talking about images only, will be another entirely different game.
     
  8. dinkish

    dinkish Power Member

    Joined:
    Apr 19, 2013
    Messages:
    689
    Likes Received:
    159
    fgets and curl functions would probably do the bulk of the work in PHP. I would run a php server locally, or find a VPS with it installed if you were going that route though. Server load could be substantial.
     
  9. Bostoncab

    Bostoncab Elite Member

    Joined:
    Dec 31, 2009
    Messages:
    2,255
    Likes Received:
    514
    Occupation:
    pain in the ass cabbie
    Location:
    Boston,Ma.
    Home Page:
    I wound up buying a custom vb bot that does it.. works pretty damn good.
     
  10. lmxftw

    lmxftw BANNED BANNED

    Joined:
    Mar 11, 2013
    Messages:
    1,010
    Likes Received:
    1,179
    care2share ?

     
  11. gullsinn

    gullsinn Jr. VIP Jr. VIP Premium Member

    Joined:
    Dec 24, 2009
    Messages:
    2,429
    Likes Received:
    2,210
    Gender:
    Male
    Occupation:
    Jobless :D
    Location:
    Graveyard
    Home Page:
    You can also get Get Article Lite bot for articles scraping.
     
  12. qlwik

    qlwik Newbie

    Joined:
    Aug 18, 2011
    Messages:
    48
    Likes Received:
    9
    Zennoposter will do it easy, I can even do it for You.
     
    • Thanks Thanks x 1
    Last edited: Aug 13, 2013