1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

recommend a good site spider/downloader

Discussion in 'BlackHat Lounge' started by scubaslick, Jul 25, 2008.

  1. scubaslick

    scubaslick Regular Member

    Joined:
    Aug 23, 2007
    Messages:
    392
    Likes Received:
    512
    I've gotten an interesting little break. Got a login/pass for a membership site (training, information, videos etc.) that's good for a few days. I'd really like to just copy the entire site and peruse at my leisure.

    I've never DLd a whole site before so I tried httrack but I don't want to spend 6 hours learning how to use it to get around the security of the site.

    I need something that will run through my browser so I don't have to dick with security settings (or something that will remember login/pass and not make me babysit it). I have all of the main browsers and will use whichever.

    I just want it to follow every link on every page that stays on the TLD, record html and images. Anything off site just ignore.

    Any ideas?

    And yes, a few nice shares will likely result from this (although there may not be anything worth sharing after all is said and done).

    Thanks peeps,
     
  2. voxclan

    voxclan Regular Member

    Joined:
    Feb 11, 2008
    Messages:
    267
    Likes Received:
    274
    scubaslick,

    I think you should look at Scrapbook. It's a Firefox extension and I am using it on a regular basis and love it.

    You can scrape forum, images, video files, pdf and more.

    This is one of my Must Have tool, and free.

    Here is the official firefox page for more info

    https://addons.mozilla.o**rg/en-US/firefox/addon/427
     
  3. scubaslick

    scubaslick Regular Member

    Joined:
    Aug 23, 2007
    Messages:
    392
    Likes Received:
    512
    Thanks Voxclan, but it's manual. I don't want to have to visit every page personally and click the button. This site has hundreds of pages and honestly I don't have time to do all that.

    Appreciate the share though. That's a great little tool and I'll definitely be using it now.
     
  4. voxclan

    voxclan Regular Member

    Joined:
    Feb 11, 2008
    Messages:
    267
    Likes Received:
    274
    scubaslick,

    No it's not manual, it has automation also. My time is also valuable...and I would not do it manually. Take a closer look at the "capture page as" fonction and you will see.


    I downloaded forum with thousands of pages with a few clicks.