1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

File Scraper

Discussion in 'Black Hat SEO Tools' started by Rougemont, Oct 28, 2011.

  1. Rougemont

    Rougemont Registered Member

    Joined:
    May 23, 2011
    Messages:
    75
    Likes Received:
    19
    I've been looking around but I haven't seen a tool that does quite what I'm looking for, and if it's listed on this site I didn't find it in my search. I'm looking for a tool where you can specify a file type and a domain and have it download all of those that it sees. I've got Scrapebox and I was thinking maybe there would be some kind of footprint setting that would do this that I'm just too new to know about. I mean, I know you can get URLs, but I want it to actually download the files too. Anybody have any suggestions?
     
  2. scraper1

    scraper1 Regular Member

    Joined:
    May 28, 2011
    Messages:
    214
    Likes Received:
    207
    Location:
    Kontiki
    Home Page:
    I don't use Scrapebox so I don't know if it can download files too, but this could be a start, for example:

    Code:
    site:www.blackhatworld.com filetype:pdf
     
  3. Stalli0n

    Stalli0n Junior Member

    Joined:
    Nov 17, 2010
    Messages:
    115
    Likes Received:
    83
    Location:
    Europe
    Google for Web Crawler.

    I used one a few months ago, but forgot the name.
    I'll check if i still have it.
     
  4. DX-GENERATION

    DX-GENERATION Jr. VIP Jr. VIP Premium Member

    Joined:
    Apr 14, 2010
    Messages:
    1,151
    Likes Received:
    254
    use scrapebox and one more software i came across in another forum it was in beta testing it is also nice one :)
     
    • Thanks Thanks x 1
  5. charmol

    charmol Junior Member

    Joined:
    Oct 7, 2011
    Messages:
    173
    Likes Received:
    84
    Location:
    ~| TrOpiCaL PaRaDiSe |~
    I use some software I got off a torrent site, called Teleport Pro,

    It can search a site for files of a certain type, download an entire site to your HDD or just download the structure of the site. I'm sure it has more functions, but those are the main ones I use.

    There is a lot of other software out there for doing the same thing, like someone above me said, search google for data mining software or web crawler or similar.

    Hope this helps

    ~|ChaRoN|~
     
  6. Rougemont

    Rougemont Registered Member

    Joined:
    May 23, 2011
    Messages:
    75
    Likes Received:
    19
    Thanks for the tips. I tried downloading a copy of Teleport Pro, but it modified and hid my host file then hijacked my Google results. Fortunately I had a copy of my host file!

    Anyway, I'll keep looking. At least it's good to know that these things are out there.

    I also emailed Scrapebox about it. They said they'd pass the suggestion on to the development team. It would be awesome if that made it into the next release or as an addon.
     
  7. softtouch2009

    softtouch2009 Senior Member

    Joined:
    Dec 2, 2009
    Messages:
    1,001
    Likes Received:
    225
    Occupation:
    Programming
    Location:
    ssdnet.biz
    Home Page:
    So you mean, you have a list or urls in your ScrapeBox harvester grid, which point to the files you want to download (like http: // domain.com/mytext.pdf or such)? All you want then is a button to actually download the files to your hdd?