1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Setup for managing large amounts of TB's?

Discussion in 'BlackHat Lounge' started by Capo Dei Capi, Feb 14, 2015.

  1. Capo Dei Capi

    Capo Dei Capi BANNED BANNED

    Joined:
    Oct 23, 2014
    Messages:
    754
    Likes Received:
    7,211
    I currently have 7 harddrives with 7.1TB of external HDD storage and 2 external HDD with an aditional 7TB for a total of 14.1 TB of storage. What kind of setup should I go for to back my files up as well as manage the files easier so I can get rid of duplicate files as well as make it easier to add more storage in the future.

    2 of my harddrives are going to be 6 years old in a few weeks so I'm kinda of getting worried about data loss.
     
  2. neu009

    neu009 Jr. VIP Jr. VIP

    Joined:
    Jul 29, 2009
    Messages:
    1,150
    Likes Received:
    309
    Well you could buy the 5TB and larger and run them in Raid. Usually unlikely for both to fail at the same time. That is of course if you value your files more than the additional cost. I'd use the oldest drives for the not so important stuff that you can easily re-download.
     
    • Thanks Thanks x 1
  3. silvermember

    silvermember Regular Member

    Joined:
    Apr 16, 2013
    Messages:
    242
    Likes Received:
    87
    Location:
    Chained on Earth Gravity
    bloody hell and i thought i was crazy lol

    I suggest before you get the plates on a new system copy your stuff on new HDD's, I have once lost a external plate by just lying around for month but that could be a manufacture failure as well!! However I suggest best for duplicate files is to remove them manually ( yeah a lot of work for you now) I always be behind that and do quality checks so I safe space in first place. As well you could cloud it but I guess this is not an option as it would be the case for most of us.

    be safe and good luck
     
    • Thanks Thanks x 1
  4. davids355

    davids355 Jr. VIP Jr. VIP

    Joined:
    Apr 25, 2011
    Messages:
    10,397
    Likes Received:
    8,099
    How have you got so much data? is it all constantly used? Can you get it into the cloud? Good storage would be expensive. Best way would be file server with raid.

    or hosted equivilant if you have a fast connection.
     
  5. Capo Dei Capi

    Capo Dei Capi BANNED BANNED

    Joined:
    Oct 23, 2014
    Messages:
    754
    Likes Received:
    7,211
    I think I may go with 4TB since they are cheaper,plus I've seen better reviews.

    Are Surveillance HDD's good for long term data storage compared to NAS storage drives

    http://www.newegg.com/Product/Product.aspx?Item=N82E16822178505&cm_re=4TB-_-22-178-505-_-Product

    vs

    http://www.newegg.com/Product/Product.aspx?Item=N82E16822145912&cm_re=4TB-_-22-145-912-_-Product#top
     
  6. Capo Dei Capi

    Capo Dei Capi BANNED BANNED

    Joined:
    Oct 23, 2014
    Messages:
    754
    Likes Received:
    7,211
    I probably have about 9TB of it used up, and since my home internet connection can only do 4mbps upload(But I only get around 3.6mbps stable over a long period) It would take me about 225 days to upload to a storage provider not counting slow downs,power outages etc. I've been thinking about RAID, but some people have claimed that RAID isn't good for storage.
     
  7. BillyPart

    BillyPart Newbie

    Joined:
    Jan 23, 2015
    Messages:
    14
    Likes Received:
    52
    RAID stripe is bad because if you lose one drive you lose ALL the data on all drives. RAID 10 (stripe + mirror) is good for both speed and data redundancy. I would discourage any consideration of external USB drives as they are unreliable.

    You might look into an NAS. Also consider most HD's die as a result of bad power.
     
    • Thanks Thanks x 1
  8. Capo Dei Capi

    Capo Dei Capi BANNED BANNED

    Joined:
    Oct 23, 2014
    Messages:
    754
    Likes Received:
    7,211
    I think I will go with a 10 bay hot swap enclosure, since I'll have room for up to 60TB.

    http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=8066427