Personal computing discussed

Moderators: renee, morphine, Steel

 
K-L-Waster
Gerbil XP
Topic Author
Posts: 498
Joined: Thu Feb 12, 2015 8:10 pm
Location: Hmmm, I was *here* a second ago...

Ageing drives, NAS, and Backup Strategy Rethink

Thu Nov 17, 2016 12:48 pm

Hello, Gerbils,

I think I need to revise my storage and backup strategy, and would appreciate any insights the community can provide. Basically, I have a number of aging drives in my main system and a NAS that is starting to fill up. I'm looking to ensure that data is better protected and easier to locate.

My current setup is:

* System drive: 1TB SSD
* 1 x 2TB WD Green HDD for primary data storage
* 1 x 1 TB WD Black HDD -- was the system drive prior to the SSD, now also primary data storage
* 2 x 1TB Seagate HDDs for internal data file "backup" (somewhat under-utilized)
* NAS with 2 x 4TB Seagate NAS drives in RAID 1 in a Shuttle Omninas KD21 -- used as shared storage and backup destination. 
(Please note: the RAID setup is in place to ensure I can remain functional if one of the NAS drives dies, but I consider this a "backup" because it contains copies of files from the main system, not simply because it's RAID. RAID /= Backup.)

Backups are performed once per week using 2nd Copy, and basically consist of copying files from the primary location to a secondary location so that I have at least 2 copies of each file each on different physical media. Files that I am particularly attached to get copied to both the internal Seagate 1TB drives and to the NAS so that if a drive goes I have 2 other copies.

I have a few concerns with this setup.

1) The WD Black and Green drives are both over 6 years old with pretty much daily use over that time. Neither has shown any signs of imminent failure, but given their age and amount of use I'm thinking they may be on borrowed time.

2) The NAS isn't full but it's getting there. 

3) My original plan was to stick to 1TB drives so that if one died, I would only be at risk of losing 1TB at most. However, finding the files I am after in one of 6 drive volumes is getting old, and I'm thinking I would rather have a smaller number of larger volumes.

As I see it, my options are:

a) Replace the NAS drives with larger ones (8TB or 10TB) and repurpose the existing NAS drives as internal storage (consolidating the 4 existing internal HDDs into 2). 
PRO: consolidates volumes and eliminates the aging drives 
CON: fairly expensive (2 x 8TB WD Reds or equivalent Seagate NAS drives will run over $900 CDN after sales tax and shipping)

b) Stop using the NAS as backup and use an alternative backup drive, thus freeing up space on the NAS. This would require buying another external backup solution such as a USB drive. 
PRO: less expensive than increasing the NAS capacity
CON: still leaves multiple drive volumes and old drives in use

Any recommendations appreciated. I also would be interested in how other gerbils are managing their data and backups. 
Main System: i7-8700K, ASUS ROG STRIX Z370-E, 16 GB DDR4 3200 RAM, MSI GTX 1080 TI, 1 TB CRUCIAL MX500, Corsair 550D

HTPC: I5-4460, ASUS H97M-E, 8 GB RAM, GTX 970, CRUCIAL 256GB MX100, SILVERSTONE GD09B
 
PainIs4ThaWeak1
Gerbil
Posts: 77
Joined: Fri Jun 26, 2009 11:13 am

Re: Ageing drives, NAS, and Backup Strategy Rethink

Thu Nov 17, 2016 2:19 pm

Personally, I enjoy the flexibility that a roll-your-own NAS provides (eg. FreeNAS, or the like), because it offers much more expandability options, performs media scrubs, takes snapshots, allows for installation of plugins (Plex, CouchPotato, etc.), and utilizes the ZFS file system (for FreeNAS, at least. And that's just its main selling points for me. The amount of configurable options within the FreeBSD based FreeNAS is pretty impressive)

I understand the initial cost is high (server chipset type board preferred, along with ECC memory, etc etc.), but I'll never go back to what I had (which was similar to what you currently have.)
Anywho, IMHO, the most cost effective option, while still being reliable would be your first option. I guess I'm just not too trusting of USB storage.
 
Captain Ned
Global Moderator
Posts: 28139
Joined: Wed Jan 16, 2002 7:00 pm
Location: Vermont, USA

Re: Ageing drives, NAS, and Backup Strategy Rethink

Thu Nov 17, 2016 2:23 pm

IMO (and solely IMO), spinning drives are not backup. Backup is the HDDs plugged in to perform the backup, then properly shut down, unplugged, placed in proper clamshell boxen, and removed to a remote location.

Even once I build a NAS, it will be backed up to magnetic media that remains offline/unpowered except for backup cycles. Once I win Powerball, that will be the latest/greatest Ultrium drive/media.
What we have today is way too much pluribus and not enough unum.
 
frumper15
Gerbil XP
Posts: 380
Joined: Mon Jan 18, 2010 3:25 pm

Re: Ageing drives, NAS, and Backup Strategy Rethink

Thu Nov 17, 2016 3:37 pm

I think you have a good idea of what your options are for adding bulk storage but I might look at something like 5TB drives as closer to the Price/capacity sweet spot.  

If I were in your shoes I would be looking at an option for offsite backup.  Unpowered like Ned suggests is possibly best from a security/safey perspective but a backup is only as good as how frequently it is done.  To that end I prefer something like Crashplan that can automatically backup to both their cloud servers (at a cost) as well as your own drives and/or friend family computer for free.  

In your situation I would probably look at doing something like using the existing NAS and changing the disk to JBOD or RAID0 to get capacity needed and fill it up at home before bringing it offsite to save the time and bandwidth for an initial install.  I might look at getting a single 5tb drive for your internal primary storage and either getting another/newer/roll your own NAS at home with more than two bays for future expansion and use your existing 1 and 2TB drives as offsite/offline drives for really important data that you update monthly/quarterly.
i7-8086K | Z370 AORUS GAMING WIFI | 32GB DDR4-2400 | EVGA GTX 1080 Ti | 512GB 960 Pro | 27" Dell 2560x1440 Gsync | Fractal R6 | Seasonic Focus Plus 850W | Win10 Pro x64.
 
The Egg
Minister of Gerbil Affairs
Posts: 2938
Joined: Sun Apr 06, 2008 4:46 pm

Re: Ageing drives, NAS, and Backup Strategy Rethink

Fri Nov 18, 2016 11:45 am

If it were me, I would probably:

  • Pull all the internal drives and replace with a single 6TB
  • Leave the same drives in the NAS, but switch them to RAID-0 (giving you 8TB)
  • Get another 6TB external for backup (to be unplugged afterward)

The suggestion with the NAS might not be popular, but you'd be adding redundancy with the external to offset this, as well as saving money over 2x 8TB.
 
zgirl
Grand Gerbil Poohbah
Posts: 3998
Joined: Tue Jan 01, 2002 7:00 pm
Location: The dark side of the moon
Contact:

Re: Ageing drives, NAS, and Backup Strategy Rethink

Fri Nov 18, 2016 2:10 pm

The Egg wrote:
If it were me, I would probably:

[list]
[*]Leave the same drives in the NAS, but switch them to RAID-0 (giving you 8TB)

The suggestion with the NAS might not be popular, but you'd be adding redundancy with the external to offset this, as well as saving money over 2x 8TB.


You are right, it is not a popular suggestion. RAID-0 is asking for trouble, especially as a storage device. If the concern is to reduce the chances of data loss this is not it. In a RAID-0 if one drive fails data will be lost across all the drives. It will result in a complete outage and restore scenario. RAID-0 is something I would only recommend when performance on multiple spindles is a priority and data protection is not a concern. E.G. I have used this for log volumes or transaction volumes in a server that required speed of the storage as the data was not going to sit there long term. Though often we took that one step further with RAID-10( a mirror copy of that stripe set).
"I used to think the brain was the most amazing organ in the entire body. Then I realized who was telling me this."
If ignorance were painful, half the posters here would be on morphine drips.
 
Ifalna
Gerbil Team Leader
Posts: 274
Joined: Sat Jan 28, 2012 11:14 am
Location: Celestis

Re: Ageing drives, NAS, and Backup Strategy Rethink

Fri Nov 18, 2016 2:53 pm

K-L-Waster wrote:
1) The WD Black and Green drives are both over 6 years old with pretty much daily use over that time. Neither has shown any signs of imminent failure, but given their age and amount of use I'm thinking they may be on borrowed time.

I'm curious as to what lead you to that fear of "drive failure being imminent". I'm using the same WD Green 2TB drive (currently 28.8K hours of powered on) and so far no signs of any hiccups. (Although I did mirror data to a newer 4TB WD-Red as well as put it into a cloud)
The backbone of modern industrial society is, and for the foreseeable future will be, the use of electrical Power.
 
The Egg
Minister of Gerbil Affairs
Posts: 2938
Joined: Sun Apr 06, 2008 4:46 pm

Re: Ageing drives, NAS, and Backup Strategy Rethink

Fri Nov 18, 2016 3:34 pm

zgirl wrote:
You are right, it is not a popular suggestion. RAID-0 is asking for trouble, especially as a storage device. If the concern is to reduce the chances of data loss this is not it. In a RAID-0 if one drive fails data will be lost across all the drives. It will result in a complete outage and restore scenario. RAID-0 is something I would only recommend when performance on multiple spindles is a priority and data protection is not a concern. E.G. I have used this for log volumes or transaction volumes in a server that required speed of the storage as the data was not going to sit there long term. Though often we took that one step further with RAID-10( a mirror copy of that stripe set).

I'm aware that RAID-0 weakens the reliability of the NAS, but he'd also be adding more redundancy with an external drive to offset that.  He's currently only doing a running backup to an internal and then a weekly to the NAS (RAID-1). 

How does JBOD generally fair when a drive goes down versus RAID-0?
 
Duct Tape Dude
Gerbil Elite
Posts: 721
Joined: Thu May 02, 2013 12:37 pm

Re: Ageing drives, NAS, and Backup Strategy Rethink

Fri Nov 18, 2016 4:01 pm

I suggest the following:

1) Pool your drives! Don't be stuck with simple RAIDs.
It's not 1999 anymore. Find a solution that lets you add and replace drives of any size anytime. Simple RAIDs only let you replace one drive with the same capacity, but if you want to upgrade drives, you often lose out on extra capacity unless you upgrade the whole thing or add drives in pairs. So whether it's Stablebit DrivePool, ZFS, LVM, FlexRAID, BTRFS, DriveBender, HDFS, CephFS, or something else, find something that lets you use a new drives of any capacity whenever you want.

2) Use offsite backup!
Do SOMETHING to get your data off of a single server. Whether that's periodically backing up to cold storage (ie: plug in a drive, copy data, then send it to a safe deposit box), backing up to a friend via Crashplan, or syncing to the cloud, keep a copy of your data further than a lightning strike or natural disaster away.

3) Version your files.
Cryptolocker viruses are often a worst-case scenario to build against. Suppose you go on vacation and somehow a new Windows 0-day is found and you come home to your entire computer being encrypted, and dutifully backed up over your old backups. After getting rid of the locker, you can save yourself a bunch of time with a proper versioning solution. When implementing #2, make sure you have copies of files from the last day/week/month/year or get software to do it for you, like Crashplan.

My current solution:
A) Local server (Windows 10 + Stablebit DrivePool + Resilio Sync + Crashplan)
B) Remote server (Ubuntu 16.04 + Resilio Sync + Samba share)
C) Cloud (OneDrive)
D) Friends (Crashplan)

Servers A and B run a file share that's synced together via Resilo Sync. The copy on A is versioned by Crashplan to a drivepool of 6 drives from 500GB to 3TB. I can add/remove drives at will, and I get email alerts if a drive goes down or has bad sectors or something. I can also customize how many copies of each file or folder I want to keep.
Server B just has a single drive with no resiliency whatsoever. It's basically a very very external shared hard drive that's kept in sync.
The Cloud (C) has a copy of the most important data from A. I am in the process of moving off of the cloud and converting solely to Resilio Sync as my own private cloud, because why not.
Friends (D) exchange Crashplan shares with me which is free.

Everything is replaceable:
If A loses a hard drive, the drivepool can automatically move data to another drive. If it loses more drives than its fault tolerance, I can still access data on the remaining drives (unlike RAID!) and rebuild the rest from B, C, and D.
If B dies, it can be rebuilt from A.
If C dies or gets overwritten with encrypted files, I can restore it from Crashplan from D or A.
If D (any friends) decide they're done hosting my shares, I can fall back to A while I make new Crashplan friends. Hopefully that doesn't take too long! :)

However, my system has room for improvement. For example, I have no drive images are anywhere. If I lose a server, it has to be rebuilt from scratch. If anyone has suggestions for me I'd welcome them (if OP would graciously allow me to piggyback off this thread).

sidenote: You know, after all these threads on backups, I'm kind of surprised we gerbils haven't come up with some P2P solution that we could self-host among each other. What if we found some distributed filesystem software solution and managed it ourselves?
 
CScottG
Graphmaster Gerbil
Posts: 1252
Joined: Fri Dec 01, 2006 9:53 pm

Re: Ageing drives, NAS, and Backup Strategy Rethink

Fri Nov 18, 2016 11:17 pm

When it comes to backup you have to question if you are chasing your own ass spending on hardware and still not having off-site backup.

https://www.backblaze.com/cloud-backup.html

..at $60 a year - that nets you several years worth of hardware cost (that you would eventually have to replace). (..just a couple of 4 TB drives are going to be 3-4 years worth of cloud-storage service.)

Amazon Unlimited is similar.
 
Ifalna
Gerbil Team Leader
Posts: 274
Joined: Sat Jan 28, 2012 11:14 am
Location: Celestis

Re: Ageing drives, NAS, and Backup Strategy Rethink

Sat Nov 19, 2016 3:36 am

Yeah, I agree.
I keep a local copy to protect against user error and individual drive failure (by far the most likely sources of data loss) and a copy in the cloud.
Just wish there was a way to get more than 1TB out of OneDrive w/o having to create multiple accounts via 5 people Office "family" pack. <_<
The backbone of modern industrial society is, and for the foreseeable future will be, the use of electrical Power.
 
HERETIC
Gerbil XP
Posts: 488
Joined: Sun Aug 24, 2014 4:10 am

Re: Ageing drives, NAS, and Backup Strategy Rethink

Sat Nov 19, 2016 3:36 am

Not so much a recommendation just a option-
Switch your NAS to JBOD-should be around 7.6TB of storage(power down when not in use)
Replace older/perhaps all drives in system box with 1 or 2 drives (recommend Hitachi)
Files your particularly attached to copy to smaller drives and put in safe or keep offsite.

Cost-possibly less than $100-It's that time of year.4TB is about the best value-thro
seeing that Hitachi 5TB listed in this weeks specials for $166..............................

This would give you a backup copy of your files plus a extra copy of your favorites at minimal cost.
good luck.
 
K-L-Waster
Gerbil XP
Topic Author
Posts: 498
Joined: Thu Feb 12, 2015 8:10 pm
Location: Hmmm, I was *here* a second ago...

Re: Ageing drives, NAS, and Backup Strategy Rethink

Sat Nov 19, 2016 4:04 pm

Thanks to everyone for the suggestions. Based the discussion, I'm leaning towards this plan:

  • Replace the internal drives with a smaller number of larger drives (either 1 x 6TB or 2 x 3TB, haven't decided which yet)
  • Convert the NAS to JBOD to increase the capacity
  • Convert the existing internal drives to local powered-off backup (attach every so often, run backups, then turn off and put in a safe place)
  • Sign up for a cloud-based backup solution for offsite backup (e.g. Backblaze or Crashplan -- will research available options to determine which service to use)
@ Duct Tape Dude -- your suggestion sounds like a really thorough approach that would be great for a consultant or a small business or the like. It seems like overkill for my use case though; I'm dealing with a home system and with sharing some files to the HTPC and the laptop, so nothing I am doing requires rigorous uptime or ensuring access to mission critical data. As long as everything is backed up somewhere I can get to with reasonable ease, I think I'm fine. 

Thanks again, everyone. 
Main System: i7-8700K, ASUS ROG STRIX Z370-E, 16 GB DDR4 3200 RAM, MSI GTX 1080 TI, 1 TB CRUCIAL MX500, Corsair 550D

HTPC: I5-4460, ASUS H97M-E, 8 GB RAM, GTX 970, CRUCIAL 256GB MX100, SILVERSTONE GD09B
 
zgirl
Grand Gerbil Poohbah
Posts: 3998
Joined: Tue Jan 01, 2002 7:00 pm
Location: The dark side of the moon
Contact:

Re: Ageing drives, NAS, and Backup Strategy Rethink

Mon Nov 21, 2016 10:03 am

The Egg wrote:
How does JBOD generally fair when a drive goes down versus RAID-0?


Depends on how you are setting up JBOD. Some systems or software allows for a drive to fail in a JBOD, RAID-0 will not. Again JBOD can be just a devastating as RAID-0 but me personally I would like some type of redundancy in the event of a single drive failure.
"I used to think the brain was the most amazing organ in the entire body. Then I realized who was telling me this."

If ignorance were painful, half the posters here would be on morphine drips.
 
Duct Tape Dude
Gerbil Elite
Posts: 721
Joined: Thu May 02, 2013 12:37 pm

Re: Ageing drives, NAS, and Backup Strategy Rethink

Mon Nov 21, 2016 10:33 am

K-L-Waster wrote:
  • Replace the internal drives with a smaller number of larger drives (either 1 x 6TB or 2 x 3TB, haven't decided which yet)
  • Convert the NAS to JBOD to increase the capacity
  • Convert the existing internal drives to local powered-off backup (attach every so often, run backups, then turn off and put in a safe place)
  • Sign up for a cloud-based backup solution for offsite backup (e.g. Backblaze or Crashplan -- will research available options to determine which service to use)
@ Duct Tape Dude -- your suggestion sounds like a really thorough approach that would be great for a consultant or a small business or the like. It seems like overkill for my use case though; I'm dealing with a home system and with sharing some files to the HTPC and the laptop, so nothing I am doing requires rigorous uptime or ensuring access to mission critical data. As long as everything is backed up somewhere I can get to with reasonable ease, I think I'm fine.

The setup you outlined sounds great! JBOD lets you pool whatever drives you want, cloud-based backup is more than a lightning strike away, and cold storage is a great bonus for fast recoveries.

Who is online

Users browsing this forum: No registered users and 1 guest
GZIP: On