Oh sheesh, I'm sorry for the long post. But I wanted to share my experience and thinking on this, for whatever it's worth.
just brew it! wrote:
BIF wrote:I do still partition my drives, for the same reasons that the OP raised:
1. Flexibility to avoid backing up low-change data more often than necessary.
2. Flexibility to reduce defrag frequency for some partitions because defrags cause incremental and differential backups to grow in size. <snip>
By definition, incremental/differential backups only back up data that has changed, so #1 and #2 are contradictory.
Not contradictory, because the backup methodology involves more than just hardware and software: Don't forget the wetware.
Furthermore, most incremental/differential backups operate on files, not on raw disk blocks; what matters is whether the contents of a file has changed, not where the file's blocks physically reside on disk.
This may be so in theory, but it has not been my observation, at least not with respect to Acronis True Image (my old backup software) and Diskeeper. If even a single 4K block is moved, the WHOLE FILE gets marked as "changed", and therefore gets picked up in the next incremental backup run. Some VST container files are really quite huge and being already compressed by the software maker, they are not very compressible by a backup program.
It's possible that Macrium only backs up blocks/sectors and that this won't be an issue.
So defrags should have no effect on what an incremental backup tool considers to have "changed".
Again, this is not in agreement with my observations, which typically went something like this before I segregated my data (this was with Acronis):
1. Schedule runs a full backup. Let's say it's a drive that contains a lot of data and the full backup is 500 GB; backed up to a 1.5 TB backup drive.
2. I use the system for a week or so. During that time, I install and update no software; but I use Office and web browser.
3. Schedule runs the first incremental backup.
4. I observe that size of incremental backup image is 300-350 GB.
5. Ask myself the question: WTF?
6. Another week or three pass. I now have several incremental images that approach anywhere from 25% to 75% of the size of the original full backup image. Now my 1.5 TB backup drive is full.
Research reveals that many of my *.WAV, *.NKI, and *.NKS files have been backed up by one or more incremental runs. Some of these are downright HUGE, so now I have many versions of them within the incremental backups.
But why? As noted in 2 above, I have not installed or updated software. For background, VST instruments don't receive a lot of updates and I certainly don't update them on a day-to-day or even month-to-month basis.
Answer: It's DEFRAG. Either Windows defrag, Diskeeper, or "other"; whichever one you have running on its schedule. It's moving stuff around and flipping on the change bits, making incremental backups big.
Diskeeper uses IFAAST, which will move pieces/parts around as the file system statistics indicate. It's not uncommon for my VSTi partitions to have all the big container files at the END of the partition with the freespace in the middle and the frequently-updated files (what few there may be) placed at the beginning of the partition. This happens over many days and weeks, and not all at once; which explains why I end up with several iterations of big incremental backups.
Eventually things quiet down and the partition is in a plateau state with not much being defragged. But once you resize the partition or update the software on it, then it starts all over again.
The initial decision to partition my drives was to give me a little bit more granularity with regards to backups and defrags, and it has been a good decision for me.
I am thinking ahead, however; and I know that in the coming years, moving more and more of my data to SSDs will reduce my need for a defragger. Even now (because half of my VST instrument samples are already on SSDs), I'm not convinced that Diskeeper is worth the money. I am trialing the latest version, and I may or may not make the upgrade purchase. If I do, it may well be my last one ever, because I expect that in 5-7 year's time, I'll only be using HDD media for backup storage and NAS devices; nothing that will need the performance improvement offered by a defragger.
Today's SSDs are not huge, so I will STILL end up with several partitions for the foreseeable future; just not for the original reasons.
Years ago, I wanted my backup jobs to run quickly, to avoid long windows of sluggish response produced by intense disk I/O and the CPU being busy compressing/encrypting my backup data, especially while using the DAW software, which could be prone to audible clicks, pops, dropouts, and even crashes. Therefore, I wanted backups, defrags, and antivirus scans to only run when I was sleeping or at my day job.
I will admit that new hardware today can take a lot of punishment and backups/defrags seem to not impact my music stuff like it did in the past. Hence, some of the original reasons for segregation are not as strong as they used to be. I have simplified my partitioning strategy accordingly, and I've also not established a strict schedule for maintenance like I did in the past. I will continue to re-examine my strategies with each new system I build; though the next one may not happen until 2018-2020. If we still have PCs, that is.