Large Hard Drive Defragging

[Ryu Connor]

Acronis isn't a traditional backup tool. It has nothing to do with what JBI is talking about.

Defrag does not change the archive attribute, which is what a traditional backup uses to define new or modified files. In fact a restore from a traditional backup would result in a disk that is contiguous. Not so with Acronis, due to the fact that it does not backup files.

Link

[just brew it!]

@BIF -

It didn't hit me until Ryu's post that you were attempting to do incremental backups at the block level.

Your issue is that you are attempting to use a disk imaging product to do periodic incremental backups. TBH this is sort of like using a screwdriver to hammer in a nail. Imaging products work at the block level, not the file level. Doing incremental block-level backups is fundamentally incompatible with defragging (and quite frankly, may not be a particularly good fit for journaling file systems either, depending on how the file system is being used).

Defragging doesn't change the archive bit, or the modification timestamps of files. These are the mechanisms that normal (file-based) incremental backup tools use to detect whether a file has changed. But a block-level incremental tool is just going to look at whether the contents of each physical disk block has changed; it isn't going to search the entire disk to figure out if the data for that block has been moved elsewhere and record that fact; it is just going to back the entire block up again.

If you're going to use incremental block-level backups on a file system that gets defragged, the only sensible way to do it is run a full backup after each defrag, and run incrementals only between defrags. A file-based incremental backup tool would be a much better solution.

[BIF]

Well, damn! No wonder I keep breaking screwdrivers and bending nails!

But I jest. My strategy works for me; I just had to find a way around the problems first. Staggered backups with varying frequencies seems to have done the trick. You're right, the "best practices" method would be to run defrags right before full backups. But Diskeeper advertises that it's best to just leave it on 24/7 so that it can move things around even very soon after they were written in a fragmented state. That's a relatively new feature, and I like the possibility that Diskeeper could defrag a file even before it gets backed up the first time after being created or updated.

Since my new system is so much more powerful than anything before it, there's no performance reason to throttle Diskeeper. I like the simplicity of leaving it on!

I'm not using any databases or other journaling software on this system (yet), so a recovery situation (for now) will tolerate a recovery of each partition to a different point-in-time.

The current setup will probably suffice for years to come. And when I install a database (which may be later this year because I have a project in mind), I will probably have to revisit this strategy with respect to the database files and transaction logs. Possibly put them on my office data partition and exclude them from Macrium's backups, then use the batch scheduler to have the database do its own backups for correct handling of locking and concurrency.

Thanks guys; I learned something today!

[just brew it!]

But I jest. My strategy works for me; I just had to find a way around the problems first. Staggered backups with varying frequencies seems to have done the trick. You're right, the "best practices" method would be to run defrags right before full backups. But Diskeeper advertises that it's best to just leave it on 24/7 so that it can move things around even very soon after they were written in a fragmented state. That's a relatively new feature, and I like the possibility that Diskeeper could defrag a file even before it gets backed up the first time after being created or updated.

OTOH, if you use a file-based backup solution fragmentation becomes irrelevant. Restore a file-based backup to a fresh drive, and it will actually be *less* fragmented than an image backup/restore of a defragmented drive.

Since my new system is so much more powerful than anything before it, there's no performance reason to throttle Diskeeper. I like the simplicity of leaving it on!

To each his own, I guess. I do not like regularly scheduled defrags since it results in extra wear and tear on the drive(s), and also carries a small risk of data corruption (especially if you are not using ECC RAM and/or do not have a UPS).

I'm not using any databases or other journaling software on this system (yet), so a recovery situation (for now) will tolerate a recovery of each partition to a different point-in-time.

All modern file systems (NTFS, EXT4, etc.) use journals internally, and can write updated data to blocks other than the ones which were occupied by the original file. This can result in extra blocks getting backed up by a block-based incremental backup tool.

The current setup will probably suffice for years to come. And when I install a database (which may be later this year because I have a project in mind), I will probably have to revisit this strategy with respect to the database files and transaction logs. Possibly put them on my office data partition and exclude them from Macrium's backups, then use the batch scheduler to have the database do its own backups for correct handling of locking and concurrency.

Yeah, databases (especially if large) won't play nice with file-based incremental backups either. In fact, for databases you may be better off with the block-based incremental, provided you disable the defrag.

Thanks guys; I learned something today!

You're welcome...