Personal computing discussed

Moderator: Dposcorp

 
klrcommute
Gerbil
Topic Author
Posts: 91
Joined: Sun Oct 11, 2015 1:20 pm

Disk imaging software suggestions

Mon Sep 25, 2017 4:51 pm

I want to do a fresh install of an OS, add all my programs and configuration, then turn this into an image that I can drop onto a backup hard drive so I can save some time when doing a fresh install (as opposed to formatting, reinstalling OS, downloading software, etc.)

Anyone have any software suggestions for what does this well? Opensource would be great (how's Clonezilla?) but I'm not opposed to spending some money on something good, if it's a one-time purchase.
 
Captain Ned
Gold subscriber
Global Moderator
Posts: 26468
Joined: Wed Jan 16, 2002 7:00 pm
Location: Vermont, USA

Re: Disk imaging software suggestions

Mon Sep 25, 2017 4:54 pm

Macrium Reflect is always a good choice and if all you want is a transferable image file, you only need the free version.
If the Earth were flat, cats would have pushed everything off of it by now.
 
bthylafh
Grand Gerbil Poohbah
Posts: 3906
Joined: Mon Dec 29, 2003 11:55 pm
Location: Southwest Missouri, USA

Re: Disk imaging software suggestions

Mon Sep 25, 2017 5:09 pm

I use Clonezilla at work and it's great. The only thing you have to watch out for (that I've noticed, at least) is that you can't restore to a smaller disk than you imaged.
Hakkaa päälle!
i7-8700K|Asus Z-370 Pro|32GB DDR4|Asus Radeon RX-580|Samsung 960 EVO 1TB|1988 Model M||Logitech MX 518 & F310|Samsung C24FG70|Dell 2209WA|ATH-M50x|Asus Xonar DX
 
Captain Ned
Gold subscriber
Global Moderator
Posts: 26468
Joined: Wed Jan 16, 2002 7:00 pm
Location: Vermont, USA

Re: Disk imaging software suggestions

Mon Sep 25, 2017 5:11 pm

bthylafh wrote:
I use Clonezilla at work and it's great. The only thing you have to watch out for (that I've noticed, at least) is that you can't restore to a smaller disk than you imaged.

Macrium can do that even in the free version.
If the Earth were flat, cats would have pushed everything off of it by now.
 
klrcommute
Gerbil
Topic Author
Posts: 91
Joined: Sun Oct 11, 2015 1:20 pm

Re: Disk imaging software suggestions

Mon Sep 25, 2017 5:22 pm

Thanks, I should mention that this will be for both Windows 10 and Linux Mint 18, so I suppose whatever it is needs to deal with ext4 and ntfs?

Is the big difference for disk imaging software is that some do it sector by sector and just create an image file that's virtually the same size before compression and others copy essential parts of the filesystem and thus reduce the image size by a lot? Do I have to fiddle with formatting, MBR, fstab, GRUB, etc. with all this stuff or does it just take care of it?
 
Captain Ned
Gold subscriber
Global Moderator
Posts: 26468
Joined: Wed Jan 16, 2002 7:00 pm
Location: Vermont, USA

Re: Disk imaging software suggestions

Mon Sep 25, 2017 5:26 pm

Macrium Free is aimed at Windows and won't do bare-metal, though the paid version will.
If the Earth were flat, cats would have pushed everything off of it by now.
 
klrcommute
Gerbil
Topic Author
Posts: 91
Joined: Sun Oct 11, 2015 1:20 pm

Re: Disk imaging software suggestions

Mon Sep 25, 2017 5:32 pm

Captain Ned wrote:
Macrium Free is aimed at Windows and won't do bare-metal, though the paid version will.

Ah, thanks. I take it bare-metal is all the sector-by-sector stuff that will capture the MBR, data, whole 9 yards?
 
Captain Ned
Gold subscriber
Global Moderator
Posts: 26468
Joined: Wed Jan 16, 2002 7:00 pm
Location: Vermont, USA

Re: Disk imaging software suggestions

Mon Sep 25, 2017 5:40 pm

klrcommute wrote:
Captain Ned wrote:
Macrium Free is aimed at Windows and won't do bare-metal, though the paid version will.
Ah, thanks. I take it bare-metal is all the sector-by-sector stuff that will capture the MBR, data, whole 9 yards?

Yep. I've done a couple with my paid version (Home) and they take forever. You also can't resize during a bare-metal image (I think).
If the Earth were flat, cats would have pushed everything off of it by now.
 
klrcommute
Gerbil
Topic Author
Posts: 91
Joined: Sun Oct 11, 2015 1:20 pm

Re: Disk imaging software suggestions

Mon Sep 25, 2017 5:42 pm

Looks like I'm gonna have to learn Clonezilla.
 
klrcommute
Gerbil
Topic Author
Posts: 91
Joined: Sun Oct 11, 2015 1:20 pm

Re: Disk imaging software suggestions

Mon Sep 25, 2017 5:47 pm

bthylafh wrote:
I use Clonezilla at work and it's great. The only thing you have to watch out for (that I've noticed, at least) is that you can't restore to a smaller disk than you imaged.

I'll have to keep that in mind if SSD prices keeping going the way they are. :lol:
 
just brew it!
Gold subscriber
Administrator
Posts: 49691
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Disk imaging software suggestions

Mon Sep 25, 2017 7:07 pm

I haven't paid for disk imaging software since the early Norton Ghost days. IIRC I used the free version of Acronis several times a number of years back, and it worked well. Lately I just use Linux CLI to sling images around (it becomes second nature after the first few times, and it's very flexible). I also use gparted if I want a nice GUI interface to futz around with partition sizes; it even handles resizing of NTFS partitions just fine as long as they were cleanly unmounted.
Nostalgia isn't what it used to be.
 
klrcommute
Gerbil
Topic Author
Posts: 91
Joined: Sun Oct 11, 2015 1:20 pm

Re: Disk imaging software suggestions

Mon Sep 25, 2017 8:15 pm

just brew it! wrote:
Lately I just use Linux CLI to sling images around (it becomes second nature after the first few times, and it's very flexible)


Is there a way to generate and restore disk images with just the command line? Like a native linux feature?
 
bthylafh
Grand Gerbil Poohbah
Posts: 3906
Joined: Mon Dec 29, 2003 11:55 pm
Location: Southwest Missouri, USA

Re: Disk imaging software suggestions

Mon Sep 25, 2017 8:39 pm

Yeah, dd will do that. Be warned that it does not suffer fools.
Hakkaa päälle!
i7-8700K|Asus Z-370 Pro|32GB DDR4|Asus Radeon RX-580|Samsung 960 EVO 1TB|1988 Model M||Logitech MX 518 & F310|Samsung C24FG70|Dell 2209WA|ATH-M50x|Asus Xonar DX
 
just brew it!
Gold subscriber
Administrator
Posts: 49691
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Disk imaging software suggestions

Mon Sep 25, 2017 8:51 pm

Elevate to root and everything (including raw disks/partitions) looks like a file. The "dd" command is the canonical way of copying raw data between devices/files. Want to clone to a disk or file on a remote system? Pipe it to a remote ssh session. Want to compress the image? Pipe through gzip. Boot from a thumbdrive live image and clone Windows disks. Etc...

Bit of a learning curve, but if you're familiar with how CLI works it feels very natural.

And yes, as bthylahf notes, you can also shoot yourself in the foot... specifying the wrong destination when cloning to a raw device is guaranteed to result in a very bad day.
Nostalgia isn't what it used to be.
 
klrcommute
Gerbil
Topic Author
Posts: 91
Joined: Sun Oct 11, 2015 1:20 pm

Re: Disk imaging software suggestions

Mon Sep 25, 2017 9:01 pm

Thanks, I'll look into that too. I don't do this stuff for a living so I have a pretty 'cookbook' approach to the CLI, but as long as I don't accidentally clone a disk onto another disk, and in fact create an image to drop into a directory, I shouldn't destroy anything it sounds like.

Anyone know of a good guide that would encompass using dd to create and restore disk images?

Think I found one https://www.linux.com/learn/full-metal-backup-using-dd-command. This seems way more straightforward than learning Clonezilla, so long as I don't screw up the syntax and destroy everything hahah.
 
Captain Ned
Gold subscriber
Global Moderator
Posts: 26468
Joined: Wed Jan 16, 2002 7:00 pm
Location: Vermont, USA

Re: Disk imaging software suggestions

Mon Sep 25, 2017 9:10 pm

just brew it! wrote:
And yes, as bthylafh notes, you can also shoot yourself in the foot... specifying the wrong destination when cloning to a raw device is guaranteed to result in a very bad day.

Which is why I stick to the training wheels that come with Macrium ("do you want to do this?", "do you really want to do this?", "you really should re-evaluate before clicking OK").
If the Earth were flat, cats would have pushed everything off of it by now.
 
klrcommute
Gerbil
Topic Author
Posts: 91
Joined: Sun Oct 11, 2015 1:20 pm

Re: Disk imaging software suggestions

Mon Sep 25, 2017 9:21 pm

Captain Ned wrote:
just brew it! wrote:
And yes, as bthylafh notes, you can also shoot yourself in the foot... specifying the wrong destination when cloning to a raw device is guaranteed to result in a very bad day.

Which is why I stick to the training wheels that come with Macrium ("do you want to do this?", "do you really want to do this?", "you really should re-evaluate before clicking OK").

I'll have to backup my backups before I try it. :lol:

Speaking of linux terminal training wheels, anyone heard of this? http://www.explainshell.com/ I always thought this was helpful for trying to parse stuff that I don't quite understand.
 
klrcommute
Gerbil
Topic Author
Posts: 91
Joined: Sun Oct 11, 2015 1:20 pm

Re: Disk imaging software suggestions

Mon Sep 25, 2017 9:47 pm

just brew it! wrote:
Elevate to root and everything (including raw disks/partitions) looks like a file. The "dd" command is the canonical way of copying raw data between devices/files. Want to clone to a disk or file on a remote system? Pipe it to a remote ssh session. Want to compress the image? Pipe through gzip. Boot from a thumbdrive live image and clone Windows disks. Etc...

Bit of a learning curve, but if you're familiar with how CLI works it feels very natural.

And yes, as bthylahf notes, you can also shoot yourself in the foot... specifying the wrong destination when cloning to a raw device is guaranteed to result in a very bad day.

What do you think is a safe blocksize for running dd, since what I'm reading about says not to overdo it else read errors turn into block errors?
 
Redocbew
Gold subscriber
Graphmaster Gerbil
Posts: 1347
Joined: Sat Mar 15, 2014 11:44 am

Re: Disk imaging software suggestions

Mon Sep 25, 2017 10:04 pm

just brew it! wrote:
And yes, as bthylahf notes, you can also shoot yourself in the foot... specifying the wrong destination when cloning to a raw device is guaranteed to result in a very bad day.


Yeah, I destroyed my gaming install of Windows that way once. I was creating a bootable flash drive from an Ubuntu ISO, and was about ready to play some Diablo 2 with a friend. A change of plans was soon put into action. :P
Do not meddle in the affairs of archers, for they are subtle and you won't hear them coming.
 
just brew it!
Gold subscriber
Administrator
Posts: 49691
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Disk imaging software suggestions

Mon Sep 25, 2017 10:12 pm

klrcommute wrote:
just brew it! wrote:
Elevate to root and everything (including raw disks/partitions) looks like a file. The "dd" command is the canonical way of copying raw data between devices/files. Want to clone to a disk or file on a remote system? Pipe it to a remote ssh session. Want to compress the image? Pipe through gzip. Boot from a thumbdrive live image and clone Windows disks. Etc...

Bit of a learning curve, but if you're familiar with how CLI works it feels very natural.

And yes, as bthylahf notes, you can also shoot yourself in the foot... specifying the wrong destination when cloning to a raw device is guaranteed to result in a very bad day.

What do you think is a safe blocksize for running dd, since what I'm reading about says not to overdo it else read errors turn into block errors?

If you've got bad blocks (or suspect that you may have bad blocks) you shouldn't be using dd in the first place, since it will abort on errors and you'll need to manually restart it past the bad block(s). There's a special variant of dd called ddrescue that is specifically designed to copy all of the good blocks from a disk with bad blocks.

For the block size, you want something that is a multiple of the physical sector size. All modern drives have either 512 byte or 4096 byte physical sectors, so using a multiple of 4096 is always safe. Larger sizes speed up the transfer (unless your transfer rate is being capped by network speed or a legacy USB 2.0 port), but it's generally a matter of diminishing returns once you get above a few MB.
Nostalgia isn't what it used to be.
 
klrcommute
Gerbil
Topic Author
Posts: 91
Joined: Sun Oct 11, 2015 1:20 pm

Re: Disk imaging software suggestions

Mon Sep 25, 2017 10:22 pm

just brew it! wrote:
klrcommute wrote:
just brew it! wrote:
Elevate to root and everything (including raw disks/partitions) looks like a file. The "dd" command is the canonical way of copying raw data between devices/files. Want to clone to a disk or file on a remote system? Pipe it to a remote ssh session. Want to compress the image? Pipe through gzip. Boot from a thumbdrive live image and clone Windows disks. Etc...

Bit of a learning curve, but if you're familiar with how CLI works it feels very natural.

And yes, as bthylahf notes, you can also shoot yourself in the foot... specifying the wrong destination when cloning to a raw device is guaranteed to result in a very bad day.

What do you think is a safe blocksize for running dd, since what I'm reading about says not to overdo it else read errors turn into block errors?

If you've got bad blocks (or suspect that you may have bad blocks) you shouldn't be using dd in the first place, since it will abort on errors and you'll need to manually restart it past the bad block(s). There's a special variant of dd called ddrescue that is specifically designed to copy all of the good blocks from a disk with bad blocks.

For the block size, you want something that is a multiple of the physical sector size. All modern drives have either 512 byte or 4096 byte physical sectors, so using a multiple of 4096 is always safe. Larger sizes speed up the transfer (unless your transfer rate is being capped by network speed or a legacy USB 2.0 port), but it's generally a matter of diminishing returns once you get above a few MB.


Got it. I'm not counting on any bad blocks, just looking to clarify what I was seeing on this archlinux wiki

"Warning: The block size you specify influences how read errors are handled. Read below. For data recovery, use ddrescue.
The dd utility technically has an "input block size" (IBS) and an "output block size" (OBS). When you set bs, you effectively set both IBS and OBS. Normally, if your block size is, say, 1 MiB, dd will read 1024*1024 bytes and write as many bytes. But if a read error occurs, things will go wrong. Many people seem to think that dd will "fill up read errors with zeroes" if you use the noerror,sync options, but this is not what happens. dd will, according to documentation, fill up the OBS to IBS size after completing its read, which means adding zeroes at the end of the block. This means, for a disk, that effectively the whole 1 MiB would become messed up because of a single 512 byte read error in the beginning of the read: 12ERROR89 would become 128900000 instead of 120000089.
If you are positive that your disk does not contain any errors, you could proceed using a larger block size, which will increase the speed of your copying several fold. For example, changing bs from 512 to 64K changed copying speed from 35 MB/s to 120 MB/s on a simple Celeron 2.7 GHz system. But keep in mind that read errors on the source disk will end up as block errors on the destination disk, i.e. a single 512-byte read error will mess up the whole 64 KiB output block."
 
just brew it!
Gold subscriber
Administrator
Posts: 49691
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Disk imaging software suggestions

Mon Sep 25, 2017 10:38 pm

Ahh, TBH I'd actually forgotten about the noerror option, since I use ddrescue if I am trying to copy a disk that is dying!
Nostalgia isn't what it used to be.
 
klrcommute
Gerbil
Topic Author
Posts: 91
Joined: Sun Oct 11, 2015 1:20 pm

Re: Disk imaging software suggestions

Mon Sep 25, 2017 10:43 pm

Here's a good article about tuning block size! http://blog.tdg5.com/tuning-dd-block-size/ I think I'll stick with the 64K though since it's getting over my head though. My 2 SSDs are also only 250GB each, so I suppose it's not gonna make a huge difference in any event.
 
ludi
Darth Gerbil
Posts: 7443
Joined: Fri Jun 21, 2002 10:47 pm
Location: Sunny Colorado front range

Re: Disk imaging software suggestions

Tue Sep 26, 2017 12:10 pm

Do note that Clonezilla is, in fact, a bootable Linux frontend that uses the dd command to do most of its heavy lifting, but it adds enough menu-driven options and cautionary "You are about to irrecoverably do 'x', are you sure??!!1" steps to stop a novice from shotgunning their own foot in one easy step. It's still possible, but it requires three or four somewhat harder steps.

Some of the options, however, are confusing or intimidating for a first-time user, so there's a learning curve. Not as much of a curve as using bare CLI, though.
Abacus Model 2.5 | Quad-Row FX with 256 Cherry Red Slider Beads | Applewood Frame | Water Cooling by Brita Filtration

Who is online

Users browsing this forum: No registered users and 6 guests