Large Hard Drive Defragging

All things storage here: hard drives, DVD RW drives, little wicker baskets.

Moderators: morphine, Steel

Large Hard Drive Defragging

Postposted on Fri Oct 21, 2011 3:16 pm

Hi,
It seems that a large terabyte hard drive would take a very long time to defrag. Is there a method one can use that would shorten the defrag time and if so, what is it?

Would this work? Putting partitions on the hard drive and only defrag the partitions the are used a lot. Is there a defrag program that just defrags a selected partition, if so, what is the name of the program?

Thanks
Dede
Gerbil
 
Posts: 45
Joined: Mon May 18, 2009 7:49 pm

Re: Large Hard Drive Defragging

Postposted on Fri Oct 21, 2011 3:19 pm

Dede wrote:Hi,
It seems that a large terabyte hard drive would take a very long time to defrag. Is there a method one can use that would shorten the defrag time and if so, what is it?

Would this work? Putting partitions on the hard drive and only defrag the partitions the are used a lot. Is there a defrag program that just defrags a selected partition, if so, what is the name of the program?

Thanks


I use Auslogics defrag for my hard disks. I have a 1TB disk that I defrag every once in a while. Doesn't take long. No need to split it into partitions unless you absolutely have to for another reason.
Main: i7 4790K - Z97 mATX - 16GiB DDR3 1866 - GTX 970 - 256GB 850 PRO - 500GB 840 EVO - HGST 3TB - U2415 - Win8 Pro x64
Work/Play: 2012 13" Macbook Air
Work: 2014 Dell Inspiron 7000
DancinJack
Minister of Gerbil Affairs
 
Posts: 2067
Joined: Sat Nov 25, 2006 3:21 pm
Location: Massachusetts

Re: Large Hard Drive Defragging

Postposted on Fri Oct 21, 2011 3:26 pm

Dede wrote:Hi,
It seems that a large terabyte hard drive would take a very long time to defrag. Is there a method one can use that would shorten the defrag time and if so, what is it?

Would this work? Putting partitions on the hard drive and only defrag the partitions the are used a lot. Is there a defrag program that just defrags a selected partition, if so, what is the name of the program?

Thanks


Windows Vista or Windows 7 and then put manual defragging out of your mind... forever. :P
"Welcome back my friends to the show that never ends. We're so glad you could attend. Come inside! Come inside!"
Ryu Connor
Global Moderator
Gold subscriber
 
 
Posts: 3598
Joined: Thu Dec 27, 2001 7:00 pm
Location: Marietta, GA

Re: Large Hard Drive Defragging

Postposted on Fri Oct 21, 2011 3:30 pm

Agreed ryu :)
2600k HT on@4705mhz 8gb Cas9 1600 mem 2x EVGA GTX770 4gb Classified cards in SLI @1320 mhz core and 2003 mhz mem,mounted in CM HAF922 with a TX-850 PSU 2xHTPC's 2xi3 2120 3.3ghz dual core,1xasus LP HD6570 1xHIS hd7750@1150core1325mem,55"PanyVT30
vargis14
Graphmaster Gerbil
 
Posts: 1325
Joined: Fri Aug 20, 2010 6:03 pm
Location: philly suburbs

Re: Large Hard Drive Defragging

Postposted on Fri Oct 21, 2011 3:33 pm

MyDefrag is great, you can even write custom scripts.
5150
Minister of Gerbil Affairs
 
Posts: 2388
Joined: Wed Jun 04, 2003 6:22 pm
Location: Sales Tax Is For Commies

Re: Large Hard Drive Defragging

Postposted on Sat Oct 22, 2011 12:24 am

Best defragger, bar none, IMHO is Raxco PerfectDisk. Not free, but it works extremely well.

It may take several hours for the initial defrag, but it will finish it if you let it (unlike Diskeeper, which seems to take many passes). The buit-in Windows defragger is just a stripped-down version of Diskeeper.
Buub
Maximum Gerbil
Silver subscriber
 
 
Posts: 4214
Joined: Sat Nov 09, 2002 11:59 pm
Location: Seattle, WA

Re: Large Hard Drive Defragging

Postposted on Sat Oct 22, 2011 12:31 am

Buub wrote:Best defragger, bar none, IMHO is Raxco PerfectDisk. Not free, but it works extremely well.

It may take several hours for the initial defrag, but it will finish it if you let it (unlike Diskeeper, which seems to take many passes). The buit-in Windows defragger is just a stripped-down version of Diskeeper.

Ditto. Defragging a drive nowadays doesn't have the same effect as it used to, as NTFS is much better at housekeeping, but even then, a defrag every few months (assuming heavy use) is quite noticeable. May be and likely is moot with SSDs though.
There is a fixed amount of intelligence on the planet, and the population keeps growing :(
morphine
Grand Admiral Gerbil
Silver subscriber
 
 
Posts: 10093
Joined: Fri Dec 27, 2002 8:51 pm
Location: Portugal (that's next to Spain)

Re: Large Hard Drive Defragging

Postposted on Sat Oct 22, 2011 1:48 am

I use Ultradefrag, nice little free software that does a good job.
Intel i5 4670K @ 4.8GHZ|ATI Radeon HD 7970 (Stock)| 12 GB RAM (Stock)| Xtreme Music with G500 5.1 | Panasonic "TH-L42E60" @ 120 HZ.
Jigar
Maximum Gerbil
Silver subscriber
 
 
Posts: 4618
Joined: Tue Mar 07, 2006 4:00 pm

Re: Large Hard Drive Defragging

Postposted on Sat Oct 22, 2011 8:28 am

MyDefrag can set up scheduled defrags on installation. It'll run a quick defrag daily and a long defrag monthly as long as the computer's on; if you're running Win7 or Vista you'll have to turn off UAC or manually approve each launch, though. However, those versions of Windows automatically schedule defrags with their built-in utilities anyway, so there's probably no need.

I've got a 1.5TB hard drive and only defrag it every few months. Doesn't seem worthwhile to do it more often, even to do it overnight when I'm not using it.
Think for yourself, schmuck!
i5-2500K@4.3|Asus P8P67-LE|8GB DDR3-1600|Powercolor R7850 2G|1.5TB 7200.11|1988 Model M|Saitek X-45 & P880|Logitech MX 518|Dell 2209WA|Sennheiser PC151|Asus Xonar DX
bthylafh
Grand Gerbil Poohbah
 
Posts: 3235
Joined: Mon Dec 29, 2003 11:55 pm
Location: Southwest Missouri, USA

Re: Large Hard Drive Defragging

Postposted on Sat Oct 22, 2011 9:10 am

morphine wrote:Ditto. Defragging a drive nowadays doesn't have the same effect as it used to, as NTFS is much better at housekeeping, but even then, a defrag every few months (assuming heavy use) is quite noticeable.

Agreed. NTFS isn't nearly as affected by fragmentation as FAT/FAT32, but an occasional defrag is still beneficial.

For XP systems I run a defrag whenever I do a general housecleaning (remove the inevitable outdated versions of Java, install latest patches for all apps, clean out wreckage from uninstalled apps, full virus scan, etc.). Nothing fancy; I generally just run 2-3 passes of the built-in defragger.

The absolute *best* way to defrag a data drive is to copy the entire contents to another drive, reformat the original partition, then copy everything back again. Not with an imaging tool (this would copy the fragmentation as well); you need to do a file copy (i.e. just select everything in the top level folder of the drive and drag it to the destination drive). This of course means you need to have a large scratch drive in an external enclosure with a fast interface (eSATA or USB 3.0); but such drives are quite affordable these days.

The Linux filesystem guys claim that ext2/3/4 isn't affected by fragmentation, and that's why they've never bothered to create an official Linux filesystem defrag tool. I don't completely buy this claim, but it does seem to be close enough to true that it doesn't matter much in the majority of use cases. For drives that have been heavily used with lots of small files being created and deleted, occasional use of the "copy out, copy back" method described above is adequate.

morphine wrote:May be and likely is moot with SSDs though.

For SSDs TRIM support is what you need. Doing a traditional defrag on a SSD is just going to waste your SSD's write cycles, for little or no performance gain.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 38128
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Large Hard Drive Defragging

Postposted on Sat Oct 22, 2011 9:32 am

just brew it! wrote:The absolute *best* way to defrag a data drive is to copy the entire contents to another drive, reformat the original partition, then copy everything back again.


For small values of "best". That's a lot of work for not much gain vs traditional techniques.

The Linux filesystem guys claim that ext2/3/4 isn't affected by fragmentation, and that's why they've never bothered to create an official Linux filesystem defrag tool.


Not really; it's just not affected as much as a simple fs like FAT. One of ext4's features is extents, which is basically used to reduce fragmentation. There are unofficial ext2 defraggers (won't work with ext3 or 4) and allegedly there's ongoing work on creating an ext4 defragger.
Think for yourself, schmuck!
i5-2500K@4.3|Asus P8P67-LE|8GB DDR3-1600|Powercolor R7850 2G|1.5TB 7200.11|1988 Model M|Saitek X-45 & P880|Logitech MX 518|Dell 2209WA|Sennheiser PC151|Asus Xonar DX
bthylafh
Grand Gerbil Poohbah
 
Posts: 3235
Joined: Mon Dec 29, 2003 11:55 pm
Location: Southwest Missouri, USA

Re: Large Hard Drive Defragging

Postposted on Sat Oct 22, 2011 10:03 am

bthylafh wrote:
just brew it! wrote:The absolute *best* way to defrag a data drive is to copy the entire contents to another drive, reformat the original partition, then copy everything back again.

For small values of "best". That's a lot of work for not much gain vs traditional techniques.

It is "best" in the sense that it reduces fragmentation to zero in a single pass, and works on any filesystem (even ones that don't support defraggers like ext4). :wink:

And while it *is* more work than running a traditional defragger, it isn't *much* work, assuming you've got a spare external drive you can use. For a home system, start the copy-out when you go to bed, and the copy-back when you leave for work in the morning. By the time you get home in the evening it's done, and you've spent maybe 5 minutes (10 tops if you had to search for the external hard drive that's been buried under a pile of crap in the family room) of your time on it. I suppose if you're the anal-retentive type who wants to defrag the disk on a weekly basis, yeah it could get tedious... but I generally defrag only when I notice a slowdown, or when I'm doing other periodic maintenance on the system.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 38128
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Large Hard Drive Defragging

Postposted on Mon Oct 24, 2011 4:15 pm

Hi,
I thank you all for the very good information I received from you folks!

I would also appreicate help with this question: Which is better to put on the C:\ disk drive, folders or partitions?

I was told quite some time ago by a small PC builder: "One is better off using folders than partitions on the C:\ disk drive. Folders can grow in size but partions are fixed in size."

What do you think about that? I have no idea.

If partitions, what size and how many do you recommend and names for them?

I plan on using a Spinpoint with one or two TB for my C drive.

Again, I want to say "Thank you very much" for the answers I received from my questions.

dede27
Dede
Gerbil
 
Posts: 45
Joined: Mon May 18, 2009 7:49 pm

Re: Large Hard Drive Defragging

Postposted on Mon Oct 24, 2011 4:37 pm

If you're running Windows, I would not bother with partitioning the hard drive. There's no real gain to doing it that way and a lot of inconvenience.
Think for yourself, schmuck!
i5-2500K@4.3|Asus P8P67-LE|8GB DDR3-1600|Powercolor R7850 2G|1.5TB 7200.11|1988 Model M|Saitek X-45 & P880|Logitech MX 518|Dell 2209WA|Sennheiser PC151|Asus Xonar DX
bthylafh
Grand Gerbil Poohbah
 
Posts: 3235
Joined: Mon Dec 29, 2003 11:55 pm
Location: Southwest Missouri, USA

Re: Large Hard Drive Defragging

Postposted on Sun Feb 12, 2012 10:23 pm

Buub wrote:Best defragger, bar none, IMHO is Raxco PerfectDisk. Not free, but it works extremely well.

It may take several hours for the initial defrag, but it will finish it if you let it (unlike Diskeeper, which seems to take many passes). The buit-in Windows defragger is just a stripped-down version of Diskeeper.

Besides Raxco's PerfectDisk 12.5 Pro ($29.99), you're missing out on several other high-end defraggers:

O&O's Defrag 15 Professional Edition ($29)
Piriform's Defraggler ($24.95)
Ashampoo's Magical Defrag 3 ($14.99)
IObit's Smart Defrag 2.3 (free)
Auslogic's Disk Defrag 3.3 (free)
Puran Defrag (free / $19.95)

and

Vopt 9 ($40) - anyone know of this one?
thegleek
Darth Gerbil
Gold subscriber
 
 
Posts: 7367
Joined: Tue Jun 10, 2003 11:06 am
Location: Detroit, MI

Re: Large Hard Drive Defragging

Postposted on Mon Feb 13, 2012 8:57 am

thegleek wrote:Puran Defrag (free / $19.95)

Reporting back to ya'll about the free version of Puran Defrag on a Win7 Pro (64) OS as epic fail:

Image

It did this twice to me so far, so I'm done with it. Uninstalling. Next!
thegleek
Darth Gerbil
Gold subscriber
 
 
Posts: 7367
Joined: Tue Jun 10, 2003 11:06 am
Location: Detroit, MI

Re: Large Hard Drive Defragging

Postposted on Mon Feb 13, 2012 5:21 pm

I use Defraggler every so often on my HDDs.

http://www.piriform.com/defraggler

Works great. Doesn't seem to take that long.
IrateAdmin
Gerbil In Training
 
Posts: 6
Joined: Sun Apr 24, 2011 8:16 pm

Re: Large Hard Drive Defragging

Postposted on Mon Feb 13, 2012 5:38 pm

It all depends on what you store on the drive and how you use it as to whether you can really speed up a defrag. My own choice of defraggers are http://www.mydefrag.com/ and http://www.piriform.com/defraggler. Defraggler is a standard defragger that also happens to be free.

Mydefrag is handy if you are a bit more technical (and it's also free.) With mydefrag you have the options of how you want to defrag and what you want to defrag. You can even script your own version. For instance I have a disk that I use for storing media files as well as for temporary downloads. So I wrote up a little script that I run periodically that moves all of the temp files to the end of the drive and the does a fast fill on the media files. With the Fast Fill option the program looks for enough free space to hold one of the files to be sorted. If it finds enough contiguous space then it fills it. If there isn't enough contiguous space then it won't defrag that file.

That means it won't be as effective as the standard defrag option but it finishes quickly. Given that the drive is just media files there isn't as much of an issue with fragmentation as there would be if the drive stored lots of smaller files that were regularly updated.


As for your comment about folders versus partitions I tend to agree with what you wrote. For most uses it's better to have a large partition and separate things into folders. There are certain cases where there isn't true (huge numbers of small files or lots of temporary data being updated all the time) but those issues aren't seen by the normal user so for most people it's best to make a drive into a single partition. (Though I might suggest even a normal user could get by with formatting a 4TB drive into a couple of partitions, should he/she be so lucky as to have one. ;) )
nanoflower
Gerbil First Class
Silver subscriber
 
 
Posts: 198
Joined: Wed Mar 04, 2009 1:10 pm

Re: Large Hard Drive Defragging

Postposted on Mon Feb 13, 2012 5:46 pm

With the built-in defragger of Vista and 7 I really don't see the need to concern yourself with this. They keep fragmentation pretty low, and if I ever noticed them defragging, it was when the system was idle and it stopped as soon as I touched the mouse. Any tiny performance edge you might get out of manually defragging is offset by the time (and even money) you spend researching, installing, testing and running the software. The only reason I can think of is if you need to be absolutely positive that something like a video recording hard drive will have 100% of its performance 100% of the time.
Firestarter
Gerbil XP
 
Posts: 498
Joined: Sun Apr 25, 2004 11:12 am

Re: Large Hard Drive Defragging

Postposted on Mon Feb 13, 2012 6:23 pm

I agree that the built in defragger for Win7/Vista is all the typical person needs. In an era of automatic defragging and NTFS, paying for a defragging program feels like a really silly thing to do. Defraggler is a good, free program if you really feel the need to go above and beyond the call, but your actual performance gain will probably be unnoticable.


Never defrag an SSD. Windows 7 actually turns off the automatic defragger for any SSD drives you have plugged in. Defragging is only a worry because of the physical nature of HDDs, as the head has to read over a larger area of the platter to retrieve its information. SSDs do not have this worry since there are no moving parts. Defragging an SSD does far more harm than anything else, as you rapidly burn through its limited read/write amount in doing so.
EsotericLord
Gerbil First Class
 
Posts: 135
Joined: Mon Jan 11, 2010 1:23 pm

Re: Large Hard Drive Defragging

Postposted on Mon Feb 13, 2012 6:26 pm

Firestarter wrote:With the built-in defragger of Vista and 7 I really don't see the need to concern yourself with this. They keep fragmentation pretty low, and if I ever noticed them defragging, it was when the system was idle and it stopped as soon as I touched the mouse. Any tiny performance edge you might get out of manually defragging is offset by the time (and even money) you spend researching, installing, testing and running the software. The only reason I can think of is if you need to be absolutely positive that something like a video recording hard drive will have 100% of its performance 100% of the time.

Vista & Win7 automatically schedule a weekly defrag event for HDDs. On Win7, I believe it starts at 1am Wednesday assuming the system is up at the time, not sure about Vista.

WinXP of course still has the barebones version of DiskKeeper Lite that was integrated with Win2k but AFAIK it can only be invoked manually, not scheduled.
He who laughs last, laughs first next time.
ludi
Gerbil Elder
 
Posts: 5490
Joined: Fri Jun 21, 2002 10:47 pm
Location: Sunny Colorado front range

Re: Large Hard Drive Defragging

Postposted on Mon Feb 13, 2012 6:47 pm

ludi wrote:Vista & Win7 automatically schedule a weekly defrag event for HDDs. On Win7, I believe it starts at 1am Wednesday assuming the system is up at the time, not sure about Vista.

WinXP of course still has the barebones version of DiskKeeper Lite that was integrated with Win2k but AFAIK it can only be invoked manually, not scheduled.



While this is true, Win 7 will also automatically defrag when the computer is idle should it miss that 1am Wednesday schedule.
EsotericLord
Gerbil First Class
 
Posts: 135
Joined: Mon Jan 11, 2010 1:23 pm

Re: Large Hard Drive Defragging

Postposted on Tue Feb 14, 2012 11:26 am

IrateAdmin wrote:I use Defraggler every so often on my HDDs.

http://www.piriform.com/defraggler

Works great. Doesn't seem to take that long.

Downloaded the free version. Installed. Defrag'd 1 drive that previously failed using that crap Puran Defrag

Defraggler was awesome! It did it! Very nicely too! I wonder what paying for it would get me? something about the pagefile i assume?
thegleek
Darth Gerbil
Gold subscriber
 
 
Posts: 7367
Joined: Tue Jun 10, 2003 11:06 am
Location: Detroit, MI

Re: Large Hard Drive Defragging

Postposted on Tue Feb 14, 2012 11:32 am

EsotericLord wrote:While this is true, Win 7 will also automatically defrag when the computer is idle should it miss that 1am Wednesday schedule.

How idle is "idle"? I have an HDD-based laptop that is never up Wednesday mornings, and in fact only runs during the weekends typically. Although it is sometimes idle for an hour or two, the level of disk fragmentation seems to be creeping upward (only checked it a couple times out of curiosity but did see 12% recently).
He who laughs last, laughs first next time.
ludi
Gerbil Elder
 
Posts: 5490
Joined: Fri Jun 21, 2002 10:47 pm
Location: Sunny Colorado front range

Re: Large Hard Drive Defragging

Postposted on Tue Feb 14, 2012 11:52 am

axeman wrote:I imagine defragging an SSD repeated could significantly shorten it's useful life, as if MLC drives didn't have enough challenges as it is.

While I'm generally inclined to agree with this statement, I can also think of a couple of reasons why it might be untrue:

1) If you're writing enough data to the SSD that wear-out is a real concern, the additional write activity from defragging is probably the least of your worries.

2) Periodically moving data around might actually help the wear-leveling algorithm, by freeing up (relatively) lightly used blocks which would otherwise remain unavailable for re-use.

That said, given that there's no performance benefit to defragging a SSD it is almost certainly best to just leave well enough alone.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 38128
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Large Hard Drive Defragging

Postposted on Tue Feb 14, 2012 2:21 pm

just brew it! wrote:
axeman wrote:I imagine defragging an SSD repeated could significantly shorten it's useful life, as if MLC drives didn't have enough challenges as it is.

While I'm generally inclined to agree with this statement, I can also think of a couple of reasons why it might be untrue:

1) If you're writing enough data to the SSD that wear-out is a real concern, the additional write activity from defragging is probably the least of your worries.

2) Periodically moving data around might actually help the wear-leveling algorithm, by freeing up (relatively) lightly used blocks which would otherwise remain unavailable for re-use.

That said, given that there's no performance benefit to defragging a SSD it is almost certainly best to just leave well enough alone.

If its such a concern to defrag a SSD, then I'm never buying one. I should be able to defrag a SSD weekly or daily if i WANT to... Without ever worrying about any degradation.
thegleek
Darth Gerbil
Gold subscriber
 
 
Posts: 7367
Joined: Tue Jun 10, 2003 11:06 am
Location: Detroit, MI

Re: Large Hard Drive Defragging

Postposted on Tue Feb 14, 2012 2:31 pm

thegleek wrote:If its such a concern to defrag a SSD, then I'm never buying one. I should be able to defrag a SSD weekly or daily if i WANT to... Without ever worrying about any degradation.

But given that there's no benefit, why would you WANT to? It's like refusing to buy a really nice shirt because the color fades if you wash it 10 times a day, every day for a month!
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 38128
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Large Hard Drive Defragging

Postposted on Tue Feb 14, 2012 2:38 pm

just brew it! wrote:It's like refusing to buy a really nice shirt because the color fades if you wash it 10 times a day, every day for a month!
In fact, it's like refusing to buy a really nice shirt that never gets dirty and is in no way improved by washing, just because if you wash it a ridiculous number of times the color fades. There are all sorts of products that don't work right or fail early if you abuse them; the people who happily buy and use them just know to avoid that abuse.

Edit: and I should add, it's not like hard drives last forever either (nor, I suspect, would continuously defragging them help with their MTBF)
Last edited by UberGerbil on Tue Feb 14, 2012 10:34 pm, edited 1 time in total.
UberGerbil
Gerbil Khan
 
Posts: 9999
Joined: Thu Jun 19, 2003 3:11 pm

Re: Large Hard Drive Defragging

Postposted on Tue Feb 14, 2012 3:39 pm

I'm never washing my shirts again.
I do not understand what I do. For what I want to do, I do not do. But what I hate, I do.
derFunkenstein
Gerbil God
Gold subscriber
 
 
Posts: 21557
Joined: Fri Feb 21, 2003 9:13 pm
Location: WHAT?

Re: Large Hard Drive Defragging

Postposted on Tue Feb 14, 2012 4:47 pm

just brew it! wrote:It is "best" in the sense that it reduces fragmentation to zero in a single pass, and works on any filesystem (even ones that don't support defraggers like ext4). :wink:


Eh, Raxco Perfectdisk will bring it to zero in a single pass too, consolidation and all (except for locked system files but that's a given). Faster too since it's unlikely to have to move every single file.

Still, unless you're on an XP machine, you don't really have to worry about this nowadays.
ChronoReverse
Gerbil Elite
 
Posts: 738
Joined: Wed Dec 12, 2007 4:20 pm

Next

Return to Storage

Who is online

Users browsing this forum: No registered users and 2 guests