Over provisioning fully used SSD

All things storage here: hard drives, DVD RW drives, little wicker baskets.

Moderators: morphine, Steel

Over provisioning fully used SSD

Postposted on Tue May 20, 2014 4:19 am

Is it possible to overprovison an Intel SSD that has previously had its full capacity used as a single partition? If I created a smaller partition and left the remainder as unallocated space am I golden? Do I need to run something to trim the unallocated space?
FDISK /MBR
Dirge
Gerbil Jedi
 
Posts: 1543
Joined: Thu Feb 19, 2004 3:08 am
Location: New Zealand

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 5:50 am

Yes - Use Windows built in disk utility (Under Administrative Tools) to shrink the volume of the SSD by say 15-20% and leave the rest un-allocated. You don't need to do anything else like trim it or re-format it. Outside of read-heavy workloads (where you get slightly higher IOPS with overprovisioning), the performance gain is negligible though on most drives and not worth the loss in space?

(Edited for a typo)
Intel C2E QX9770 @ 4.2Ghz, 1.4v | Gigabyte GA-X38T-DQ6 | 8GB Corsair DDR3 1600 | GTX 750 Ti 2GB | Crucial 512GB M550 SSD
geekl33tgamer
Gerbil Elite
 
Posts: 543
Joined: Tue Aug 25, 2009 7:25 pm
Location: England

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 6:37 am

geekl33tgamer wrote:Yes - Use Windows built in disk utility (Under Administrative Tools) to shrink the volume of the SSD by say 15-20% and leave the rest un-allocated. You don't need to do anything else like trim it or re-format it. Outside of read-heavy workloads (where you get slightly higher IOPS with overprovisioning), the performance gain is negligible though on most drives and not worth the loss in space?

(Edited for a typo)

You are assuming that shrinking the partition automatically tells the drive to trim the extra space. I am not certain this is the case; and if it is not true, you won't get the full benefits of over-provisioning since the drive will think any previously allocated blocks in the extra space are still in use.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37464
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 6:38 am

Hi geekl33tgamer I am more concerned with increasing the endurance of my SSD.

Hi JBI that is what I am concerned about and not sure how to remedy.
FDISK /MBR
Dirge
Gerbil Jedi
 
Posts: 1543
Joined: Thu Feb 19, 2004 3:08 am
Location: New Zealand

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 6:56 am

Well, if you want to be absolutely certain, you can back up all the data, use the manufacturer's tool to restore the drive to factory-fresh state, re-partition, and copy the data back. Or shrink the existing partition, create a second partition in the unused space, format and trim it (does formatting a partition implicitly trim? not sure...), then never touch the second partition again.

But taking a step back, IMO in the grand scheme of things you're probably worrying too much about endurance. Unless the drive's wear leveling algorithm sucks, or you've got an outlier workload that writes a ton of data to the SSD, it'll be obsolete before it wears out. (And if either of those two aggravating factors applies to you, over-provisioning probably isn't going to make much of a difference.)
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37464
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 7:10 am

It is one of the blessings of old friends that you can afford to be stupid with them. Ralph Waldo Emerson.
Captain Ned
Global Moderator
Gold subscriber
 
 
Posts: 20091
Joined: Wed Jan 16, 2002 7:00 pm
Location: Vermont, USA

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 11:16 am

The samsung magician software recognises a drive as overprovisioned as long as there's some unallocated space. If it's unallocated, the drive uses it however the hell it wants, I guess - so no need to TRIM or GC as far as I'm aware.
<insert large, flashing, epileptic-fit-inducing signature (based on the latest internet-meme) here>
Chrispy_
Gerbil Jedi
Gold subscriber
 
 
Posts: 1751
Joined: Fri Apr 09, 2004 3:49 pm

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 11:27 am

Chrispy_ wrote:The samsung magician software recognises a drive as overprovisioned as long as there's some unallocated space. If it's unallocated, the drive uses it however the hell it wants, I guess - so no need to TRIM or GC as far as I'm aware.

I'm sure that's true of new drives. But if the drive has previously been partitioned to use all of the space (as in this case), it is unclear what the best way is to ensure that the freed space is considered unallocated going forward. Unless the partitioning tool issues trim commands, there is another step required.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37464
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 2:39 pm

just brew it! wrote:
Chrispy_ wrote:The samsung magician software recognises a drive as overprovisioned as long as there's some unallocated space. If it's unallocated, the drive uses it however the hell it wants, I guess - so no need to TRIM or GC as far as I'm aware.

I'm sure that's true of new drives. But if the drive has previously been partitioned to use all of the space (as in this case), it is unclear what the best way is to ensure that the freed space is considered unallocated going forward. Unless the partitioning tool issues trim commands, there is another step required.


I have multiple Samsung 830 series SSDs, and I can confirm that a disk which was fully partitioned, and then resized after, is then recognized as having overprovisioned space by the Magician software. No zeroing or manual TRIMming was done.

I can't say it's the same for every drive model and manufacturer, but Samsung seems to have a very simple and explainable version of overprovisioning. ANY space not occupied by a partition is considered overprovisioning space. I actually have one of my drives partitioned oddly, due to a deleted partition mid-drive, and Magician IDs that as OP space.
I've been here long enough that I think I can forgo a signature.
Forge
Darth Gerbil
 
Posts: 7956
Joined: Wed Dec 26, 2001 7:00 pm
Location: SouthEast PA

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 4:02 pm

Hi guys thanks for all your detailed posts thus far.

For the record I have an Intel 530 series SSD. It is interesting what Samsung do with their Magician software and how it alows you to manually overprovision space. According to Forge it will also recognise an SSD which was fully partitioned, and then resized after, as having overprovisioned space. Unfortunately the Intel SSD Toolbox utility doesn't appear to recognise unpartitioned space and cannot perform a secure erase on boot drives or drives with a partion.

I guess the discussion is somewhat academic at this point because SSDs already have great endurance as proven by TRs Endurance Experiment. Its my first SSD, I like to tweak things and had my heart set on overprovisoning the drive, but forgot at the last minute and let the OS create a single partition. Damn :)
FDISK /MBR
Dirge
Gerbil Jedi
 
Posts: 1543
Joined: Thu Feb 19, 2004 3:08 am
Location: New Zealand

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 5:01 pm

I overprovision not for lifespan, but for performance. Even just a little OP space lets the Samsung go much longer without TRIMming before performance starts to sag.
I've been here long enough that I think I can forgo a signature.
Forge
Darth Gerbil
 
Posts: 7956
Joined: Wed Dec 26, 2001 7:00 pm
Location: SouthEast PA

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 5:58 pm

Hi huys, quick question on the topic: With TRIM enabled on an SSD, isn't overprovisioning basically the same as just having some free space on it? is it different on the hardware(controller) level?

As far as I understand it, the drive needs empty blocks in order for the wear levelling to function optimally. It shouldn't matter if the space is unallocated or simply marked free as the controller just scatters writes over available blocks to avoid wearing out some areas of the flash prematurely.

TRIM helps exactly with this - it actually tells the SSD to erase blocks which hold deleted data. For example, all areas marked as free in the Master File Table on an NTFS partition. This is needed because the file systems do not care about actually deleting the sectors containing the data, they only mark them as free in the FS lookup tables so they can be overwritten later. This was(is) not a problem for HDDs, but it messes with the SSDs' garbage collection schemes - they still see the blocks as occupied even though they aren't.

Unless I'm missing something, as long as the drive is TRIMmed regularly (which is automatic on modern OSes), just not filling it completely should have more or less the same effect on durability and performance as leaving some space permanently unallocated. The difference is that that you could, if needed, fill the extra space - performance should return to normal once the space is freed and TRIMmed afterwards. Is this really the case or is the TRIMming not as ideal in real-world conditions? :)
ASUS Rampage Formula|Q9550 3825MHz|4x2GB OCZ Reaper HPC 1066|GTX 460 1GB|ASUS Xonar DX|Intel SSD 330 180GB|5,5TB Storage
HP Elitebook 8530w|C2D T9900 3066MHz|2x4GB DDR2 800|Quadro FX 770M 512MB|Crucial C300 128GB + 750GB WD Black|15.4" 1920x1200
rado992
Gerbil
 
Posts: 24
Joined: Tue Aug 10, 2010 4:19 pm

Re: Over provisioning fully used SSD

Postposted on Tue May 20, 2014 6:30 pm

rado992 wrote:Is this really the case or is the TRIMming not as ideal in real-world conditions? :)

The idea here is to hide a chunk of space from the OS so that the controller on the drive can do what it needs to do without the OS poking its head in and making things go all pear-shaped.
It is one of the blessings of old friends that you can afford to be stupid with them. Ralph Waldo Emerson.
Captain Ned
Global Moderator
Gold subscriber
 
 
Posts: 20091
Joined: Wed Jan 16, 2002 7:00 pm
Location: Vermont, USA

Re: Over provisioning fully used SSD

Postposted on Wed May 21, 2014 2:24 am

Captain Ned wrote:
rado992 wrote:Is this really the case or is the TRIMming not as ideal in real-world conditions? :)

The idea here is to hide a chunk of space from the OS so that the controller on the drive can do what it needs to do without the OS poking its head in and making things go all pear-shaped.

I think this one is always going to divide opinion. There's a performance benefit (albeit slight) by leaving a small block free on the disk, compared to if the whole disk was one partition. The OP is looking to increase the life of his drive, but again I'm just not to sure whether it makes a significant difference (sorry)?

For the most part, Trim in the OS keeps on top of making sure everything is ok from a wear levelling perspective. Most drives also have a pretty decent garbage collection system within their firmware too. This own sites SSD endurance test seemingly shows that for (most) of the drives everything is still running as fast as it did when it was brand new. I'm assuming that each drive they are using for that is just one large partition? You'll probably want to replace the drive before it physically dies, as SSD's get cheaper at larger capacity points...
Intel C2E QX9770 @ 4.2Ghz, 1.4v | Gigabyte GA-X38T-DQ6 | 8GB Corsair DDR3 1600 | GTX 750 Ti 2GB | Crucial 512GB M550 SSD
geekl33tgamer
Gerbil Elite
 
Posts: 543
Joined: Tue Aug 25, 2009 7:25 pm
Location: England

Re: Over provisioning fully used SSD

Postposted on Wed May 21, 2014 5:23 am

IMO, overprovisioning isn't much use for consumer workloads. You need to be doing intensive database work, running virtual machines or extensive productivity with high IOPS before consumer drives noticeably slow down. I guess overprovisioning would help with that, but even when it slows down, IOPS rarely drop below 1000 so a tortured, full, completely fragmented SSD is still 10-15x faster than a 7200RPM mechanical drive. Sequentially you might see write speeds drop to mechanical speeds, but sequential speeds have never been that important in consumer desktop workloads unless you're concerned about having the highest numbers for your benchmark screenshot bragging rights.

Desktop use is typically bursty and the only thing that will really give you visible differences is a series of prolonged synthetic benchmarks or running a benchmark after a large amount of sequential copying that uses up all the free pages faster than the controller can reorganise and garbage-collect for more free space.
<insert large, flashing, epileptic-fit-inducing signature (based on the latest internet-meme) here>
Chrispy_
Gerbil Jedi
Gold subscriber
 
 
Posts: 1751
Joined: Fri Apr 09, 2004 3:49 pm

Re: Over provisioning fully used SSD

Postposted on Wed May 21, 2014 9:36 am

So there seems to be quite a bit of information on how Samsung and its Magician software behaves (what if I don't install the software?). How do Intel drives behave then, any documentation? Just leave space unpartitioned? Or just free space in an allocated partition would do?
Image
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
Flying Fox
Gerbil God
 
Posts: 24284
Joined: Mon May 24, 2004 2:19 am

Re: Over provisioning fully used SSD

Postposted on Wed May 21, 2014 9:56 am

Flying Fox wrote:So there seems to be quite a bit of information on how Samsung and its Magician software behaves (what if I don't install the software?). How do Intel drives behave then, any documentation? Just leave space unpartitioned? Or just free space in an allocated partition would do?


I'm just guessing, but I believe that reflects the greater number of Samsung drives in the machines we're looking at. Intel has a pretty large presence in OEM drives, but Samsung is massive, moves a ton of drives.
I've been here long enough that I think I can forgo a signature.
Forge
Darth Gerbil
 
Posts: 7956
Joined: Wed Dec 26, 2001 7:00 pm
Location: SouthEast PA

Re: Over provisioning fully used SSD

Postposted on Thu May 22, 2014 6:33 am

The reason I mentioned Samsung's Magician software is because (on Samsung drives, at least) it proves that you can just free up previously used space by resizing the partition and providing "unallocated space" and the manufacturer calls that "overprovisioning"

I infer that to mean "unallocated space = overprovisioned" and it doesn't matter whether the pages are dirty or clean because the SSD controller can do anything it wants with unallocated space.

Given that all SSD controllers work on the process of internal defragmentation routines to create empty pages, I would assume that unallocated space is all they need when improving these routines and therefore "overprovisioning" just means giving the controller more spare area to work with than it already has by not filling all the usuable capacity with a partition and volume.
<insert large, flashing, epileptic-fit-inducing signature (based on the latest internet-meme) here>
Chrispy_
Gerbil Jedi
Gold subscriber
 
 
Posts: 1751
Joined: Fri Apr 09, 2004 3:49 pm

Re: Over provisioning fully used SSD

Postposted on Thu May 22, 2014 4:43 pm

Chrispy_ wrote:The reason I mentioned Samsung's Magician software is because (on Samsung drives, at least) it proves that you can just free up previously used space by resizing the partition and providing "unallocated space" and the manufacturer calls that "overprovisioning"

I infer that to mean "unallocated space = overprovisioned" and it doesn't matter whether the pages are dirty or clean because the SSD controller can do anything it wants with unallocated space.

Given that all SSD controllers work on the process of internal defragmentation routines to create empty pages, I would assume that unallocated space is all they need when improving these routines and therefore "overprovisioning" just means giving the controller more spare area to work with than it already has by not filling all the usuable capacity with a partition and volume.


Sounds like a reasonable assumption, but I really wish I could confirm this was a feature of Intel 530 Series SSDs.
FDISK /MBR
Dirge
Gerbil Jedi
 
Posts: 1543
Joined: Thu Feb 19, 2004 3:08 am
Location: New Zealand

Re: Over provisioning fully used SSD

Postposted on Thu May 22, 2014 4:57 pm

If you want to make sure the drive is in a "clean" state before partitioning, do a secure erase using Intel's SSD utility. If the secure erase won't work (I've had trouble with it in the past), I think there's a manual TRIM function you can run on the drive's free space.
Steel
Global Moderator
Gold subscriber
 
 
Posts: 2326
Joined: Wed Dec 26, 2001 7:00 pm


Return to Storage

Who is online

Users browsing this forum: No registered users and 4 guests