Personal computing discussed

Moderators: renee, morphine, Steel

 
Igor_Kavinski
Minister of Gerbil Affairs
Topic Author
Posts: 2077
Joined: Fri Dec 22, 2006 2:34 am

Impact of compression on SSD write endurance

Wed Aug 20, 2014 2:37 am

From Tom's Hardware's NTFS compression on SSDs article:

There's one more point to consider, though. With information being compressed on the fly, you're consuming more of an SSD's available write cycles than if you were writing the files uncompressed. This could have negative implications on the drive's endurance. Although we're not necessarily concerned about larger SSDs, models with less capacity inherently have lower endurance ratings. Thus, NTFS compression could conceivably affect their useful lives more significantly.


The highlighted part has me confused. Isn't compression and decompression done in memory? How would that affect the SSD's write cycles?
 
arunphilip
Gerbil Team Leader
Posts: 259
Joined: Sun Jul 28, 2013 11:46 am

Re: Impact of compression on SSD write endurance

Wed Aug 20, 2014 4:47 am

I agree it appears counter-intuitive, but seeing the comments clears it up:

because when you modify even just one byte of a file that is compressed, you can end up changing a significant portion of the file, not just that byte. it's good if you can fit the change in one block erase; what if you can't? you'll end up writing more info on the "disk" then.


Correctomundo. Compression involves replace repeated occurrences of data with references to a single copy of that data existing earlier in the input (uncompressed) data stream. That's why it's not right to think of a compressed archive as a container that stores any given file into a discrete space. If anything, the files kind of overlap in a big mixing pot.

When you compress on the fly, you have to completely decompress all the files in an archive and recompress it when you're done. Hence it's all random transfers for the most part.
 
Igor_Kavinski
Minister of Gerbil Affairs
Topic Author
Posts: 2077
Joined: Fri Dec 22, 2006 2:34 am

Re: Impact of compression on SSD write endurance

Wed Aug 20, 2014 5:31 am

Hmm that makes sense. Now the question is, how much on average would compression decrease an SSD's lifetime? Seems like an idea to try in the next SSD experiment, don't you think?
 
Melvar
Gerbil XP
Posts: 381
Joined: Tue May 14, 2013 11:18 pm
Location: Portland, OR

Re: Impact of compression on SSD write endurance

Wed Aug 20, 2014 6:11 am

I doubt it really changes the overall writes that much. Modifying a file will cause more wear on a compressed drive, but writing a file in the first place will cause less. Given how the SSD endurance experiment turned out, it's probably the difference between the drive lasting until you die of old age vs 10 years longer than that.
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Impact of compression on SSD write endurance

Wed Aug 20, 2014 6:17 am

Those comments from the THG article that were quoted above are a little off base.

Per this article, NTFS compression works on 16-cluster chunks of the file at a time. So changing one byte of a file will alter -- at most -- 16 clusters on disk. For small files, yeah... this could end up being "a significant portion of the file" as stated in the first comment; but if the file is small the write amplification is less of an issue anyway.

The second comment seems to be assuming a ZIP archive (or similar archive-based compression), not NTFS on-the-fly compression of individual files.

It is also worth noting that many (probably most...) applications tend to rewrite entire files instead of altering bytes in place. If you change one letter in a Word document, the entire file gets rewritten on disk. (Databases would be one notable exception to this, but using on-the-fly file compression on a database would be a bad idea anyway, for performance reasons.)
Nostalgia isn't what it used to be.
 
Kougar
Minister of Gerbil Affairs
Posts: 2306
Joined: Tue Dec 02, 2008 2:12 am
Location: Texas

Re: Impact of compression on SSD write endurance

Wed Aug 20, 2014 11:20 pm

just brew it! wrote:
Per this article, NTFS compression works on 16-cluster chunks of the file at a time. So changing one byte of a file will alter -- at most -- 16 clusters on disk. For small files, yeah... this could end up being "a significant portion of the file" as stated in the first comment; but if the file is small the write amplification is less of an issue anyway.


That's not quite right if we are discussing SSDs? The page pool size of a modern SSD has increased to 16KB, particularly amongst the cheapest models. Also, SSDs must write at the block level. So 16KB times 512 pages per block means a block size of ~8MB per single write. So if I did that right a worst case scenario is 8MB * 16 clusters turns into 128MB assuming each cluster/page was located in a different block. Best case it'd be 8MB if it was all within the same block. Given everyone uses KB I assume it's bytes. If I got any of that right, then we'd have to factor in drive spare area because that would affect the probability of the 16 clusters/pages being spread across different blocks.
 
Wirko
Gerbil Team Leader
Posts: 296
Joined: Fri Jun 15, 2007 4:38 am
Location: Central Europe

Re: Impact of compression on SSD write endurance

Thu Aug 21, 2014 2:42 am

Kougar wrote:
Also, SSDs must write at the block level. So 16KB times 512 pages per block means a block size of ~8MB per single write.


The smallest unit for writing is one page, and the smallest unit for erasing is one block. Are there any reasons NTFS compression would make page-by-page writing impossible?
 
Wirko
Gerbil Team Leader
Posts: 296
Joined: Fri Jun 15, 2007 4:38 am
Location: Central Europe

Re: Impact of compression on SSD write endurance

Thu Aug 21, 2014 3:08 am

just brew it! wrote:
It is also worth noting that many (probably most...) applications tend to rewrite entire files instead of altering bytes in place. If you change one letter in a Word document, the entire file gets rewritten on disk. (Databases would be one notable exception to this, but using on-the-fly file compression on a database would be a bad idea anyway, for performance reasons.)


Another exception are log files. They can grow huge, which makes them tempting for NTFS compression. Server apps tend to append to them at a fast pace. Even with write cache, I suppose that log file writing puts a lot of stress (lots of writes) on the SSD. It would be interesting to know what's the effect of NTFS compression on that.

You're right about database file compression but it's not because of additional CPU cycles (if that's what you mean). It's because of this:

http://superuser.com/questions/693275/avoiding-extreme-fragmentation-of-compressed-files-on-ntfs

... and because of the very nature of database files: they are created large and empty, thus highly compressible, and later filled with data, becoming less and less compressible. A perfect recipe for heavy fragmentation.
 
Kougar
Minister of Gerbil Affairs
Posts: 2306
Joined: Tue Dec 02, 2008 2:12 am
Location: Texas

Re: Impact of compression on SSD write endurance

Thu Aug 21, 2014 11:05 pm

Wirko wrote:
Kougar wrote:
Also, SSDs must write at the block level. So 16KB times 512 pages per block means a block size of ~8MB per single write.


The smallest unit for writing is one page, and the smallest unit for erasing is one block. Are there any reasons NTFS compression would make page-by-page writing impossible?


You're correct so I'm a little off, technically speaking. However my point was still correct. Unless you are using a brand new or just secure-erased SSD, any SSD would have to clean out leftover data in the pages before it can write to them which necessitates a full block level erase. A drive would probably group most of the 16 cluster writes together in the same block so my worst case scenario wouldn't really happen. But even say two 8MB block erases per 16 clusters would still begin adding up over time.

SSDs try to balance cleaning out discarded page data (for performance) versus leaving it there as long as possible to minimize write amplification and hence un-needed wear on the NAND. If an SSD always kept the pages clear and ready for writes then it would be aggressively adding wear to the NAND.
 
derFunkenstein
Gerbil God
Posts: 25427
Joined: Fri Feb 21, 2003 9:13 pm
Location: Comin' to you directly from the Mothership

Re: Impact of compression on SSD write endurance

Mon Aug 25, 2014 11:13 am

would the same be true (if it is indeed true overall) on drives that do their own compression, i.e. Sandforce controllers?
I do not understand what I do. For what I want to do I do not do, but what I hate I do.
Twittering away the day at @TVsBen
 
Kougar
Minister of Gerbil Affairs
Posts: 2306
Joined: Tue Dec 02, 2008 2:12 am
Location: Texas

Re: Impact of compression on SSD write endurance

Mon Aug 25, 2014 11:42 am

derFunkenstein wrote:
would the same be true (if it is indeed true overall) on drives that do their own compression, i.e. Sandforce controllers?


I don't think it would change much There might be less wear involved, then again since it's compressing already compressed data maybe not.
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Impact of compression on SSD write endurance

Mon Aug 25, 2014 3:57 pm

Kougar wrote:
derFunkenstein wrote:
would the same be true (if it is indeed true overall) on drives that do their own compression, i.e. Sandforce controllers?

I don't think it would change much There might be less wear involved, then again since it's compressing already compressed data maybe not.

Really difficult to say whether it would help or hurt, given that the two compression engines may interact in unpredictable ways. For files which are written sequentially it is probably a wash; Sandforce's internal compression probably won't be able to compress the (already compressed) data much, but the drive will be fed less data to begin with. For files which are updated randomly, who knows.
Nostalgia isn't what it used to be.
 
Kougar
Minister of Gerbil Affairs
Posts: 2306
Joined: Tue Dec 02, 2008 2:12 am
Location: Texas

Re: Impact of compression on SSD write endurance

Tue Aug 26, 2014 2:27 pm

just brew it! wrote:
Really difficult to say whether it would help or hurt, given that the two compression engines may interact in unpredictable ways. For files which are written sequentially it is probably a wash; Sandforce's internal compression probably won't be able to compress the (already compressed) data much, but the drive will be fed less data to begin with. For files which are updated randomly, who knows.


Even if the SF drive doesn't need to open or re-write the old stored data each time it's "compressed" at the file system level (which I'm not sure it can do without risking data integrity), then it still has to make notations about the change in whatever table it uses to rebuild the original compressed data I'd imagine. Either way yeah, there should still be some wear involved even if there is less of it.

Who is online

Users browsing this forum: No registered users and 1 guest
GZIP: On