TR Forums

Two people over on the front page comments said games actually can load faster with faster SSDs. It all went a bit flamewar-mode and it doesn't look like anyone took them seriously, but I'm thinking it deserves a bit more attention.

Obviously conventional wisdom in these parts is that faster (PCIe) SSDs aren't going to do anything at all for games. AFAICT, this is mainly based on TR's testing. The bit that leaves some room for doubt is that TR hasn't actually tested that many games in this way, and they only appear to have tested one loading sequence for each of them (do we know which one?). That leaves plenty of room for some games to see improvements in some areas (areas meaning launch -> menu versus menu -> world, I think).

So, anyone who has gone from a SATA SSD to a PCIe one: have you noticed any improvements in normal or gaming use, and if so, what were they? The most relevant bit would be what part of what game, but the rest of the hardware could be interesting too (if most of a game's load time is streaming and decompression, some things could speed up decompression and make storage a bigger issue).

TPU tested level load times with Intel's SSD750, and vs cheap SATA SSDs the difference to the slowest drive in each test was never more than 2s, and half the time it was 1s or less.

Crysis 3 and BF4
https://www.techpowerup.com/reviews/Int ... TB/12.html

Wolfenstein & Arma 3
https://www.techpowerup.com/reviews/Int ... TB/13.html

This is better illustrated by having side by side video. Techytalk puts up Samsung's fastest 960 Pro vs a cheap Crucial MX300 and there's really nothing to it. Most of the time the differences are in the hundredths of a second:
https://www.youtube.com/watch?v=ecCA0gx_eZk

Actually, if the big bottleneck to level load times is decompression, then the biggest advantage will come from more cores. Just look at the 8/10 core i7 Extremes doing WinRAR or 7Zip. Besides that it seems that any cheap SSD, even the SATA3 ones are fully capable of saturating the CPU when it comes to decompression. 7Zip is one of the fastest archivers out there, but 7600MIPS in 7z translated to only about 83MB/s decompression speed, which means even the 10-core i7-6950x (49000MIPS) will max out at just 535MB/s assuming 100% of the work is just decompression, which it probably won't be.

I'm unimpressed by the TPU stuff because they're using a Pentium G3240 and 1333 CL9 RAM. If only some systems are going to show differences, that sure isn't going to be one of them. The video is much more convincing, and looks to bring our game sample size up a fair bit.

It's far from a given that decompression is the big limit. There are all kinds of other shenanigans games might be doing in loading, and general-purpose compression isn't the most useful thing on the bulk of game data. Decompression has highly variable speed, but 500 MB/s between ten cores puts an algo on the slower end of things, IIRC. A fast one could be nearly that on each core.

More editing: I realized that in my first post, I implied that decompression was the big non-storage cost. It's a likely one, but I didn't mean to make it sound that definitive.

Yet more editing: Here are some speed numbers on an Athlon 64. Stuff can be as fast as this (I use that instead of swapping to disk).

As someone who may have participated with some less than stellar comments (I promise I won't post from work anymore, and my sincere apologies to Josiah), I should at least state my position. I time-trialed with stopwatch loading Cities Skylines from a Kingston HyperX 3K 120GB Sandforce SSD and from a Plextor/Lite-On PCIe NVMe 512GB Marvell-controller SSD. I have something like 20 out of 39 mods enabled, so it's a CPU-heavy load, but the PCIe SSD was clearly faster (and smoother, with less laggy stoppage during the process.) On an FX-8320 @ 4.2 GHz system with 32GB DDR3-2133, the times were about 3.5 minutes vs 5 minutes, favoring PCIe. On a friend's Sandy Bridge i7-2600 @ 4.5 GHz with 16 GB DDR3-1866 and the same two drives moved over, the times dropped pretty dramatically. I believe that comparison was about 2.5 minutes vs 3 minutes, again favoring PCIe, but I can't find where I wrote the second set of times down. Clearly Sandy Bridge vs FX made a bigger difference in load times for this one CPU-heavy scenario than SATA vs PCIe NVMe, but there was still an improvement worth having. That is the only game that I've actually stopwatched.

synthtel2 wrote:

Yet more editing: Here are some speed numbers on an Athlon 64. Stuff can be as fast as this (I use that instead of swapping to disk).

Those benchmarks are based on compressing TEXT. Actual programs, and especially textures and audio are much less compressible and slower to decompress. You can even go to the lzbench github page and see the folder for the large text file they are compressing. There is no way you are getting 3000MB/s on actual game assets.

The benchmark result in the forum link I posted was using 7Zip, which is already one of fastest archivers around, and can load up as many cores as a system has.

MOSFET wrote:

I time-trialed with stopwatch loading Cities Skylines from a Kingston HyperX 3K 120GB Sandforce SSD and from a Plextor/Lite-On PCIe NVMe 512GB Marvell-controller SSD.

Aha, that is interesting! I happen to know Unity has some shenanigans in that area (definitely not just decompression). That gives me a few more ideas for research, which I'll see about getting done tomorrow.

jihadjoe wrote:

Those benchmarks are based on compressing TEXT. Actual programs, and especially textures and audio are much less compressible and slower to decompress. You can even go to the lzbench github page and see the folder for the large text file they are compressing. There is no way you are getting 3000MB/s on actual game assets.

The benchmark result in the forum link I posted was using 7Zip, which is already one of fastest archivers around, and can load up as many cores as a system has.

Wanna bet? That's 3000 MB/s per core, and it multithreads just fine too. Can it decompress a game at 3000 MB/s? Probably not. Can it blow 7z out of the water? Hell yeah it can. I'll do some testing tomorrow on actual game data.

I look forward to that, because:

ANSI-C and C++ source code in LZMA SDK is subset of source code of 7-Zip.

If you're thinking that LZ4 and LZMA are the same thing, they're not. (7z is LZMA.)

You actually made me read up on LZ4. And right, it is blazing fast. Compression ratios are a bit down on traditional zip-like archives though, but it would make sense to distribute files using zip and then convert to LZ4 on install. Either way I don't see a huge benefit for PCIe storage for desktop use, many of those NVMe drives can't even reach their full transfer speed at QD1.

I got the PCIe SSD at the same time that I upgraded my primary gaming PC from a Core i7-4770K with PC3-14900 to a Core i7-6700K with PC4-21300 last year, so I cannot isolate the faster SSD as the only factor in performance improvement.  The newer system with Samsung SM951 SSD loads maps and objects in Guild Wars 2 noticeable faster than the older system with Samsung 840 SSD does.  The 1-2 second difference between the two is much less than the 10-20 second gap back to Core i5-3570K and Core i7-2600K systems that have small SSDs lacking space for much more than the OS so load the game from 7200 rpm hard-drives.

Pricing of PCIe+NVMe SSD's has gotten a lot closer to SATA+AHCI this year.  The Intel 600p PCIe+NVMe SSDPEKKW512G7X1 is just 26¢ per GB.
http://pcpartpicker.com/products/intern ... 80&sort=a7

SSDs using slower SATA+AHCI communication like the Crucial MX300 are 24¢ per GB.
http://pcpartpicker.com/products/intern ... 80&sort=a7

P.S.:  Conveniently enough, today Newegg has put the 0.51 TB version of the Intel 600p on sale for just $133.  Buy yours today (assuming that you already have a motherboard with an M.2 slot).

If I was going to use a PCIe SSD, I'd only be interested in an NVMe one anyway. AHCI is too much of a bottleneck in the real world, mixed-load performance. Sure, you can get 1GB/s or more instead of the 550MB/s of AHCI and SATA but if you monitor your SSD and log/graph the performance, you'll see how very very rarely an SSD ever reads/writes at its peak speeds in daily use.

No, if I had a PCIe SSD I'd use it for my OS and key applications, retiring the older SATA SSD to games-library duties. Loading game levels is (as already mentioned) almost entirely bottlenecked by your CPU's IPC through decompression of the compressed gamefiles. If you want faster level loads, stick your games on a low-overhead cheap SATA SSD and then spend the money on an overclocked i7.

There is no real difference between PCIe and SATA SSD media in almost every gaming title and situation out there under a single-user environment. Majority of the gaming titles aren't pulling enough I/Os and/or bandwidth to make a SATA SSD woefully inadequate. The few titles that do benefit from it the returns you get aren't really that much to make it worthwhile. It may help reducing those annoying hic-ups when the game needs to pull massive data (GiBs worth) from non-volatile memory. It is nothing like the jump from a HDD and SATA SSD media.

The benefits of the PCIe SSD are wasted for the most part for gaming usage patterns. They only make a noticeable difference if you are gaming while doing I/O intensive applications/task in the background.

PCIe SSD media is the spiritual successor to the high RPM SCSI HDDs of the old days. They didn't offer much benefit for mainstream workloads to justify the extra cost but were definitely worth it if you were multi-tasking and doing heavy I/O workloads.

Like so many things, "it depends". Yes, many games are CPU bound during loading. But different game engines, different CPUs, etc. will all factor into how much of an effect SSD speed has.

Krogoth the Unimpressed wrote:

The returns you get aren't really that much to make it worthwhile.

The price difference this weekend is just 9%.  When PCIe+NVMe SSDs were priced at more than double what similar SATA+AHCI SSDs cost, I would have agreed with you.  With the prices as close as they are today, I'd say, "Why not?"

JustAnEngineer wrote:

Krogoth the Unimpressed wrote:

The returns you get aren't really that much to make it worthwhile.

The price difference this weekend is just 9%.  When PCIe+NVMe SSDs were priced at more than double what similar SATA+AHCI SSDs cost, I would have agreed with you.  With the prices as close as they are today, I'd say, "Why not?"

That's assuming you have a NVMe-capable platform on hand and/or have enough PCIe lanes to throw around.

Well, yeah... if the price differential is small and you have the slot, there's really no reason not to.

I would not pay a large premium or upgrade a platform just to get it though.

I went from an X25-M G2 to a 950 Pro and noticed a huge difference. But I also went from a wolfdale cpu to 6600k, ddr2 to ddr4 at the same time.

Hah, yes. I think the performance difference may have something to do with skipping forwards six generations of processor, quadrupling your storage connectivity standard, quadrupling your RAM bandwidth and skipping forward approximately four vaguely-definied SSD controller generations and about as many generations of NAND too.

"Yes, this new V8 Camaro is faster than my old donkey and cart"

Chrispy_ wrote:

"Yes, this new V8 Camaro is faster than my old donkey and cart"

Anyone got tips on turbo charging a donkey?

MileageMayVary wrote:

Chrispy_ wrote:

"Yes, this new V8 Camaro is faster than my old donkey and cart"

Anyone got tips on turbo charging a donkey?

Feed it lots of beans and light a match behind it?

That begs the question, what kind of ECC do we need to course-correct a ballistic donkey?

Redocbew wrote:

That begs the question, what kind of ECC do we need to course-correct a ballistic donkey?

Rocket Jesus.

Alright, I got a round of compression testing done (I was curious about these results anyway). It's all on a ramdisk, and the test data is maps.bundle from Rust (an increasingly unimpressive game, but useful for various testing here). I think the data in it is most kinds of graphical data with the exception of textures, and evidently they're not pre-compressed (I tried Borderlands 2 data first, but it had already been compressed). One other thing to note is that 7zip compression (but not decompression) auto-parallelized and the others didn't, so 7zip's per-core compression speed is a bit lower than that. Also -9 has that behavior with a lot of algos (not slowing down decompression), so that bit isn't 100% fair in the opposite direction. Gzip is deflate, same as plain old .zip, for the record. This testing is on a Pentium G3258 at 4.3 GHz (3.8 uncore) with 8 GB of DDR3-2133 at 9-11-10-28. The software is whatever's in up-to-date Arch Linux as of last Monday. I've modified the following commands' output by replacing a bunch of superfluous 7zip output with [...]. Some cleanup was run between commands in another terminal.


.... or, more concisely:



This makes LZ4 and LZMA each look potentially very useful for games, but in very different ways.

One more wrench in the works as far as applying this to games is that it doesn't do much for most texture data. Textures are messy. S3TC has a rather bad compression ratio, you can't apply something off-the-shelf like PNG to further compress an S3TC image, and good S3TC compressors are pretty slow. AFAIK, most games either just deal with the space taken by S3TC textures or use a proprietary tool to further compress them on disk. There are other options though, and they have the potential to make any theoretical numbers in this space look pretty different.

On the theory that Unity shenanigans are disk-heavy, the next thing I want to try is loading Rust from a ramdisk (it's the Unity game I own that loads slowly). There may be trouble with that, though, because I only have 8 GB of RAM, Rust takes 5 GB, and putting it on an LZ4'ed ramdisk might (or might not) have some heavy side-effects. I'll see what I can do, though (probably tomorrow).

I'll say this - LZ4 is great. I blindly enable it on *everything* I run in production (which includes > 40 PiB of ZFS-backed storage) since it's so friendly to CPU resources and can increase speeds dramatically when data is compressible.

Even on the nodes that run 10 GB/s read/write simultaneously, LZ4 is running. The compression and decompression speeds have never been a bottleneck for anything except minimum latency (and only a bit)...which matters little in my world.

I think testing Rust on a ramdisk is a failure. 2.5 GB of RAM remaining isn't enough for it to do its thing, and decompression from LZ4 doesn't seem to work out either (presumably due to my CPU only being 2C2T). Everything I've got handy that's significantly smaller either loads fast enough it's irrelevant or isn't a game I've tried yet (I need to work through some Steam backlog).