Our testing methods
I'm just going to come right out and say it: SSD testing is hard. In the mechanical era, storage was nice and predictable. Today's solid-state drives are rather more complex, especially since their performance depends not just on the test you're running, but also on the test you ran before that. And don't forget about the performance implications of the block-rewrite penalty inherent to flash memory—or the TRIM and garbage-collection routines designed to combat it.
To ensure consistent and repeatable results, the SSDs were secure-erased between almost every component of our test suite. This returns the drives to their factory fresh state, erasing any remnants of previous workloads.
For some benchmarks, we've deliberately tested drives in a used state to illustrate their long-term performance potential. Other tests create their own used states, usually by writing across the full extent of the drive before launching into a workload. In all cases, the SSDs were tested in the same states as their peers, ensuring an even playing field for all. I've even gone so far as to avoid running certain benchmarks overnight to ensure that some SSDs don't spend more time than others idling between tests.
Steps have also been taken to ensure that Sandy Bridge's power-saving features don't taint any of our results. All of the CPU's low-power states have been disabled, effectively pegging the 2500K at 3.3GHz. Transitioning in and out of different power states can affect the performance of storage benchmarks, especially when dealing with short burst transfers.
We're using fresh Rapid Storage Technology drivers from Intel and the latest firmware revisions on virtually all of the SSDs. The 120GB Force SSDs were tested with Corsair's 1.3 firmware, while the 240GB drives ran a slightly newer 1.3.2 revision. Corsair tells us the changes incorporated in 1.3.2 don't impact performance, so the discrepancy shouldn't tarnish our results. For the Intel 510 Series, our 250GB drive used PWG2 firmware, while the 120GB model ran PPG4. Intel doesn't offer firmware updates for the 510 Series, so you're stuck with what you've got. For what it's worth, the company says changes in the SSD's firmware haven't been substantial enough to warrant a public release.
We run all our tests at least three times and report the median of the results. We've found IOMeter performance can fall off after the first couple of runs, so we use five runs in total and throw out the first two. We used the following system configuration for testing:
|Processor||Intel Core i7-2500K 3.3GHz|
|Motherboard||Asus P8P67 Deluxe|
|Platform hub||Intel P67 Express|
|Platform drivers||INF update 18.104.22.1680
|Memory size||8GB (2 DIMMs)|
|Memory type||Corsair Vengeance DDR3 SDRAM at 1333MHz|
|Audio||Realtek ALC892 with 2.62 drivers|
|Graphics||Asus EAH6670/DIS/1GD5 1GB with Catalyst 11.7 drivers|
|Hard drives||Corsair Force Series 3 120GB with 1.3 firmware
Corsair Force 3 Series 240GB with 1.3.2 firmware
Corsair Force Series GT 120GB with 1.3 firmware
Corsair Force Series GT 240GB with 1.3.2 firmware
Crucial m4 128GB with 0009 firmware
Corsair m4 256GB with 0009 firmware
Intel 320 Series 120GB with 4PC10362 firmware
Intel 320 Series 300GB with 4PC10362 firmware
Intel 510 Series 120GB with PPG4 firmware
Intel 510 Series 250GB with PWG2 firmware
WD Caviar Black 1TB with 05.01D05 firmware
|Power supply||Corsair Professional Series Gold AX650W|
|OS||Windows 7 Ultimate x64|
Thanks to Asus for providing the systems' motherboards and graphics cards, Intel for the CPUs, Corsair for the memory and PSUs, Thermaltake for the CPU coolers, and Western Digital for the Caviar Black 1TB system drives. Also, thanks to all the SSD makers for providing their drives.
We used the following versions of our test applications:
- Intel IOMeter 1.1.0 RC1
- HD Tune 4.61
- TR DriveBench 1.0
- TR DriveBench 2.0
- TR FileBench 0.2
- Qt SDK 2010.05
- MiniGW GCC 4.4.0
- Duke Nukem Forever
- Portal 2
The test systems' Windows desktop was set at 1280x1024 in 32-bit color at a 75Hz screen refresh rate. Vertical refresh sync (vsync) was disabled for all tests.
Most of the tests and methods we employed are publicly available and reproducible. If you have questions about our methods, hit our forums to talk with us about them.