We initially tested these drives with revision 1275 of Indilinx’s firmwaretheir shipping configurations. Curious to see whether the latest firmware would affect performance, we secure-erased the Vertex to put it into factory-fresh form, flashed it with the most recent 1370 firmware, and ran it through our test suite once more. The newer firmware didn’t affect the drive’s performance in any of our tests, though.
Up to this point, we had tested all of the SSDs in a used state. We secure-erased each drive before its run through the gauntlet, but the first test in our benchmark suiteHD Tach’s full disk benchmark, which we run three timesfills all flash pages by writing across the entire drive.
Testing SSDs in a used condition is important because of the block rewrite penalty associated with flash memory. In an SSD, flash memory is split between 4KB pages organized into 128-page, 512KB blocks. Empty pages can be written to directly. However, occupied pages can only be written on a block-by-block basis, which involves reading the contents of the block, modifying the necessary pages, and rewriting the block as a whole. These extra steps take time, resulting in a performance penalty.
Deleting files simply marks pages as available, rather than actually clearing their contents, so any SSD will encounter the block rewrite penalty after prolonged usage. Even a drive that, cosmetically, shows plenty of available storage capacity can be out of empty pages. Once in this used state, every write request will be slowed by the block rewrite penalty.
We believe that testing SSDs in this used state is the best way to evaluate their performance. A solid-state drive may post impressive benchmark scores when it’s fresh from the factory, but it’s more important for SSD controllers to deal gracefully with a lack of empty pages. As we discovered, some do a much better job than others.
We briefly explored the performance of used versus factory-fresh SSDs in our initial round-up, comparing the 4k random write performance of each drive in each state. The Indilinx drives showed no obvious signs of weakness; response times slowed when the drives were used, but by a lesser degree than drives powered by Intel or Samsung controllers. In fact, in a fresh state, the Barefoot drives had the quickest response times of the lot.
To determine whether their used state was responsible for the Indilinx drives’ poor file copy performance, we ran a handful of freshly-erased drives through iPEAK and FC-Test. These drives were tested with the same system configuration as our initial round-up, this time with the OCZ Vertex updated to Indilinx’s 1370 firmware revision.
Our iPEAK results represent an average service time across all nine of our custom workloads, four of which feature file copy operations. In a factory-fresh state, the Indilinx-powered Vertex isn’t substantially slower than the Intel drive or the Samsung-powered Corsair. However, the Vertex’s service times are nearly four times longer when the drive is in a used state. The Intel and Corsair drives slow down, too, but only to about twice their original service times.
Next we’ll look at some real-world file creation, read, and copy operations courtesy of FC-Test. Here we’ve presented the average throughput across all five of the app’s test patterns, which contain a mix of large and small files, and large and small numbers of files.
The Vertex offers quicker file creation speeds than the X25-M when both are in a fresh state. However, while the Intel drive’s performance only drops by about 20% in a used state, the OCZ’s falls by nearly 70%. The Samsung controller inside the Corsair P256 doesn’t particularly like being in a used state, either; its file creation performance drops by nearly 50% going from fresh to used.
In theory, flash memory’s block rewrite penalty should have no impact on read performanceused drives should read as quickly as fresh ones. And they do, at least with the X25-M and P256. However, the Vertex’s read performance when used lags behind that of a fresh drive by about 10%. That’s not a huge margin, but it’s a performance gap that, as far as we know, shouldn’t exist at all.
Copy tests combine read and write operations, and as one might expect, the Vertex doesn’t fare so well here. Its used performance is about 60% slower than a fresh drive, which is a much bigger margin than either the Corsair or Intel drives. The former’s performance drops by 30%, while the latter is only 15% slower.
The Indilinx controller would appear to have performance problems when dealing with drives in a used state. But is the controller to blame? Our testing was conducted with Windows XP Service Pack 2 and an older 220.127.116.112 AHCI driver revision for our system’s ICH7R south bridge chip. To make sure we weren’t missing any critical software updates for our test platform, we brought the OS up to Service Pack 3 with all the latest hotfixes and installed the most recent Intel 18.104.22.1689 AHCI drivers, in addition to new chipset drivers. Once more, we secure-erased the Vertex and ran it through our benchmark suite, only to find that this new configuration’s performance was essentially identical to that of the old one. Even with the latest drivers, firmware, and XP updates, the Indilinx controller’s used-state performance in file creation, read, and copy operations was still rather poor.
All of this testing was conducted on a relatively old system based around a Pentium 4 processor, 955X chipset, and ICH7R storage controller. While that’s hardly cutting-edge hardware, this very same test system had no problems wringing excellent performance from SSDs based on Intel and Samsung controllers.
Could our use of Windows XP explain the Indilinx controller’s woes? Perhaps. We’re currently conducting Vista testing to see how these SSDs fare. Other sites have also reported better FC-Test performance from the Vertex running under Vista x64. However, based on some initial results of our own Vista testing, we remain skeptical. And the problem with XP performance remains.
Exploring the problem
The block rewrite penalty affects all flash-based SSDs, but as we’ve discovered, some do a better job of managing this problem than others. Indilinx’s Barefoot controller appears to be much worse off on this front, at least when running on Windows XP, than controllers from Intel and Samsung. But why?
One potential problem is Windows XP’s default 63-sector offset, which starts the first partition right in the middle of an SSD page rather than at the beginning of one. Since flash can only be written in 4KB pages, the overlap caused by an initial misalignment can result in more pages being written than necessary for a given write request. The performance implications of this situation are particularly troubling when the drive has no empty pages. Microsoft suggests misalignment can drop SSD performance by up to 50% when block rewrite penalties come into play.
One solution to XP’s default partition offset is to create partitions manually with a custom offset. Doing so is possible using a Diskpart utility available from Microsoft. Such tweaking isn’t necessary for all SSDs, though. When asked whether the X25-M takes XP’s 63-sector offset into account, Intel told us that its SSD tech is alignment-agnostic, just as long as there’s some alignment present in the operating system. Also, the Samsung controller doesn’t seem to be as adversely affected by XP’s default offset as the Indilinx chip in the Vertex.
In fact, custom partition alignment may not even be necessary with the Vertex. When asked about the issue, OCZ Vice President of Technology Development Michael Schuette also suggested that an SSD could side-step XP’s 63-sector offset by employing a default 128KB offset in its firmware. So why hasn’t such an offset been incorporated into the Vertex’s firmware? Because that firmware comes from Indilinx; according to Schuette, OCZ doesn’t have access to the source code.
We fired off an email to Indilinx regarding the issue, specifically asking about XP’s 63-sector offset and the comparatively poor used-state performance we observed from the Barefoot controller:
I’ve been testing a couple of Indilinx-powered SSDs (OCZ’s Vertex and Super Talent’s UltraDrive ME) and have discovered that the drives offer much lower performance in a used state than when in factory-fresh form. This is of course an issue for all SSDs, but the Indilinx drives suffer a much greater performance drop from fresh to used than SSDs powered by Samsung and Intel controllers.
I’ve been testing with Windows XP, whose 63-sector offset is apparently to blame for the Indilinx controller’s disproportionally poor performance in a used state. Does the Indilinx controller take XP’s 63-sector offset into account? Is it possible to add a default offset to the controller’s firmware so that XP users don’t have to resort to manual partition alignment for optimal performance?
Indilinx’s replied with the following:
We support TRIM Command to recover the performance drop after fragmentation(used).
This command shows the original performance like Intel.
Rather than directly addressing XP’s 63-sector offset and the controller’s poor used-state performance, Indilinx is essentially suggesting that we avoid a used state altogether. The TRIM utility erases any pages marked as available but occupied with old data, reducing the chance of a block-rewrite penalty to slow a write request. This program effectively pushes a drive toward its factory-fresh state without erasing the data you actually want to keep.
That all sounds well and good, but Indilinx’s TRIM utility doesn’t appear to be ready for prime time. In a post releasing a beta version of the app in its SSD forums, an OCZ rep notes that he repeatedly encountered data corruption when running the utility on a 64-bit Vista system with an ICH10R south bridge. In fact, although the app is called 100% safe for 32-bit operating systems, OCZ says it’s only “looking around 50% safe” for 64-bit operating systems. Support for 64-bit Vista and Windows 7 is listed as “limited”, and Windows XP 64-bit isn’t supported at all. Incidentally, we tried the latest TRIM utility on our original test system, a 32-bit XP machine, and it didn’t work. OCZ’s suggestion was to run the app up to six times, rebooting after each.
Windows 7 promises native support for the TRIM command, but the OS hasn’t been finalized, and Indilinx’s home-brewed utility doesn’t look like a reliable solution for those running other operating systems. Based on the accounts of OCZ’s own SSD forum admin, I wouldn’t trust the app with a 64-bit operating system. The fact that data corruption has been observed would make me wary of running the utility on a 32-bit OS, as well.
Although the TRIM utility clearly isn’t a robust solution, it appears to be the only one Indilinx is offering to address the poor used-state performance of Barefoot-based drives in Windows XP. And before you argue that Windows XP is far too antiquated to matter, keep in mind that it’s still widely used by PC enthusiasts and mainstream folks alike. If Intel and Samsung have more effectively addressed used-state performance with their SSD controllers, shouldn’t we expect the same from Indilinx or anyone else hawking an SSD controller?
We asked OCZ’s Michael Schuette how much pull the company has with Indilinx in terms of getting new features (like a default partition offset) introduced to firmware and were told that OCZ is “doing a huge part of the debugging and performance analyses” related to the drive. Schuette went on to say that, “With more access to the details on the hardware and firmware, I am sure we could make this controller really fly, it is pretty good already, but we are stuck behind the iron curtain.”
Indilinx’s Barefoot controller appears to be, charitably, a work in progressas are the drives based on it. At present, their used-state performance issues have not been addressed adequately. Indilinx may be the only party capable of rectifying the problem via firmware changes, but that doesn’t absolve OCZ, Super Talent, and others selling Barefoot drives of responsibility for a glaring flaw in the products they are selling to consumers.
If you want to dabble with Windows 7 release candidates, fine-tune partition alignments, or roll the dice with a TRIM utility that could corrupt your data, SSDs based on Indilinx’s Barefoot controller certainly have intriguing potential. However, for those seeking a solid-state drive that doesn’t require extensive tweaking for optimal performance, products based on the latest controllers from Intel and Samsung are much safer betsincluding, potentially, OCZ’s own recently introduced Summit series.
In highlighting the used-state performance penalty associated with the Barefoot controller, we’ve focused exclusively on Windows XP performance with our ICH7R-based storage test system. But that’s not all we’ve been doing. A new round of SSD testing has already begun on an updated system built around a Core 2 processor, ICH10R south bridge, and Vista x64. Stay tuned for more.