A quick look at chipset PCI Express performance

In addition to support for Sandy Bridge CPUs, Intel’s latest P67 Express core-logic chipset brings with it two major improvements over the old P55: a couple of 6Gbps Serial ATA ports and full-speed PCI Express 2.0 lanes throughout. Those features have been present in AMD chipsets since the SB850 south bridge debuted some ten months ago, making Intel more than a little late to the party. At least on the SATA front, the wait was worth it. While comparing four mid-range P67 boards, we discovered that Intel’s 6Gbps Serial ATA ports are faster than AMD’s.
So, what about the P67’s PCI Express lanes? Those familiar with the P55 will remember that it, too, has PCIe lanes claiming second-generation status. But those lanes run at half speed and offer no more bandwidth than gen-one PCIe links. The P67’s PCI Express 2.0 lanes are cranked up to the standard’s full 5.0GT/s data rate, setting up a three-way comparison with the P55 and AMD’s SB850.

We had planned to incorporate a new batch of PCIe 2.0 performance tests in our initial motherboard round-up. However, it took a little time to find an appropriate peripheral card with a PCIe 2.0 x1 interface. And then it took a little longer for that card to arrive at the Benchmarking Sweatshop.

Incidentally, there are plenty of PCIe peripherals available these days—just few that profess to support second-generation PCI Express. Most of the PCIe solid-state drives use x4 interfaces that make isolating a single lane impossible, so we settled for the next best thing: a RAID card attached to a couple of 2.5″ SSDs.

The only suitable candidate we could find was Syba’s SY-PEX40032, which situates dual 6Gbps Serial ATA RAID ports behind a PCIe 2.0 x1 interface and sells for $40 at Newegg. You’ve probably never heard of Syba, and neither had we. The card uses the Marvell 88SE9128 6Gbps SATA RAID controller we’ve seen on just about every enthusiast-oriented motherboard to be sold in the last year, though. With a couple of SandForce-based Agility 2 and Vertex 2 solid-state drives from OCZ arranged in a striped RAID 0 array, we had ourselves nice little PCIe 2.0 SSD. We’ve seen each of those solid-state drives push transfer rates of nearly 200MB/s on their own. Teaming them in RAID 0 should be enough to exceed the 250MB/s of one-way bandwidth offered by a first-generation PCI Express lane. PCIe 2.0 offers 500MB/s of bandwidth per lane, and this array is our best shot of exploiting it.

Before creating our RAID 0 array, we secure-erased the SSDs to ensure optimal performance. We settled on a stripe size of 32KB (64KB was the only other option in the Marvell BIOS) for the array because it offered the best performance in the benchmarks we’d selected for testing.

To represent Intel’s P67 Express chipset, we called on the company’s own DP67BG motherboard using the same system configuration as in our mobo round-up. The Asus 890GX motherboard from that article sat in to represent AMD’s latest south-bridge PCIe implementation. However, we had to find a different P55 board to get a direct path to the chipset’s built-in PCIe lanes. Most enthusiast-oriented P55 boards employ switch chips that share and distribute the P55’s PCIe lanes between multiple expansion slots and onboard peripherals. That kind of middleman isn’t terribly helpful when you’re trying to isolate a single PCIe lane, so I dug through the mess of boxes in my office and unearthed a MSI P55-GD65 motherboard with an x1 slot leading straight to the P55 PCH.

First up: HD Tune, which we’ve been using to test hard drive and SSD performance in storage reviews for nearly a year now. We’re sticking with read speed tests to avoid writing to the array and dodge the block-rewrite penalty inherent to flash-based storage. Unfortunately, TRIM commands can’t yet be passed through a RAID controller.

The P67 Express’ PCIe 2.0 lanes allow our SSD RAID array to achieve much higher read speeds than when it’s attached to the AMD chipset. That result is a little unexpected given that the 890GX’s SB850 south bridge claims to offer a full-speed PCIe 2.0 implementation. In fact, the AMD platform trails the P67 by a much larger margin than it enjoys over the P55, which has a built-in handicap.

HD Tach produces similar results. The P67 has a substantial lead over the 890GX, while the P55 brings up the rear.

Our SSD array hits higher speeds here than in HD Tune. Even so, we’re still a ways off PCIe 2.0’s maximum per-lane bandwidth of 500MB/s in each direction.

Interestingly, HD Tach reveals higher CPU utilization for the AMD chipset. We wouldn’t normally make much of deltas in CPU utilization between platforms using different processors. In this case, however, the gap is much larger than we’d expect between a six-core Phenom II and the quad-core, Hyper-Threading-equipped Core i7 CPUs used on the Intel boards.

For what it’s worth, when we ran the same tests on a P55 board that employs a PCIe switch chip, we saw even higher transfer rates than on the P67 Express. Motherboard makers appear to have done a good job of working around the P55’s limited PCIe bandwidth. We should also point out that the RAID 0 performance of Marvell’s 88SE9128 is pretty poor considering what just one of those SSDs can do on its own. With each drive capable of pushing reads at 190MB/s, some performance is clearly being left on the table.

Regardless, the Marvell controller is a popular solution that performs substantially better when connected to the PCI Express lanes attached to Intel’s P67 Express chipset than it does on competing platforms. The P55 is much slower because its PCIe 2.0 lanes have been dialed back to half speed. However, the SB850 doesn’t have that excuse.

Perhaps we shouldn’t be surprised. AMD inherited a history of chipset issues when it acquired ATI, while Intel’s core-logic record is nearly spotless through a multitude of generations. The P67 Express looks poised to live up to that reputation.

Comments closed
    • Dissonance
    • 9 years ago

    Although I’m still waiting for Asus to confirm specific lane assignments, I have done some additional testing with the RAID card in the x4 slot. It’s faster, but still a ways off the P67.

    HD Tune: 153MB/s burst, 161MB/s average, 178MB/s max
    HD Tach: 228MB/s burst, 166MB/s average, 6% CPU

    The x1 RAID card shouldn’t be able to extract any additional bandwidth from the x4 slot, so I suspect it’s faster because those lanes branch off the 890GX north bridge, while the x1 slot hangs off the SB850.

    • libradude
    • 9 years ago

    Sorry if it has been mentioned elsewhere, and I apologize if could go look at whitepapers and see what the chipset is supposed to be capable of – but I figured I would ask here in case someone knows… do the improvements in PCIe throughput extend to the H67 chipset, or just P67? I recently bought an i5-760 to start a new system build but am now thinking about selling that and going with Sandy Bridge parts for the obvious speed increase and power efficiency improvements. This will be for a mini-itx box and those boards usually are built around the Hxx chipset instead of Pxx, hence my question. Thanks in advance!

    • MarcHFR
    • 9 years ago

    Hi

    Sorry but it seems there is some problem in your test.

    Using an ASROCK SATA 3 PCI-E x1 card on ASUS 890GX motherboard i get 200 MB/s on HD Tune with one single C300 128 Go drive. Using bigger block size (64 KB by default) in HD Tune i can get more than 300 MB/s.

    Using IOMETER 2MB read test i get 342 MB/s on P67, 328 MB/s on 890GX. On 2.5 GT/s line (PCI-E x1 on X58 Southbridge) i get 195 MB/s.

      • axeman
      • 9 years ago

      Maybe the Asus board is borken (sic). OTOH, this doesn’t surprise me at all, given DAAMIT’s history of messing pretty much everything up in some way. It’s pretty irrelevant, but I’d be curious how Nvidia’s implementation works (worked). Not that they were perfect, but after NForce 4, I don’t recall any major issues, unlike ATi with their broken AHCI, and terrible USB performance that lasted…until…did they ever fix this?

    • Ruthless Reuban
    • 9 years ago

    Let’s consider this more carefully. I think that PCI Express SATA controller was put on Asus Motherboard’s PCI Express x1 slot. Looking at board’s available PCI Express lanes, we have following:

    Total available: 24 (22 on North bridge, 2 on South bridge)
    PCI Express Graphics: 16 (North bridge)
    PCI Express x4 slot: 4 (North bridge)
    LAN Controller: 1 (North bridge?)
    SATA/PATA controller: 1 (North bridge?)

    We have 2 lanes left. Main question is: is board’s PCI Express x1 slot connected to South bridge OR North bridge?

    If it is connected to South bridge, then data must first travel to South bridge, after that to North bridge and finally to system. That is very slow way. It is hard to say where this x1 PCI Express connector is connected.

    However, PCI express x4 connector cannot be connected to any other than North bridge. That is why I suggest re-running tests with controller connected to PCI Express x4 connector. That x1 connector is not meant for fast devices, x4 connector is. This is probably testing AMD’s South bridge performance, it still trounces P55. But North bridge performance may be much higher.

    • Bensam123
    • 9 years ago

    Love the article.

    I would note however I’ve seen cheap raid cards perform worse in raid then using a single drive. Did you guys consider looking at single drive performance using the card? It could really just be a ultra terrible card and may not even have full pci-e 2.0 support either.

    Curiously does this affect video card performance on newer high end card or even sli/crossfire on a stick variants?

    • flip-mode
    • 9 years ago

    Wow, so, the AMD platform is an embarrassing conglomeration of fail, while the Intel platform is an infuriatingly castrated cabal. Great. I feel so much better.

    Oh well. At least it’s still a pretty easy call. P67 + i5 2500K would be an easy pick for me.

    The level of compromise has become too great, AMD. Higher power draw; lower performance in both CPU and peripherals. And it goes on and on and on each year. Each chipset my hopes are dashed. When the time comes to upgrade (still a good ways off) I’ll gladly give my money to the 800 pound gorilla if AMD doesn’t alleviate these disparities.

      • insulin_junkie72
      • 9 years ago

      It’s the evil vs. the incompetent!

      /insert SPACEBALLS quote here

      • flip-mode
      • 9 years ago

      Well, I am surprised to find such a comment got thumbed into negative territory. Not going to worry about it. Oh, sweet, coffee’s just finished. Time for a cup…

      • Krogoth
      • 9 years ago

      How is AMD platform made from fail?

      The differences between the chipset platforms are trivial at best. You just need to stop looking at the bars. Look at the acutal numbers. AMD’s only potential downfall is CPU utilization, even so it is only at 8%. I am assuming Geoff meant as in utilization of one of the cores. It is a non-issue for majority of desktop users. In systems where the allocation of CPU resources matters, they will opt for pure hardware solutions (HBA controllers, NICs).

        • axeman
        • 9 years ago

        There is major KNOWN issues with pretty much every ATI chipset that require the operating system to work around. This isn’t big news, most if not all hardware has errata. It is news when the same issues stick around for revision after revision of the hardware. In some cases it’s not a matter of working around them, because the design is just plain flawed, so even with workarounds performance is compromised. For instance, every ATI/AMD USB controller features substandard (compared to any other chipset) USB performance, and the USB 2.0 spec is ancient by technology standards. It’s not confidence inspiring that they can’t even get that figure out yet – and it’s not even the performance, because apparently it takes an OS patch to keep your computer from crashing because of their USB controller.

        [url<]https://techreport.com/articles.x/5976/12[/url<] Even ATI's very first southbridge chip couldn't match others on USB throughput. [url<]https://techreport.com/articles.x/18539/9[/url<] Seven years later, they're still at it, but they did add some new functionality: [url<]http://support.microsoft.com/kb/982091[/url<] Windows 7 needs to workaround broken USB controller to keep from crashing.

      • shank15217
      • 9 years ago

      Well that was the southbridge, now lets see how the north bridge stacks up. The p67 is one hop closer to the cpu.

    • anotherengineer
    • 9 years ago

    So this was tested on the PCIe slot with the syba card/marvell chip on the AMD board and not through the 850SB?

    Scott/Geoff from this review [url<]https://techreport.com/articles.x/20190/7[/url<] you said you used the latest AMD AHCI drivers on the asus 890gx pro/usb3.0, do you remeber What VER BIOS the mobo was running? And did you run any tests with the defualt MS driver?

    • Next9
    • 9 years ago

    I doubt you have tested SB850 PCIe performance, since SB850 itself has only 2x 1 PCIe lanes. Almost all of the PCIe connectivity on AM3 platform comes from northbridige not the SB850 southbrdige.

    Since only Gigabyte publish the motherboard scheme in manual, I think you could not recognize which one of the PCIe 1x slots is connected to the southbridge, if there was any.

      • axeman
      • 9 years ago

      Hmm, maybe Geoff should re-run a couple of tests with the card in a different slot?

        • Next9
        • 9 years ago

        Running such test on Gigabyte motherboard (connection of slots is documented) or running such test on ASUS PCIe 4x or PCIe 16x slot would give a clear answer on AMD 800 PCIe performance question.

        Since SB850 itself has only 2 PCIe 1x slots, and manufacturers usually use them for storage or network controllers or do not use them anyway, I do not think it is meaningful to search some minor motherboard with SB850 PCIe 1x port connected to the slot and analyze its performance.

    • Meadows
    • 9 years ago

    Now there’s new information about chipset differences, and quite relevant to gaming I believe, too.
    Well done.

    • Sumache
    • 9 years ago

    How about the X58 chipset? I’d be interested to see how it stacks up.

      • Vaughn
      • 9 years ago

      The X58 should provide equal performance to the new chipset as it always had more bandwidth to begin with.

      Its one of the reasons on my list for going bloomfield over lynnfield!

        • Dissonance
        • 9 years ago

        Depends on where the PCIe lanes are coming from. The X58’s ICH10R south bridge has gen-one PCIe, while the north bridge is gen-two.

          • Vaughn
          • 9 years ago

          Ah thanks for the clarification I was under the impression it was Gen two from both north bridge and south bridge.

          • Bensam123
          • 9 years ago

          Sounds like a question looking for an answer!

      • NeelyCam
      • 9 years ago

      I wouldn’t mind seeing the results for H67 as well… to see if there’s any difference between P and H. My 2600k is in the mail, but I have trouble deciding between P and H – overclocking or integrated graphics…

      Such a curious decision to put the overclocking capabilities and better IGPs in the same chip when neither (current) chipset supports both..

      EDIT: I meant H67, obviously…

        • bimmerlovere39
        • 9 years ago

        Agreed. And, while we’re at it, why not take a look at the 890FX, too?

          • axeman
          • 9 years ago

          And test something on an NForce board, they apparently exist with PCIe 2.0 as well. 🙂

    • bcronce
    • 9 years ago

    They should’ve just used a PCIe solid-state drive and covered some of the PCIe fingers with nail polish.

    I know when Crossfire first came out, some place wanted to compare performance of a single video card running on only 8 lanes instead of 16 because they wanted to see what difference the reduced bandwidth would do.

    They just took nail polish and covered up some of the fingers. Because PCIe is packet switched, they could actually cover up the fingers in a way to give 7 lanes or even an async upload/download like 16x down and 8x up.

    PCIe really doesn’t care. Take a 4x SSD and cover up the fingers needed to make it 1x.

    Unlike PCI, PCIe is serial, so each upload/download lane is isolated from the others. Add packet switching and it’s only natural to think it doesn’t care.

      • flip-mode
      • 9 years ago

      likes this

      • UberGerbil
      • 9 years ago

      Yeah, PCIe spec requires the host and peripheral to negotiate the number of lanes and link speed as part of the initial handshake at power up (there may be sleep modes where it gets renegotiated at wake also, though most of that power-saving stuff got added after 1.0 which is the last spec I read completely). Which is why it’s possible to put cards with fewer lanes into slots with more, or even (with slots with open backs) cards with more lanes into slots offering less, not to mention the slots that change their lane width when other slots are in use or the always-popular “x16 mechanical but x8 electrical.”

      I suppose nail polish would work, as long as it’s not the popular (and potentially conductive) glitter variety — and given how fashion-forward Geoff is, that’s always a risk. He might not have been able to find any polish without the bling in his bathroom. After all, what looks better paired with fingerless biking gloves than nails painted to complement the metal-flake of the frame?

      Anyway, there’s actually paint-on insulator that can be used for this purpose; I’ve seen it at Fry’s.

      • Bensam123
      • 9 years ago

      I agree, or use tape, that is what I did with my perc 5/i in order to get around a bus issue with intel chipsets. Some scotch tape works wonders.

      I would have to wonder if this may cause conflicts though? In a perfect world it should work exactly as you say, but product makers don’t live in a perfect world and laziness as well as compatibly issues live there.

      • NeelyCam
      • 9 years ago

      This sounds reasonable, but does it really work? PCIe PHY is AC coupled, so unless the nail polish is thick enough to minimize the coupling capacitance, wouldn’t the link still work?

        • NeelyCam
        • 9 years ago

        I take the “-1” to mean I’m wrong. I’d still like to know how I’m wrong…

          • axeman
          • 9 years ago

          Well, what makes you think you’re right? Did you test it? How do you know nail polish has usable dielectric properties? Seriously, if you do know this, I need to know how.

            • NeelyCam
            • 9 years ago

            I don’t know if I’m right or not – that’s why I was asking.

    • juampa_valve_rde
    • 9 years ago

    I have heard of Syba and bought some products, they made/import a lot of small generic things related to computers.

      • The Wanderer
      • 9 years ago

      And some not so small. They made my previous monitor, a very solid 17″ CRT, which also happens to be the only 17″ monitor I’ve ever personally seen prove itself capable of handling 1600×1200.

      I ran on that monitor for several years, and only replaced it with a 3008WFP. It’s still sitting in my closet; I’ll probably use it for a server or something one day.

    • indeego
    • 9 years ago

    [quote<]Unfortunately, TRIM commands can't yet be passed through a RAID controller.[/quote<] I thought Intel fixed this with latest storage drivers?

      • Dissonance
      • 9 years ago

      IIRC, the Intel drivers only allow TRIM commands to be passed to SSDs acting as single drives while RAID arrays are also hanging off the controller. TRIM can’t be passed to SSD members of an array.

        • axeman
        • 9 years ago

        Intel’s fake RAID is for wieners anyhow.

          • NeelyCam
          • 9 years ago

          Are you one of those who believe RAID0 is not RAID?

            • indeego
            • 9 years ago

            No, I believe he’s one of the probably tens of thousands that have seen onboard RAID fail hard.

            And yeah RAID0 is AID.

      • Vaughn
      • 9 years ago

      Noooooo!

      Why do people keeping saying this everytime a new RST driver is released!

        • indeego
        • 9 years ago

        Because it’s in the release notes from Intel.
        [url<]http://www.intel.com/support/chipsets/imsm/sb/CS-031491.htm[/url<] Looks like they corrected it later.

    • bdwilcox
    • 9 years ago

    What do these numbers mean in real world scenarios? I’m sure those differences are there as your testing shows, and it’s interesting to see, but would most users ever notice them? For example, how much time would it take each to transfer an 8GB volume. That is much more relatable to me (and I would guess most users) than simple numbers on a chart.

    Another interesting venue to explore is how stripe size affects SSD RAID arrays and if that differs depending on the SSD’s controller chips (Intel, Indilinx, SandForce, etc.)

    • sabrewulf165
    • 9 years ago

    I’m not exactly sure how you did your HDTune testing, but I can easily get 230-250 MB/s read across the entire drive with my single Intel X25-M and budget P55 board, so your “HD Tune – Maximum Read Speed” of barely over 100 MB/s for P55 with a pair of RAIDed drives definitely makes me ask some questions.

      • cygnus1
      • 9 years ago

      that’s raided on a card that’s plugged into a half speed 1x pciE slot, not using the built in sata ports. the point was to narrow down the performance of 1 lane on each chipset

      • Dissonance
      • 9 years ago

      If you’re using a budget P55 board, you’re probably attaching that X25-M directly to the chipset’s SATA controller. We’ve achieved similar results with such a configuration with our storage test systems: [url<]https://techreport.com/articles.x/20087/3.[/url<] We're testing the P55's PCIe implementation here, not the speed of its SATA controller. If you'd like to see how the performance of the various chipset SATA controllers compare, start here: [url<]https://techreport.com/articles.x/20190/7.[/url<]

        • sabrewulf165
        • 9 years ago

        Ah, thanks for the clarification guys. Guess I should’ve read the test setup a little more closely *embarrassed*

    • cygnus1
    • 9 years ago

    [quote<] when we ran the same tests on a P55 board that employs a PCIe switch chip, we saw even higher transfer rates than on the P67 Express. [/quote<] since as you said, many p55 motherboards employ a pci express why not show an example in the benchmarks? it seems like you at least partially tested it. would you speculate the switch allows better performance because it's bonding all the pci express lanes off the p55 southbridge?

Pin It on Pinterest

Share This