TR Forums

Topic text says it all, but I'll expound further anyway:

All eight of the SATA ports on the motherboard are occupied, but I'd like to add more storage. It will be software RAID1 in linux, so I'm just just looking for something PCIe that is reliable with good drivers that provides at least 4 ports. There's a lot of cheap stuff out there, which some of my perusal of NAS forums suggests would indeed be suitable, but I am availing myself of the experience here.

Any ideas?

Have a browse at these:
http://www.newegg.com/Syba-Controllers- ... D-8344-410
I've used this brand in the past.

I used to run these: http://www.newegg.com/Product/Product.a ... 6816124064 in my FreeNAS box, they played nicely, they just weren't terribly fast with 4 drives loaded up on them.

Conventional wisdom used to be that you should avoid Marvell chipsets on Linux, since they were notoriously difficult for the Open Source community to work with due to IP/NDA issues, resulting in poor quality drivers. Not sure if this has gotten better in the past few years.

I've also seen several cases where controllers based on older chipsets/designs don't play nice with UEFI motherboards (hang during POST). Is the system this will be going into UEFI?

JBI wrote:

Conventional wisdom used to be that you should avoid Marvell chipsets on Linux, since they were notoriously difficult for the Open Source community to work with due to IP/NDA issues, resulting in poor quality drivers. Not sure if this has gotten better in the past few years.

Yeah, that was kind of my recollection as well, but it's hard to discern when reading current forums if the advice of the members is reflecting out-of-date or misunderstood information. At any rate there is definitely still an aversion to marvell, so I really appreciate Waco's input. That's exactly the kind of response I was looking for, thank you.

I guess I don't know how to put it politely, but in the NAS forums there seem to be several very influential posters who have managed to turn certain ideas into unquestionable dogma. Sometime they're just misguided, and sometimes they are just wrong. But it makes certain questions a bit harder to get a good feel for, even if they aren't related to the known dogmas I've encountered. The prevalence of the attitude means you have to be critical about most of what you hear unfortunately.

JBI wrote:

I've also seen several cases where controllers based on older chipsets/designs don't play nice with UEFI motherboards (hang during POST). Is the system this will be going into UEFI?

That's good to know, and yes it is. Though, it has the ability to boot either way, and I actually used legacy I think. I'll keep that in mind.

just brew it! wrote:

Conventional wisdom used to be that you should avoid Marvell chipsets on Linux, since they were notoriously difficult for the Open Source community to work with due to IP/NDA issues, resulting in poor quality drivers. Not sure if this has gotten better in the past few years.

I've also seen several cases where controllers based on older chipsets/designs don't play nice with UEFI motherboards (hang during POST). Is the system this will be going into UEFI?

I've got a couple of these in my frankenNAS

http://www.newegg.com/Product/Product.a ... 6816124027

Silicon Image controllers are very well supported in linux, windows, os x, bsd, etc. Just make sure to flash it to the non-raid bios and all is good. Even though it is Sata II, that is still more than enough for mechanical drives.

Marvell did used to suck, but the modern controllers work out of the box on damn near everything.

If you know the flavor you'll be running, I could confirm if they're bootable without a hassle.

If you're referring to CyberJock on the FreeNAS forums, I feel your pain. He's an intolerable dick.

I've used these http://www.newegg.com/Product/Product.a ... 6816124060 and they work great under Windows and FreeBSD (FreeNAS), I'd assume that means they will work under Linux as well.

Waco wrote:

If you're referring

Wow! You're good!

Waco wrote:

If you know the flavor you'll be running, I could confirm if they're bootable without a hassle.

I wouldn't ever be booting from it, but it'll be Ubuntu 16.04.

Thanks again for the help, as well you immediately homing in on what I was talking about. I guess it's not just me!

---

Thanks to everyone else as well, if these have worked well for you they'll likely work just as fine for me. Appreciate it.

Waco wrote:

I used to run these: ... in my FreeNAS box, they played nicely, they just weren't terribly fast with 4 drives loaded up on them.

I've been using the same controller with Windows 2012 for the last 6 months. No issues.

Glorious wrote:

Waco wrote:

If you're referring

Wow! You're good!

Yeah, I really hate that guy. He's a detriment to the entire community. I honestly tried to have a few real conversations with him about very specific things...and he just went of the deep end about how I "clearly never work with storage" and how I "will learn if I ever do enterprise storage". Ha, it's my day job.

Glorious wrote:

Waco wrote:

If you know the flavor you'll be running, I could confirm if they're bootable without a hassle.

I wouldn't ever be booting from it, but it'll be Ubuntu 16.04.

Thanks again for the help, as well you immediately homing in on what I was talking about. I guess it's not just me!

I'll toss one of mine into my PC and boot a 16.04 LiveCD...if it shows up, I'll let you know. I expect it'll be flawless since Marvell drivers, in the past 2-3 years, have gotten very mainstream.

Ha, it booted no problems. Posting from Ubuntu 16.04, booting from the Marvell controller I linked above.

I haven't touched Ubuntu since...version 6? It was ~2004/2005 or so, with the exception of a quick check last year when I realized I hated their new desktop compositor.

This feels very...normal. I'm used to RHEL/CentOS/TOSS. I like.

Anyway - no troubles out of the the box with the currently nightly build of 16.04.

I've had a SYBA SI-PEX40064 since Feb, 2014 for $24.99 delivered (at the time, $31 now). No issues on Win7 or Linux with mechanical HDs, SSDs and a DVD burner. No driver was ever needed. I've never tried to boot from it. Use it in at least a PCIe 2.0 slot to get full performance.

Waco wrote:

I'll toss one of mine into my PC and boot a 16.04 LiveCD...if it shows up, I'll let you know. I expect it'll be flawless since Marvell drivers, in the past 2-3 years, have gotten very mainstream.

That's very good to know. I had still been avoiding Marvell in general, due to a couple of so-so experiences in the past. I guess I can stop worrying now.

A lot of newer cards (and motherboards, when they have an extra SATA controller) seem to be using Asmedia now. Any thoughts? Asmedia's USB 3.0 controllers seem to have a pretty bad rep, but I haven't heard much about their SATA stuff. I've tested a couple, and the performance seems decent; but I haven't used them enough yet to have a good feel for overall stability.

Waco wrote:

I haven't touched Ubuntu since...version 6? It was ~2004/2005 or so, with the exception of a quick check last year when I realized I hated their new desktop compositor.

Yeah, I really tried to like the Unity desktop, and it does seem to have a few fans. But after several weeks of fighting with it to try and get it to behave the way I wanted, I gave up and installed the alternate KDE desktop. Haven't looked back. Kubuntu is still my "daily driver" OS.

just brew it! wrote:

Waco wrote:

I'll toss one of mine into my PC and boot a 16.04 LiveCD...if it shows up, I'll let you know. I expect it'll be flawless since Marvell drivers, in the past 2-3 years, have gotten very mainstream.

That's very good to know. I had still been avoiding Marvell in general, due to a couple of so-so experiences in the past. I guess I can stop worrying now.

Marvell controllers have a mainline driver in the Linux kernel now. I'm not sure how long it's been that way. FreeBSD support is non-existent and people still say they are a crappy company to deal with, but they have some Linux support.

I'm specifically thinking of this board, by the way: http://www.newegg.com/Product/Product.a ... -_-Product

It would be interesting to see how these cheap cards compare. There are enough of us that build stuff out commodity parts for home use to make it worth while.

MarkG509 wrote:

I've had a SYBA SI-PEX40064 since Feb, 2014 for $24.99 delivered (at the time, $31 now). No issues on Win7 or Linux with mechanical HDs, SSDs and a DVD burner. No driver was ever needed. I've never tried to boot from it. Use it in at least a PCIe 2.0 slot to get full performance.

What kind of performance are you getting?

I'm not always a fan of the Startech stuff, but I found a few of the vendors who used SATA (or eSATA) devices swore by Startech SATA controllers and confirmed their stuff worked with them. One was IOSafe who was saying for Windows Server that Startech's controllers (a few specific ones) were the only cards they could guarantee would work properly. I bought a few of them and have had great luck with zero drive disconnects like I did with other on-board eSATA controllers. FWIW.

Flatland_Spider wrote:

FreeBSD support is non-existent and people still say they are a crappy company to deal with, but they have some Linux support.

Marvell stuff works fine in FreeNAS, which is PC-BSD I guess, but close enough that I'd expect FreeBSD to be straightforward.

Waco wrote:

Flatland_Spider wrote:

FreeBSD support is non-existent and people still say they are a crappy company to deal with, but they have some Linux support.

Marvell stuff works fine in FreeNAS, which is PC-BSD I guess, but close enough that I'd expect FreeBSD to be straightforward.

FreeNAS is based on FreeBSD.

SuperSpy wrote:

Waco wrote:

Flatland_Spider wrote:

FreeBSD support is non-existent and people still say they are a crappy company to deal with, but they have some Linux support.

Marvell stuff works fine in FreeNAS, which is PC-BSD I guess, but close enough that I'd expect FreeBSD to be straightforward.

FreeNAS is based on FreeBSD.

FreeNAS is PC-BSD, which is a derivative of FreeBSD. No?

Yes, last time I looked at PC-BSD.

PC-BSD or PCBSD, is a trademarked Unix-like, desktop-oriented operating system built upon the most recent releases of FreeBSD.

FreeNAS is a free and open-source software network-attached storage (NAS) system based on FreeBSD and the OpenZFS file system.

(^^ mind the 2.5 years gap)

Are Marvell's 88SE92xx better than the older 91xx in performance or other properties?

What about versus ASMedia's ASM1601? Based on old discussions the ASM1601 was faster than Marvell's 91xx, and maybe with better Windows drivers (TRIM). Though only 2 channels.

What about 9215 vs. 9230? A specs summary suggests 9230 just adds a 2nd PCIe lane, AES, and RAID, but I wonder if there's more to it.

I assume Silicon Image aren't a good option for PCIe. Are there any other companies to consider?

Whoa necro!

I bought something with a Marvell 88SE9215 chip.

It sucked. I threw it out. (this was on linux ~kernel 4.4)

viewtopic.php?f=7&t=118488&p=1322157#p1322157

So I can't recommend that.


I'd go with Silicon Image out of your list of options, but at this point I'd only get like a LSI enterprise/professional thing off ebay or something.

If you're willing to spend a bit more, it's hard to go wrong with one of these: https://www.amazon.com/gp/product/B002RL8I7M/

The prices fluctuate a lot since it's all "new old stock" from third-party sellers; I've seen them dip as low as $65. It's based on older tech, but it supports 6 Gbps SATA which is probably all you need. (Newer versions of the LSI cards support 12 Gbps SAS, but SAS drives are pretty spendy and overkill for a home server.)

A few FYIs if you think you might go this route:

1. You need a spare PCIe x8/x16 slot.

2. Make sure you flash it to the latest firmware. You actually have two choices -- RAID firmware (but you can just opt to not use the RAID functionality), and non-RAID "IT mode" firmware (which effectively turns it into a "dumb" 8-port controller). Also note that the product support pages for the older LSI controllers are a mess since they got sold (twice, once to Avago, then to Broadcom); even though this controller is a 9211-8i, you need to search for the 9210-8i firmware (logically the same board, just a different connector layout). And although they do provide a firmware flash tool for Linux, you need to download the Windows package too and extract the firmware image from that to use with the Linux flash tool. Duh.

3. Since they're actually SAS controllers (but backward compatible with SATA drives), they don't have standard SATA connectors. Instead of a set of 8 normal SATA cables, you'll need a couple of SAS-to-SATA 4-way breakout cables, e.g.: https://www.amazon.com/Cable-Matters-In ... 012BPLYJC/

++ for truth from brew!

For years I'd drooled over the LSI (nay! 3ware of old) SATA/SAS controllers, but never could afford until now'ish.

I have a 9650 and love it. Got the 12 drive version, but found it 1/2 price on amazon after the newegg "window" for price matching. (insert bitter taste)

The RAID, JBOD and speed are pretty darn good for a SATA2 card with SATA 3 drives. 120+ MB/s depending on destination to/from.

Heck, got Corsair 750D after major price drop with 9 drives, 3 x 8TB RAID 5s and 3 spares. [word to wise: this thing is a beast/heavy AF]

Newer/faster cards exist, but if you don't mind a step back at a cheaper price point...my $.02

just brew it! wrote:

If you're willing to spend a bit more, it's hard to go wrong with one of these: https://www.amazon.com/gp/product/B002RL8I7M/

The prices fluctuate a lot since it's all "new old stock" from third-party sellers; I've seen them dip as low as $65. It's based on older tech, but it supports 6 Gbps SATA which is probably all you need. (Newer versions of the LSI cards support 12 Gbps SAS, but SAS drives are pretty spendy and overkill for a home server.)

A few FYIs if you think you might go this route:

1. You need a spare PCIe x8/x16 slot.

2. Make sure you flash it to the latest firmware. You actually have two choices -- RAID firmware (but you can just opt to not use the RAID functionality), and non-RAID "IT mode" firmware (which effectively turns it into a "dumb" 8-port controller). Also note that the product support pages for the older LSI controllers are a mess since they got sold (twice, once to Avago, then to Broadcom); even though this controller is a 9211-8i, you need to search for the 9210-8i firmware (logically the same board, just a different connector layout). And although they do provide a firmware flash tool for Linux, you need to download the Windows package too and extract the firmware image from that to use with the Linux flash tool. Duh.

3. Since they're actually SAS controllers (but backward compatible with SATA drives), they don't have standard SATA connectors. Instead of a set of 8 normal SATA cables, you'll need a couple of SAS-to-SATA 4-way breakout cables, e.g.: https://www.amazon.com/Cable-Matters-In ... 012BPLYJC/

just brew it! wrote:

If you're willing to spend a bit more, it's hard to go wrong with one of these: https://www.amazon.com/gp/product/B002RL8I7M/

You've only gone and made me spend more money.

Got a link to any guides on flashing them?

cheesyking wrote:

just brew it! wrote:

If you're willing to spend a bit more, it's hard to go wrong with one of these: https://www.amazon.com/gp/product/B002RL8I7M/

You've only gone and made me spend more money.

Got a link to any guides on flashing them?

I was gonna find the support page for that family of cards on their site and link it for you, but their site seems to be messed up (well, more messed up than usual) right now. Search function just sits and spins forever.

There's a document that describes the flash procedure, I can send you a copy if necessary.

Edit: Here's the page: https://www.broadcom.com/products/stora ... #downloads (there's a PDF in the firmware installer package that describes how to use the firmware flash tool)

Since we're here, I've got a couple bookmarked links for LSI flashing. Probably a bit dated, and one is more cross-flashing Dells back to LSI.

LSI

Dell>LSI

Oh, and if it helps allay any concerns people might have about flashing these LSI cards, they are purportedly brick-proof. Failed or incorrect firmware flashes can supposedly be recovered without special tools. (I have never had to test this assertion myself, as they've always flashed successfully on the first try.)

Glorious wrote:

I bought something with a Marvell 88SE9215 chip.
It sucked. I threw it out. (this was on linux ~kernel 4.4)

Could it have been an issue with Linux, or card-specific (heatsinking or whatever)?
Most people seems to like this 9215 card.

I'd go with Silicon Image out of your list of options

I think they quit the biz, maybe even before the buyout. Their cards are rare, more expensive, and I don't think they have a 4 port PCIe chip.

I'd only get like a LSI enterprise/professional thing off ebay or something.

I won't be RAIDing, so it's probably beefier than I need (power/heat, I'm guessing). And I don't see any PCIe 1x cards.

meerkt wrote:

Glorious wrote:

I'd only get like a LSI enterprise/professional thing off ebay or something.

I won't be RAIDing, so it's probably beefier than I need (power/heat, I'm guessing). And I don't see any PCIe 1x cards.

Yeah, given that the cards are designed for enterprise applications, PCIe x1 doesn't cut it (not enough bandwidth to saturate all of the disk ports with I/O).