How do SAS enclosures and cables work?

All things storage here: hard drives, DVD RW drives, little wicker baskets.

Moderators: morphine, Steel

How do SAS enclosures and cables work?

Postposted on Thu Mar 14, 2013 4:57 pm

Ok, actually, my questioning is a bit more specific. Let's take an example of the HP DL320s: http://h18004.www1.hp.com/products/quic ... 71_na.html

It supports 14 SAS/SATA drives. Now the integrated controller is a P400: http://h18000.www1.hp.com/products/quic ... 00_na.html

Now here's where I get a bit lost. The P400 has "Eight (8) SAS physical links distributed across 2 internal x4 wide port connectors." My understanding of SAS is that that would be 8 SAS/SATA drives. However, "The SA-P400 Controller supports up to 18 drives depending on the server implementation."

...what? And even more: "Up to 38TB of total storage with 38 x 1TB 3.5" SATA MDL HDD"

So it can support 8 drives? Or 18? Or 38? But all with 8 physical links/channels for SAS?

Or how about the P800: http://h18004.www1.hp.com/products/serv ... index.html
http://h18000.www1.hp.com/products/quic ... 16_na.HTML

You can put a P800 in that DL320s to handle the internal drives and hook up to an external SAS enclosure. 16 physical links via 2 x 4x internal and 2 x 4x external SAS connectors. Now this sucker can link to an external SAS chassis like thus: "Up to 96 TB of external storage per PCI slot with 8 HP 60 Modular Smart Array enclosures and 96 x 1TB 3.5" SATA MDL hard drives"

The MSA 60 enclosure is this: http://h18004.www1.hp.com/products/quic ... 7_div.HTML

I get that you can daisy chain a bunch of them. But I still have issues with the physical SAS links.

Do the SAS backplanes built into these servers and enclosures multiplex all the drives and just flood the SAS interface going to the controller with the max bandwidth it'll take? Like an ethernet switch, conceptually? 7 ports on your ethernet switch are all dumping max data and it's getting parceled out into the one remaining output port, so to speak?

I see some DIY storage solutions where you need, say, 20 drives so you have 1 controller with 5 SAS 4-channel ports or multiple SAS controllers to handle all the drives. Yet HP and other vendors like Dell have controllers that seem unable to handle the physical load of all the drives they claim to handle. What do they do different?
Scrotos
Graphmaster Gerbil
 
Posts: 1028
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: How do SAS enclosures and cables work?

Postposted on Thu Mar 14, 2013 6:10 pm

I think you answered most of your questions.

HP and Dell probably use have their own SAS daisy chain cabling to do what they described in their documentation.
Ivy Bridge i5-3570K@4.0Ghz, Gigabyte Z77X-UD3H, 2x4GiB of PC-12800, EVGA 660Ti, Corsair CX-600 and Fractal Refined R4 (W). Kentsfield Q6600@3Ghz, HD 4850 2x2GiB PC2-6400, Gigabyte EP45-DS4P, OCZ Modstream 700W, and PC-7B.
Krogoth
Maximum Gerbil
Silver subscriber
 
 
Posts: 4395
Joined: Tue Apr 15, 2003 3:20 pm
Location: somewhere on Core Prime

Re: How do SAS enclosures and cables work?

Postposted on Thu Mar 14, 2013 8:38 pm

Good coverage here... seems you just need an expander to attach more but the bandwidth looks to get shared
http://en.wikipedia.org/wiki/Serial_attached_SCSI
Arvald
Gerbil XP
Silver subscriber
 
 
Posts: 333
Joined: Tue Sep 27, 2011 12:14 pm
Location: Gerbil-land, Canada

Re: How do SAS enclosures and cables work?

Postposted on Fri Mar 15, 2013 11:23 am

Ya, so here I am bouncing around between vendor sites and I never bother to wiki it up. Duh. Thanks for the link.

Interesting. So I guessed the gist of it. I always had in my mind that a cascaded set of SAS drives would have like trillions of bytes of bandwidth and all that jazz. But no, it's just for capacity in most cases because you're limited by X SAS links unless you have some enclosure that lets you put multiple SAS links into it.

Whew. Makes sense now. Thanks again!
Scrotos
Graphmaster Gerbil
 
Posts: 1028
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: How do SAS enclosures and cables work?

Postposted on Fri Mar 15, 2013 1:40 pm

Yes, the bandwidth is measured on the number of "ports" you have and can put into an expander.

With SAS, it is 3Gb/s/port, and SAS2, 6gb/s/port. Sometimes you will hear this referred too as "lanes" as well. Each port can connect to a drive, or it can connect to another expander.

So, if you have a P400, which sports, 8 SAS ports (2 connectors, 4 ports per connector), you will have 8 ports x (3Gbs x 2 (send/receive)), or 48Gb/s of bandwidth that the card can use.

However, there is a catch... SAS is full duplex, SATA is not.

If you use SATA drives, you only have 24Gb/s of bandwidth (and technically less due to SATA over SAS encoding loss).

Interestingly, I believe your P400 card, is only 4x and PCIe 1.0, so, it's maximum bandwidth with SAS drives is 20Gb/s. You would not get full performance out of it, if you put an expander in. Now, 20Gb/s is about 1GB/s which is not too shabby, but do not expect 20 SSD's to deliver amazing performance, when 8 medium end SATA drives will perform the same with that controller (the exception being random seek latency with the SSD's).

I know this from real life of using a RAID 0 of 4 SSD's and a P400. It's not all that fast.

HTH!
TechNut
Gerbil
 
Posts: 15
Joined: Tue Apr 03, 2007 8:58 pm

Re: How do SAS enclosures and cables work?

Postposted on Fri Mar 15, 2013 2:14 pm

Scrotos wrote:Ya, so here I am bouncing around between vendor sites and I never bother to wiki it up. Duh. Thanks for the link.

Interesting. So I guessed the gist of it. I always had in my mind that a cascaded set of SAS drives would have like trillions of bytes of bandwidth and all that jazz. But no, it's just for capacity in most cases because you're limited by X SAS links unless you have some enclosure that lets you put multiple SAS links into it.

Whew. Makes sense now. Thanks again!

You are welcome.
I've taken to checking Wikipedia as one of my first sources for background technical information like this... if it is there it is in nearly textbook form so it should be accurate
Arvald
Gerbil XP
Silver subscriber
 
 
Posts: 333
Joined: Tue Sep 27, 2011 12:14 pm
Location: Gerbil-land, Canada

Re: How do SAS enclosures and cables work?

Postposted on Fri Mar 15, 2013 2:20 pm

You pretty much figured it out for yourself, and the network analogy works well.

SAS expander = switch, typically 24-36 ports (there are "true" SAS switches out there but keeping it simple)

SAS adapter = NIC, usually 4-8 ports.
Anything bigger usually means a built-in expander. You can chain expanders if done properly, this is where some of the monster enclosures come in.
Expanders won't have trouble keeping up with moderate ratios of spinning rust per "true" system ports, ~4:1 should be easy, after that depends.
SSDs on the contrary will notice quickly.

SAS = full duplex, native. Some drives are dual port as well, multipath capable.
SATA = half duplex, tunneling overhead.

SAS cable = usually 4 ports per.
Most enterprise stuff uses backplanes to split for drives instead of fanout cables. Fanout cables have a caveat people often mess up: they are directionally specific.
blah blah blah signature blah blah blah
Bauxite
Gerbil Elite
 
Posts: 609
Joined: Sat Jan 28, 2006 12:10 pm
Location: electrolytic redox smelting plant

Re: How do SAS enclosures and cables work?

Postposted on Fri Mar 15, 2013 2:56 pm

Aye, it's actually unrelated to my SSD in a HP project. Thanks fer the help everyone! I've got one more topic that's throwing me for a loop lately and I'm sure it'll cause some arguments. I'll pop that in a different thread.
Scrotos
Graphmaster Gerbil
 
Posts: 1028
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.


Return to Storage

Who is online

Users browsing this forum: No registered users and 1 guest