Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Don't see a specific place for your hardware question? This is the forum for you!

Moderators: mac_h8r1, Nemesis

Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 1:31 pm

So I gots me an HP DL380 G5 with a P400i RAID controller and 8 slots to play with. I've ordered 8 x Samsung 840 Pro 256 GB SSDs. I'm going to overprovision them by a bunch, maybe get them to 200 GB or less usable space.

The overprovisioning seems to help with performance consistency: http://www.anandtech.com/show/6553/sand ... ew-256gb/4

I am aware that the P400i RAID controller tops out at SATA1/1.5Gbps and will be a limiting factor. I'm going to guess it's still going to be faster than mechanical 15k drives. The P400i is battery backed and it's 256 MB of cache. The system has redundant PSUs and will be on an UPS.

I have hope that this will be viable based on:

http://sysops.pblogs.gr/2012/04/ssds-on ... llers.html
http://h30499.www3.hp.com/t5/ProLiant-S ... -p/1128019
http://www.overclock.net/t/1049163/hp-p ... pport-ahci
http://serverfault.com/questions/433596 ... ssd-drives
http://serverfault.com/questions/431437 ... 456#431456
http://serverfault.com/questions/401026 ... 030#401030
http://h20000.www2.hp.com/bizsupport/Te ... =c02963968 (I know, different controller, but not entirely irrelevant)

The server was flashed to the latest firmware as of a few months ago so the controller should support NCQ. I'll be making sure the SSDs are at the latest firmware when I go to provision them.

I will be putting Win2K8 R2 Enterprise on it, maybe Standard depending on if my existing user CALs will work on Enterprise or not. What should I do to test these drives? What should I do to make sure their performance isn't degrading? Any suggestions? Any dire warnings?
Last edited by Scrotos on Thu Jan 24, 2013 2:50 pm, edited 1 time in total.
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 2:00 pm

I don't know about testing the hardware, but Windows Server 2008 CALs don't care about Standard/Enterprise. You don't even install them. You just have to own them.
absurdity
Gerbil Elite
 
Posts: 871
Joined: Sat Mar 02, 2002 7:00 pm
Location: VT

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 2:05 pm

What are you going to be doing with the server? What raid lvl?
w00tstock
Gerbil
 
Posts: 43
Joined: Thu Nov 08, 2007 2:25 pm

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 2:49 pm

Ok, groovy. My licensing guy never got back to me about it (server/enterprise). I know 2012 would need all new CALs so I'm sticking with 2008R2.

This will be a database server and I'll probably run RAID6 with a hot spare. 32 GB of RAM and dual quad-core Xeon 2.66 GHz CPUs. But... here's the deal, we're replacing a DOS-based program that ran on a WinXP box locally for one of our users with some whiz-bang vendor who wants a database server and an app server to do the same thing. I'll be putting on Win2K8 R2 Enterprise and SQL Server 2012 Standard. The vendor was all blah blah blah about how we have more than 5000 records so this would be I/O bound and we really need RAID 10 with 15k drives, etc.

Seriously. You're replacing a DOS program run on a Core 2 Duo and before then some dumpy P4 or P3. Our entire business (multiple databases, many more records, multiple processes hitting the db) runs on Win2K3 and SQL Server 2005 using some dumpy old homebrew server. I have 8 146GB 15k SAS drives on order in case this doesn't work out, but I figure wtf, let's try some of these 840 Pros I've been deploying to workstations as upgrades.

The vendor who is so concerned about I/O said it's fine to combine the dedicated app and db servers onto one server and if we wanted to, we could virtualize the servers. Mixed signals and I'm skeptical of their claims, but who knows.
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 4:03 pm

So I misunderstood overprovisioning. I thought some utility could change the SSD to report a lower capacity and BOOM I'm done. But no, the process is to just make a partition smaller than the full size of the drive. That leads me to the question of whether or not having something in a RAID will utilize it 100% even if the partition made from that RAID is smaller than what's fully available?
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 4:06 pm

Heh, some how I think going form about 200 iops of a 15k drive to 80,000 iops of a SSD is going to be ok. You could probably get away with raid 1 with room to spare. The only thing that I can see as a potential issue is lack of trim on windows.

I would probably set up a linux server with zfs and run the windows servers virtually because to the potential trim issue.
w00tstock
Gerbil
 
Posts: 43
Joined: Thu Nov 08, 2007 2:25 pm

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 4:09 pm

Scrotos wrote:So I misunderstood overprovisioning. I thought some utility could change the SSD to report a lower capacity and BOOM I'm done. But no, the process is to just make a partition smaller than the full size of the drive. That leads me to the question of whether or not having something in a RAID will utilize it 100% even if the partition made from that RAID is smaller than what's fully available?


You can still do it by individually issuing raw sata commands to the disks. Then add them to the array.
w00tstock
Gerbil
 
Posts: 43
Joined: Thu Nov 08, 2007 2:25 pm

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 4:42 pm

I had seen something about that though it seems to be a *nix thing and I'm more a Windows kind of guy. I might look into that. Beyond that, since the P400i doesn't seem to let me make smaller arrays than the reported disk sizes, I wonder if the overprovisioning will kick in on the SSD controllers if I just make a smaller partition. Right now that seems to be the big question that I don't know the answer to. I gotta buy some more drive caddies as I only have 4 but the 4 seem to be ok with the controller. Mmmm SATA is hot-swappable, ain't it?
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 6:06 pm

It sounds like you aren't really needing a high performance database for this. An 8 SSD RAID array will be fine for most needs, but if you need really fast queries, you may want to investigate 4 SSDs in 4 RAID 1 arrays instead (1 + hot spare). For example, let's say you have a query that pulls millions of rows for two tables and two indexes. By carefully arranging the data storage - each table / index on a seperate RAID 1 array - you could really maximize the IOPS and the real benefits of ultrafast seek times of each drive. By effectively merging all drives into one big drive, you lose many benefits the SSDs have in order to simplify your storage.
lilteap0t
Gerbil In Training
 
Posts: 3
Joined: Wed Jan 26, 2011 10:38 am

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 6:15 pm

The Samsung software coming with the SSD runs on Windows and you can configure overprovisioning there?
Image
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
Flying Fox
Gerbil God
 
Posts: 24422
Joined: Mon May 24, 2004 2:19 am

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 6:29 pm

Flying Fox wrote:The Samsung software coming with the SSD runs on Windows and you can configure overprovisioning there?


My first thought. However, upon trying to use it, it's not useful. Here's the steps it takes from a completely blank out-of-the-box SSD:

1. Load Samsung's software, go to Overprovisioning, click through the warning, you get a screen to select an SSD
2. Select an SSD and click Analyze
3. There are no partitions on the drive. Stop. You cannot progress.
4. Go to Disk Management and make a partition.
5. Quit out of Samsung's software and go back into it since you can't "Analyze" again on the same drive.
6. Analyze the dang thing.
7. It will list a partition. Analyze the partition.
8. It will show you your partition as a bar. It will give a recommended overpartitioning size; for 256 GB it was 23.2 GB.
9. The only number you can enter in is the size of overpartitioning you want to have done.
10. Click "resize" or whatever the button is.
11. Watch as the ONLY thing this does is shrink your partition size. You can delete the partition and recreate it at full size.
12. Useless!

Now maybe I missed something, but all that utility does is resize the partition and fail if your partition is too full. That's why I said I seem to have misunderstood how overprovisioning works and read up. Basically you just have unallocated space and the controller futzes with it from there.
Last edited by Scrotos on Thu Jan 24, 2013 6:38 pm, edited 1 time in total.
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 6:34 pm

lilteap0t wrote:It sounds like you aren't really needing a high performance database for this. An 8 SSD RAID array will be fine for most needs, but if you need really fast queries, you may want to investigate 4 SSDs in 4 RAID 1 arrays instead (1 + hot spare). For example, let's say you have a query that pulls millions of rows for two tables and two indexes. By carefully arranging the data storage - each table / index on a seperate RAID 1 array - you could really maximize the IOPS and the real benefits of ultrafast seek times of each drive. By effectively merging all drives into one big drive, you lose many benefits the SSDs have in order to simplify your storage.


Yeah, but their requirements are something like 160 GB for "logging" and 90 GB for "data" and that's coming up on the 256 GB (or less depending on my overprovisioning adventure) SSDs that I'm working with. I guess I could split them up on different drives, i.e. one for OS, one for "data", one for "logging", but I don't know that we'll need that much performance out of this. I could be wrong, though, and your scheme is a consideration, so thank ya.
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Thu Jan 24, 2013 7:11 pm

One note about the P400i - it should require 512MB cache to enable RAID-6. HP is a little strange with its support of RAID-6. Usually they require a cache upgrade to enable RAID-6 on the older controllers or the purchase of the Array Advanced pack for the newer ones.

Also, your vendor is nuts about the storage performance needed. I would ask for benchmarking numbers backing up the claim (and with that they should be able to give you the parameters the scenario ran under). As already been stated, though, even a single SSD would blow 8 x 15k SAS (or even FC) drives out of the water.

EDIT: Also, it appears the P400i only supports 6 drives (see this http://h18000.www1.hp.com/products/quic ... 0_div.HTML)

This link also shows that a P400i in RAID-6 cannot have a hot spare assigned: http://itcoop.wordpress.com/2011/02/02/ ... y-trouble/
Matt
mattshwink
Gerbil
 
Posts: 85
Joined: Wed Jul 16, 2008 7:54 am
Location: Alexandria, VA

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Fri Jan 25, 2013 1:10 am

If you need this for logging, then RAID it all the way! That's about the very definition of 'No performance needed'.
lilteap0t
Gerbil In Training
 
Posts: 3
Joined: Wed Jan 26, 2011 10:38 am

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Fri Jan 25, 2013 4:01 am

@Scrotos -

Yeah, it sounds like the "whiz-bang vendor" is blowing smoke (or smoking something)! He's either clueless, or just trying to justify his existence.

Your statement that you're "running the business" on "some dumpy old homebrew server" concerns me. Perhaps this is an area that ought to be addressed first, instead of getting this one user off of his old DOS-based app?

I think you're fine on the over-provisioning issue. The basic idea is to simply not use (refrain from writing to) some percentage of the drive, thereby giving the wear leveling algorithms a little breathing room. Whether this is done at the raw partition level (underneath the RAID implementation) or by partitioning the RAID volume shouldn't make a difference, especially given that you're not going to have TRIM support either way.

@w00tstock -

Sounds like it's a Windows shop. Expecting them to set up a Linux server (and run a filesystem that is not natively supported by any of the major distros on top of that) is a pretty tall order. I'm just guessing... not gonna happen! :lol:

I'm curious regarding how you think virtualization would help with the TRIM issue. Seems to me that passing TRIM commands through the virtualization layer would be at least as problematic as getting them through that ancient RAID controller. That being the case, the fact that ZFS supports TRIM doesn't help you, since the host OS still isn't going to know when blocks of the virtualized guest NTFS filesystem are freed.

Also... you seem to be implying that there are standard SATA commands to tell a drive to over-provision itself. While this would be a cool thing to have, I was not aware it existed; additional info, please?

@lilteap0t -

It's a freakin' legacy DOS app. That right there screams "no performance needed" even before you start getting into any other factors! :D
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37705
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Fri Jan 25, 2013 4:10 am

Upon further reflection, I take back what I said about the over-provisioning. The RAID controller is going to issue writes across the entire array when the array is initially built, so over-provisioning on top of the RAID probably doesn't work. You need to tell the RAID controller not to use the entire capacity of the drives up front (assuming it lets you do that). This should accomplish the same thing as over-provisioning at the raw partition level.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37705
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Fri Jan 25, 2013 4:12 am

mattshwink wrote:One note about the P400i - it should require 512MB cache to enable RAID-6. HP is a little strange with its support of RAID-6. Usually they require a cache upgrade to enable RAID-6 on the older controllers or the purchase of the Array Advanced pack for the newer ones.

EDIT: Also, it appears the P400i only supports 6 drives (see this http://h18000.www1.hp.com/products/quic ... 0_div.HTML)

This link also shows that a P400i in RAID-6 cannot have a hot spare assigned: http://itcoop.wordpress.com/2011/02/02/ ... y-trouble/
Matt


Ya, I know. I looked into that too. However, on the three DL380 G5's with P400i and 256 MB of BBU cache, I can not only select RAID 6 but I can also have a hot spare. And 8 drives. I mean, the enclosure has 8 slots and the backplane is 2 channels of 4 ports, I don't see why it wouldn't support all 8 drives. Maybe that came in with a firmware update. I don't see in the specs where it says only 6 drives, though.

Contrast that with a DL380 G6 or G7 (I forget offhand, not at work to look it up as it's offsite anyway) I got. To enable RAID 6 I had to get a cache upgrade, battery, and buy a special license to activate it on the controller. Seriously, enter in a special key in the firmware and BOOM you get RAID 6. I am pretty sure that was a P410i controller in that machine and I upgraded the chassis to handle 16 drives total. Ah yes, that's the Array Advanced pack you mentioned.

I could be wrong and there aren't P400i controllers in those servers, but I'm pretty sure there are. I can't imagine anyone upgraded them before I got the machines at discount.

I'm hoping they fixed whatever Ian ran into with the RAID 6 hot spare issue. ;) I have the luxury of running the firmware update from an HP universal DVD and that got me pretty current on those systems. I don't anticipate upgrading the firmware when I got drive issues.

mattshwink wrote:Also, your vendor is nuts about the storage performance needed. I would ask for benchmarking numbers backing up the claim (and with that they should be able to give you the parameters the scenario ran under). As already been stated, though, even a single SSD would blow 8 x 15k SAS (or even FC) drives out of the water.


Yeah, I was rolling my eyes so hard through email I think I strained something. We'll see how the SSDs turn out. My CDW rep kept trying to sell me on a PCIe solution like the Intel 910 (or whatever it is) because the consumer-level SSDs are only suitable for notebooks. I grant that the MLC is hand-picked and the controllers are tweaked and they are factory-overprovisioned to a large degree, but I have a hard time dropping $3200 on a PCIe solution when I don't need max speed anyway.
Last edited by Scrotos on Fri Jan 25, 2013 4:22 am, edited 1 time in total.
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Fri Jan 25, 2013 4:19 am

just brew it! wrote:@Scrotos -
Your statement that you're "running the business" on "some dumpy old homebrew server" concerns me. Perhaps this is an area that ought to be addressed first, instead of getting this one user off of his old DOS-based app?


Management dictates what's NOW NOW NOW and migrating this stupid thing is NOW NOW NOW. It got dumped in my lap on Tuesday to be implemented by Monday, more or less.

However, I did get budget approved for migrating our SQL Server to new hardware and software. $15k of software purchases later (OS, SQL, SQL CALs) and now I just need to get some hard drives. It'll be basically the same server except no SSDs. The dumpy server is on a mirror at least and does get backed up fully each morning and transaction logs every 15 minutes, compressed and moved offserver to two different locations, one offsite. It'll take longer to get up and going as our in-house programmer will have to update some of his .NET apps, methinks. We'd be moving from a SQL authentication scheme to one using active directory instead. At least, I think that's the plan. Makes sense to me, anyway. Now that we're off Netware we can actually do stuff like this...

just brew it! wrote:Also... you seem to be implying that there are standard SATA commands to tell a drive to over-provision itself. While this would be a cool thing to have, I was not aware it existed; additional info, please?


I've seen that mentioned in some of my research but it always seemed to be some *nix hard drive utility/command. I'd be interested in more info, too, if at least to satisfy my curiousity.
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Fri Jan 25, 2013 4:24 am

just brew it! wrote:Upon further reflection, I take back what I said about the over-provisioning. The RAID controller is going to issue writes across the entire array when the array is initially built, so over-provisioning on top of the RAID probably doesn't work. You need to tell the RAID controller not to use the entire capacity of the drives up front (assuming it lets you do that). This should accomplish the same thing as over-provisioning at the raw partition level.


Yeah, but does it really? I thought most RAIDs just marked the MBR (or whatever) and then in a 5 or 6 just wrote parity as it striped? As long as the container/array/whatever isn't filled, why would the controller be writing to the entire drives?
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Fri Jan 25, 2013 4:42 am

Scrotos wrote:
just brew it! wrote:Upon further reflection, I take back what I said about the over-provisioning. The RAID controller is going to issue writes across the entire array when the array is initially built, so over-provisioning on top of the RAID probably doesn't work. You need to tell the RAID controller not to use the entire capacity of the drives up front (assuming it lets you do that). This should accomplish the same thing as over-provisioning at the raw partition level.

Yeah, but does it really? I thought most RAIDs just marked the MBR (or whatever) and then in a 5 or 6 just wrote parity as it striped? As long as the container/array/whatever isn't filled, why would the controller be writing to the entire drives?

Because the OS can issue reads to areas that it hasn't written to yet. Windows does this, if you format a partition with the "quick format" option unchecked.

If the RAID controller hasn't initialized the parity up front any reads to unwritten areas will result in RAID parity errors. I'm pretty sure most RAID implementations pre-initialize the parity for this reason. (I suppose some RAID controllers may have clever ways to work around this.)

Edit: Linux's software RAID definitely does an initial parity initialization pass when an array is created or rebuilt. I have no prior experience with the P400i.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37705
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Mon Jan 28, 2013 10:10 am

I'm probably not going to bother trying to overprovision them anyway. Feeling a bit better about my lowly consumer-grade "notebook" SSDs: http://www.webhostingtalk.com/archive/i ... 59132.html

http://sudomakeinstall.com/linux-system ... 720-16-bay

I realize that's talking about an 830, but I figure it's similar enough. I don't know that I want to try issuing SATA commands to artificially limit the drive size shown to the RAID BIOS/ORCA.
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Mon Jan 28, 2013 10:29 am

Scrotos wrote:I realize that's talking about an 830, but I figure it's similar enough. I don't know that I want to try issuing SATA commands to artificially limit the drive size shown to the RAID BIOS/ORCA.

I'm not sure this is even *possible*. The only manually issued SATA commands I'm aware of people using to tweak their SSDs is the "secure erase" command, which essentially batch TRIMs the entire drive (restoring it to factory fresh blank state).
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37705
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Mon Jan 28, 2013 10:40 am

You mean like this?

http://tinyapps.org/docs/wipe_drives_hdparm.html

I'm guessing this limits the visible size of the drive?

Code: Select all
# hdparm -N p78165360 /dev/sdx

/dev/sdx:
setting max visible sectors to 78165360 (permanent)
max sectors   = 78165360/78165360, HPA is disabled


http://destructdata.com/blog/?page_id=282

I don't know much about hdparm but it seems that -N does the dirty work: http://linux.die.net/man/8/hdparm
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Mon Jan 28, 2013 11:07 am

Ahh, interesting. I wasn't aware of the HPA. Yes, that looks like it might be a way to under-provision at the drive firmware level!
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37705
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Mon Jan 28, 2013 11:12 am

You have to love any man page with so many "This is EXTREMELY DANGEROUS and will very likely cause massive loss of data. DO NOT USE THIS COMMAND" and "VERY DANGEROUS, DON'T EVEN THINK ABOUT USING IT" and "EXCEPTIONALLY DANGEROUS. DO NOT USE THIS FLAG!!" and "VERY DANGEROUS, DATA LOSS IS EXTREMELY LIKELY."

Looks like they have this version of pain for Windows, too: http://disablehddapm.blogspot.com/

I'm still somewhat undecided on what to do with this. Any thoughts?
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Mon Jan 28, 2013 11:32 am

Scrotos wrote:You have to love any man page with so many "This is EXTREMELY DANGEROUS and will very likely cause massive loss of data. DO NOT USE THIS COMMAND" and "VERY DANGEROUS, DON'T EVEN THINK ABOUT USING IT" and "EXCEPTIONALLY DANGEROUS. DO NOT USE THIS FLAG!!" and "VERY DANGEROUS, DATA LOSS IS EXTREMELY LIKELY."

Looks like they have this version of pain for Windows, too: http://disablehddapm.blogspot.com/

I'm still somewhat undecided on what to do with this. Any thoughts?

Well, hdparm is something that most users will never deal with. A lot of those options appear to exist for the sole purpose of helping Linux kernel developers debug disk controller device drivers, so it's a given that it will provide a number of ways to creatively shoot yourself (or at least your storage device) in the foot.

I'd say the -N option should be fairly safe (other than potentially wiping any pre-existing data on the drive). But if the RAID controller will allow you to under-provision at that level (by telling it to use only part of the drives) that would be better since it'll allow you to leave the SSDs in their factory configuration.

Or just screw the under-provisioning and don't worry about it.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37705
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Mon Feb 04, 2013 2:43 pm

For anyone curious...

HP ProLiant Array Diagnostics and SmartSSD Wear Gauge Utility (GUI and CLI) for Windows
9.10.22.0 (4 Jun 2012)

Not surprising, but this does not work to give anything useful with these SSD drives. Doesn't detect them as SSDs. Trying to install this utility via HPSUM (HP Smart Update Manager, what's in PSP (Proliant Support Pack) and probably the firmware update DVDs (10.10 was the last before they moved over to something else, I think; I used 10.00)) gets dependencies that weren't present. They were:

- HP ProLiant Advanced System Management Controller Driver for Windows
- HP ProLiant iLO Advanced and Enhanced System Management Controller Driver for Windows Editions

Turns out I can't find a version of the first one past Win2K3 and the second one's Win2K8 x64 version, when run in compatibility mode, didn't have hardware present to install anyway so didn't work. On a lark I downloaded the SmartSSD thing manually and it installed just fine. Go figure. Re-running HPSUM no longer showed it as an item to install, too.

EDIT: I guess the "Smart" Update Manager isn't so "smart" after all:

The HP SMARTSSD Wear Gauge™ requires a controller listed below (for pre-Gen8 servers) or a Gen8 Smart Array controller for HP ProLiant Gen8 servers.

HP Smart Array P212 Controller
HP Smart Array P410 Controller
HP Smart Array P411 Controller
HP Smart Array P711m Controller
HP Smart Array P712m Controller
HP Smart Array P812 Controller

In order to take advantage of SMARTSSD Wear Gauge™, Smart Array Firmware version 5.0 or greater is required and HP Array Configuration Utility (ACU) or HP Diagnostic Utility (ADU) must be running
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Mon Feb 04, 2013 6:14 pm

mattshwink wrote:EDIT: Also, it appears the P400i only supports 6 drives (see this http://h18000.www1.hp.com/products/quic ... 0_div.HTML)

This link also shows that a P400i in RAID-6 cannot have a hot spare assigned: http://itcoop.wordpress.com/2011/02/02/ ... y-trouble/
Matt


So just to clarify for anyone checking later, currently running with 8 drives in RAID 6 with a hot spare, all works. I can't duplicate a soft "drive is about to fail" SMART error but I can just yank a drive. That worked and the RAID automatically started rebuilding using the hot spare. Once it rebuilt I plugged the "failed" drive back in and it detected it and is restoring back to that drive, hot-plug. I think once it's done the drive assigned to be the hot spare will go back to being an inactive hot spare to back up any other failed drive. The array rebuild took less than an hour.

Edit: Some vindication for mattshwink. He might have been thinking about this: http://h20000.www2.hp.com/bizsupport/Te ... =c02963968

The best practices document states: "For the current generation of SSD's and Smart Array's, 6 SSD's is the limit for good performance scalabilty with typical workloads. With a 70/30 read/write workload using 8 KiB requests, the SSD's range from ~8K-12K IOPS per drive and ~6 drives exhaust the ~50K IOPS limit of the Smart Array. More drives can be added for capacity requirements but performance will not scale well after 6 SSD's." So you can have 8 if you want but don't expect the performance to scale. This of course on the P410 not on the dumpy P400i I'm using.

I don't know that stripe size will matter too much with SSDs? I'll have to look into that.

I will do some iometer testing when I figure it out to see how this array is doing versus some 15k 146 GB SAS drives in an identical configuration.
Last edited by Scrotos on Tue Feb 05, 2013 3:04 pm, edited 1 time in total.
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Mon Feb 04, 2013 11:00 pm

**Disclaimer I work for HP.. My Opinions are my own, not HP's official position**

I happen to have several HP DL380/360's G5's and 585 G2's.

I have used SSD's with the P400i controllers, with both 256 and 512MB BBWC.

The array does NOT support TRIM. The performance can and does degrade over time. As long as you have drives with good GC then you should be ok. Just be aware TRIM is not supported at all.

Also, on the P400i, the speed will be limited to SATA I. You will only get 1.5Gpbs per drive. You can see this if you install the HP Array tools on the system. You will get 3Gpbs SAS on the 146GB drives. In fact, I have two systems. A DL360G5 and a DL380G5. The DL360 has 4x120GB SSD and the DL380 has 8x72GB 15K SAS. SAS is a duplex protocol, meaning it can do both read and write to the drive interface at the same time. SATA is only half duplex. Plus, you pay a overhead penalty with SATA trancoding inside SAS transport.

From experience, the SSD's feel faster, and are very good at random IO. The 15K SAS drives will burst higher than the SSD's, but on sequential read and write. The reason of course, the SSD's are being held back by the slower interface speed. It makes sense random IO will be better on the SSD's, which is why it "seems" faster.

I would suggest, you want to stay HP, that you look for a P410i or a P420i controller. They can better use the drives. eBay should have the P410i on fairly cheap.

I have found that the MegaRAID ELP8888 controllers to be cheap and faster than the P400i. If you search hard enough, IBM re-brands the MegaRAID and they are about 100$ or less on eBay, They work better than the P400i.

HTH
TechNut
Gerbil
 
Posts: 15
Joined: Tue Apr 03, 2007 8:58 pm

Re: Suggestions? DIY SSD RAID on HP DL380 G5 w/P400i

Postposted on Tue Feb 05, 2013 10:58 am

Thanks for the info. I had thought about replacing the P400i but some of the links I read said the P410i was more finicky about SSDs having proper temperature response on their SMART or whatever. Ah, this: http://sysops.pblogs.gr/2012/04/ssds-on ... llers.html

I don't know what I should look at on SMART to know if this drive will work? For instance, a regular SATA drive shows ID 194 HDD Temperature but the SSD shows ID 190 Airflow Temperature. When I look in hpacucli the drives look like thus:

Code: Select all
      physicaldrive 1I:1:5
         Port: 1I
         Box: 1
         Bay: 5
         Status: OK
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 256.0 GB
         Firmware Revision: DXM04B0Q
         Serial Number: S12RNEACC72439Z
         Model: ATA     Samsung SSD 840
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         Current Temperature (C): 40
         Maximum Temperature (C): 70
         PHY Count: 1
         PHY Transfer Rate: 1.5Gbps


If a temperature is listed, does that mean that the P400i is getting the proper readouts from this drive and it should work in a P410i? That being the case, I might do just as you suggest and go for a P410i or later. Wouldn't need to mess with the backplane any, right? One disadvantage is that with the P410i and later I'd have to get the Advanced Array Pack to enable RAID 6 which is an additional cost.
I don't foresee this server being used anywhere near capacity so hopefully lack of TRIM won't be an issue.

Thanks for your input!
Scrotos
Graphmaster Gerbil
 
Posts: 1035
Joined: Tue Oct 02, 2007 12:57 pm
Location: Denver, CO.

Next

Return to General Hardware

Who is online

Users browsing this forum: DPete27, Google Adsense [Bot] and 6 guests