"New" work server and all the quirks that go along with it

Hang out, sip some ice tea, and shoot the breeze with TR regulars.

Moderators: emkubed, Captain Ned

"New" work server and all the quirks that go along with it

Postposted on Sat Nov 10, 2012 10:05 pm

So my Dad is really big into folding & home and a bunch of other BOINC type projects, meaning that he goes through a fair bit of hardware, including the occasional CPU upgrade.
I became the beneficiary of a Core 2 Q9550 motherboard + RAM that he has retired after building an Ivy Bridge box. As it happens, there was a need for a new server at the office for VPN and email archival purposes so...

The Hardware for a fun "New" server
What I got for free:
1. Motherboard (Foxconn P45.. nothing fancy but it works)
2. RAM (8 GB DDR2-800)
3. CPU: Q9550
4. Power Supply: Seasonic 400 watt fanless that I used for several years before a recent GPU upgrade required a beefier PSU.
5. GPU: Old ATI Radeon 4350 (NO FAN!). This is a headless server so the GPU is actually a pretty nice piece of overkill for what I need, but during the install process I did need a direct terminal and the 4350 performed its job admirably.

So that's most of the guts, but I needed a case, and more importantly a decent chunk of storage:
1. Antec 300 case. ($48)
2. Intel G2 80GB SSD ($60.. but only $40 after a $20 MIR ($0.50/GB), and the G2s are relatively reliable even though they are not speed demons.)
3. 2X WD Red WD20EFRX 2TB drives ($130 each, put into a RAID 1 mirror).


And here are some fun pictures:

Motherboard & disks mounted before dropping in the PSU and wiring everything up:
Image

So I have a 2.5" SSD and no mounting adapters and a case that doesn't have 2.5" bays? No problem: I screwed in the SSD to one side of the 5.25" bays near the base. The SSD is elevated from the base of the bay by several millimeters, so I shoved in pieces of static-safe packing foam under the SSD to provide support and prevent vibration:
Image

And after dropping in the PSU and hooking up the connectors (more on that below) here's the almost finished product (I did clean up the cabling a bit afterward):
Image

Configuration & Software:

As you might have guessed, the SSD is the boot & system software drive while big data get stored on the 2 TB hard drives. Fortunately, the Intel SSD came with the most up to date firmware revision (Linux users: you can check the revision with
Code: Select all
sudo hdparm -i /dev/sdX
where "X" is the appropriate device ID.) The WD Red drives get paired into a RAID 1. All the disks are using EXT4, and I am using the newer GPT partitioning scheme. The good news is that the GPT partitions are automatically setup for SSDs (they are correctly aligned for use with 4K blocks on SSDs). The bad news is that GPT is a little more complicated to use with older BIOS motherboards for getting a bootable disk. See below for more on how GRUB 2 is configured to make the disk bootable.

I knew this would be a Linux box, and I've been using Arch Linux for several years. So I thought "I'll just slap Arch on there!" Well, I've been *using* Arch for a long time, but I haven't actually done a fresh Arch install since 2009 (the wonders of a rolling release + disk imaging where I just copied a working install instead of reinstalling). *Wow* is the install process different than what I remembered! In some ways it is easier since the "pacstrap" and "arch-chroot" utilities made installation of packages on the SSD easier, but partitioning the SSD and getting GRUB 2 setup was definitely more complex, including the required for a 2MB dummy partition at the beginning of the drive that GRUB 2 uses for dumping legacy boot image data. Fortunately, Arch has excellent installation documentation so I was able to get it setup with a minimum of grief.

I think the biggest change with Arch is that the old install I have (that is quickly being deprecated into oblivion) relies heavily on a single config file, the /etc/rc.conf file that any Arch user is well aware of. In the new system, all of the functionality that used to be condensed into that rc.conf file is now spread out over multiple configuration files and systems. The big facilitator of killing the rc.conf file appears to be systemd, which now handles all the starting and stopping of services, and is pretty deeply hooked into other services like networking as well. The deprecation of the rc.conf file is so big that I do not even have one on the new server install! The good news is that things appear to be very well supported, and systemd is actually pretty nice once you get the hang of it. Also, it makes the boot time ridiculously fast, but that's less important for a 24/7 server.

Right now I am running an OpenVPN server on the box, and I am using the "getmail" utility to archive emails on the big storage array. More services could be installed as needed in the future too.

Fun Hardware Quirks, or: Never Trust HP

OK, so the hardware setup went.. mostly OK. The backplate for the CPU HSF was not adhesive, so I had to get awkward and hold the backplate with one hand while holding the HSF and screwdriver with the other hand during install. It worked out OK though and the CPU cores idle near 35C, so the (not so huge) HSF is doing its job.

The real issue boiled down to the SATA cables. I need 3 and I thought I had three when I gave my supplies a casual glance. Unfortunately, one of my cables was actually an eSATA cable that doesn't fit the internal connectors. So I needed another SATA cable and I'm stuck at my office which is not exactly a Newegg warehouse. Then I thought: Hey, I almost never use the DVD in my regular work PC, I'll just pull the SATA cable and replace it when I bring in a spare cable later. Oh man, bad idea. The issue is not with using the SATA cable, but with the fact that my office PC is an HP with some of the worse @#%@%# cables I've ever seen. One end of the cable in the HP machine would not release from the motherboard header and... you can see this coming... I managed to rip the plastic socket right out of the motherboard while trying to pull the cable.

I had to take a flathead screwdriver to the connector to finally pop the motherboard header of and get acess to the connector. HP uses such sub-standard parts that this SATA plug is too big and it barely fit in the header on my server motherboard too. Fortunately the server's motherboard is built much better than the HP and it accepts the low-grade plug with a minimum of issues (and this is a Foxconn motherboard, not exactly a high-end board by any stretch of the imagination).

Good news: it looks like there is another SATA header in my crappy HP motherboard, so when I get a real SATA cable I'll be able to reconnect my DVD. The rest of the PC still runs fine BTW.


So that's it for this build. This is actually the first from-parts build I've put together since 2010. I'm making sure I still have enough mad-skillz to put together a functioning Haswell box next year, and so far I haven't caught anything on fire, so I'll call it a success.
4770K @ 4.7 GHz; 32GB DDR3-2133; GTX-770; 512GB 840 Pro (2x); Fractal Define XL-R2; NZXT Kraken-X60
--Many thanks to the TR Forum for advice in getting it built.
chuckula
Gerbil Elite
Gold subscriber
 
 
Posts: 544
Joined: Wed Jan 23, 2008 8:18 pm
Location: Probably where I don't belong.

Re: "New" work server and all the quirks that go along with

Postposted on Sun Nov 11, 2012 1:47 am

You can mount the SSD on the bottom of the case there's four holes for a 2.5 inch drive. Your first and third pictures of the case shows one of screw holes.
biffzinker
Gerbil First Class
 
Posts: 109
Joined: Tue Mar 21, 2006 2:53 pm
Location: Anchorage, Alaska

Re: "New" work server and all the quirks that go along with

Postposted on Sun Nov 11, 2012 3:33 am

Just a thought but you might want to look into better ways of archiving email than getmail. Most mailservers (I mostly deal with postfix and a little exchange) have an archiving option of some sort which just sends a copy of every mail sent or received on to a special archive address. That address can be on the same system or a completely separate remote one, connected over the internet or though a VPN. I then have a script that sorts the mail in that archive account into dated folders so they can be easily transferred out to long term storage.

The main advantages of doing archiving this way is that:
- Any message passing through the server is automatically archived. There's no risk of a message getting to a user and being deleted before your getmail script runs.
- Archiving happens as new mail is received so the load gets spread out... you don't have one getmail run having to download tones of messages in one go.
- if the archive server stops for some reason messages waiting to be archived will be held on your main mail server's queue which will hopefully give you enough time to get it running again!
- You don't need to have any user's passwords sitting on your archive server for getmail to use. That's not just a security thing it's also a pain in the neck to keep up to date!

Of course this only applies if you've got control of the main mail server, if you're archiving webmail account or something then you're stuck with getmail.

AND not to forget the obligatory comment about... "so you've got raid, but where's your backup?" :wink:
Fernando!
Your mother ate my dog!
cheesyking
Minister of Gerbil Affairs
 
Posts: 2221
Joined: Sun Jan 25, 2004 6:52 am
Location: That London (or so I'm told)

Re: "New" work server and all the quirks that go along with

Postposted on Sun Nov 11, 2012 9:48 am

cheesyking wrote:AND not to forget the obligatory comment about... "so you've got raid, but where's your backup?" :wink:


Lol... this server is the backup! ;-) Our main email service is off in the magical "cloud" provided by a certain small mom & pop operation known as "Google".
I'm just looking into archiving of emails in case people at the office need to refer to copies of emails that are deleted if they exceed Google's storage limit (yes, even with 25GB inboxes this can happen.. and believe me I'd use better solutions but I ain't the boss). We are also using it to store copies of PST files for people who are still married to Outlook.

Unfortunately, since our email is all cloudified, I don't have full control of the mail servers. However... it looks like it might be possible to auto-forward to an archive account while leaving the regular message alone in the gmail system. I may look into this. It is more complicated that getmail, but it could produce better results too... THANKS!
4770K @ 4.7 GHz; 32GB DDR3-2133; GTX-770; 512GB 840 Pro (2x); Fractal Define XL-R2; NZXT Kraken-X60
--Many thanks to the TR Forum for advice in getting it built.
chuckula
Gerbil Elite
Gold subscriber
 
 
Posts: 544
Joined: Wed Jan 23, 2008 8:18 pm
Location: Probably where I don't belong.

Re: "New" work server and all the quirks that go along with

Postposted on Sun Nov 11, 2012 9:50 am

biffzinker wrote:You can mount the SSD on the bottom of the case there's four holes for a 2.5 inch drive. Your first and third pictures of the case shows one of screw holes.


Ahh that takes all the fun out of it! Thanks for the tip though. The good news is that the drive is pretty secure right now and this server sits in a nice stable closet, but if I need to work on it in the future I'll look into dropping the drive to the bottom of the case.
4770K @ 4.7 GHz; 32GB DDR3-2133; GTX-770; 512GB 840 Pro (2x); Fractal Define XL-R2; NZXT Kraken-X60
--Many thanks to the TR Forum for advice in getting it built.
chuckula
Gerbil Elite
Gold subscriber
 
 
Posts: 544
Joined: Wed Jan 23, 2008 8:18 pm
Location: Probably where I don't belong.

Re: "New" work server and all the quirks that go along with

Postposted on Sun Nov 11, 2012 12:57 pm

Looks pretty good.

I'm a bit leery of Foxconn mainboard reliability to be honest with you, after having one or two abruptly go south on me for no reason, but overall it looks well done.

I might also suggest an external eSATA or USB2 drive for scheduled backups if the data on the server is mission critical.
Core i7-2600K @4.2GHz, GIGABYTE GA-Z68XP-UD5, 4x4GB G.Skill RipJaws PC1866
Corsair 650D case, Seasonic X750 Gold PSU
WD `Raptor 600GB boot, WD Caviar Black 1TB data, NEC 7200 DVDRW
2x Radeon 6970 2GB in Crossfire, X-Fi Titanium, Dell 2407WFP
LoneWolf15
Gerbil XP
Silver subscriber
 
 
Posts: 375
Joined: Tue Feb 17, 2004 7:36 am
Location: SW Meecheegan

Re: "New" work server and all the quirks that go along with

Postposted on Sun Nov 11, 2012 1:38 pm

LoneWolf15 wrote:Looks pretty good.

I'm a bit leery of Foxconn mainboard reliability to be honest with you, after having one or two abruptly go south on me for no reason, but overall it looks well done.

I might also suggest an external eSATA or USB2 drive for scheduled backups if the data on the server is mission critical.


In this case the motherboard had been in continuous operation for about 3 years so it has had a "burn in" period (but then again it could die tomorrow for all I know). As for backup, the good news is that there are eSATA ports out the back so if I want to backup the data then that is possible. When I say that this server is "archiving" emails, I mean that it is effectively already acting as a backup for the primary email storage that is out in the magical Google cloud.
4770K @ 4.7 GHz; 32GB DDR3-2133; GTX-770; 512GB 840 Pro (2x); Fractal Define XL-R2; NZXT Kraken-X60
--Many thanks to the TR Forum for advice in getting it built.
chuckula
Gerbil Elite
Gold subscriber
 
 
Posts: 544
Joined: Wed Jan 23, 2008 8:18 pm
Location: Probably where I don't belong.

Re: "New" work server and all the quirks that go along with

Postposted on Sun Nov 11, 2012 5:02 pm

chuckula wrote:
cheesyking wrote:AND not to forget the obligatory comment about... "so you've got raid, but where's your backup?" :wink:


Lol... this server is the backup! ;-)

Don't forget to backup your backups... and then back them up too :P

chuckula wrote:Our main email service is off in the magical "cloud" provided by a certain small mom & pop operation known as "Google".


With paid Gmail there are a hole bunch of routing options that allow you to do things like I've described at domain level so you don't need to set it up in each user's account. http://support.google.com/a/bin/answer.py?hl=en&answer=2685650&ctx=cb&src=cb&cbid=-1su67mham082l&cbrank=2

Fairly soon I'm told Vault will be available to gapps users who signed up before July this year which sort of makes these archive systems obsolete... Only "sort of" because you're putting a lot of trust in Google and just trusting one company seems to go against the backup of the backups motto.
Fernando!
Your mother ate my dog!
cheesyking
Minister of Gerbil Affairs
 
Posts: 2221
Joined: Sun Jan 25, 2004 6:52 am
Location: That London (or so I'm told)

Re: "New" work server and all the quirks that go along with

Postposted on Mon Nov 12, 2012 8:19 am

cheesyking wrote:
chuckula wrote:
cheesyking wrote:AND not to forget the obligatory comment about... "so you've got raid, but where's your backup?" :wink:


Lol... this server is the backup! ;-)

Don't forget to backup your backups... and then back them up too :P

chuckula wrote:Our main email service is off in the magical "cloud" provided by a certain small mom & pop operation known as "Google".


With paid Gmail there are a hole bunch of routing options that allow you to do things like I've described at domain level so you don't need to set it up in each user's account. http://support.google.com/a/bin/answer.py?hl=en&answer=2685650&ctx=cb&src=cb&cbid=-1su67mham082l&cbrank=2

Fairly soon I'm told Vault will be available to gapps users who signed up before July this year which sort of makes these archive systems obsolete... Only "sort of" because you're putting a lot of trust in Google and just trusting one company seems to go against the backup of the backups motto.



Thanks for the link! Quick question on the email forwarding: Is there a good way to get a bulk transfer of existing emails that are already sitting on Google's servers? The forwarding looks very nice for new emails, but I need to grab copies of existing data too. Thanks again!
4770K @ 4.7 GHz; 32GB DDR3-2133; GTX-770; 512GB 840 Pro (2x); Fractal Define XL-R2; NZXT Kraken-X60
--Many thanks to the TR Forum for advice in getting it built.
chuckula
Gerbil Elite
Gold subscriber
 
 
Posts: 544
Joined: Wed Jan 23, 2008 8:18 pm
Location: Probably where I don't belong.

Re: "New" work server and all the quirks that go along with

Postposted on Mon Nov 12, 2012 8:58 am

chuckula wrote:Thanks for the link! Quick question on the email forwarding: Is there a good way to get a bulk transfer of existing emails that are already sitting on Google's servers? The forwarding looks very nice for new emails, but I need to grab copies of existing data too. Thanks again!


This is something I've been pondering for a while now. There isn't a built in method for transferring email from one user to another or to another any kind of archive for existing mail in the system. Vault will supposedly let you download to "standard archive formats" or something but you need to have Vault and I don't think it works with mail received before you got Vault anyway.

I there are various gmail backup systems out there (google brings up loads of results) but if you're probably going to want the mail in something like Maildir format. I suppose the best way to do that is with getmail and imap.
Fernando!
Your mother ate my dog!
cheesyking
Minister of Gerbil Affairs
 
Posts: 2221
Joined: Sun Jan 25, 2004 6:52 am
Location: That London (or so I'm told)


Return to The Back Porch

Who is online

Users browsing this forum: No registered users and 6 guests