New home server/file storage/ftp

Building a new system? Need help choosing between parts? Then step in and let our trained gerbils assist you.

Moderator: JustAnEngineer

New home server/file storage/ftp

Postposted on Tue Sep 13, 2011 8:02 pm

Hi people. I am currently in the research phase of this project ;)

I have some parts already, an old 780G gigabyte mobo, X2 3.1 Ghz (65nm), 4 gigs ddr2-800, probably use the onboard video and give away the HD3850 or sell it for 30 bucks and put it towards a psu, and I have 2 WD 640GB blacks. (I may add 2 more for 4 disks)

What I would like to do is basically have it set up for file storage. I was thinking of getting a decent NAS, but it's cheaper for me to just get a few parts and make a PC instead. Also I may setup an FTP on it in the future, so the relatives all over the world can access it.

So here is my list of questions.

What would be a good OS - I have a spare copy of Windows 7 Professional x64 - good enough?

What is an inexpensive raid card that is fairly reliable??

Should I raid 1 the 2 drives or get 2 more drives and raid 5 them?

Is there any issue with having a boot drive on the mobo chipset and having a raid 0 and/or raid 5 card with file storage for the network/future ftp?

I have a gigabit network and will also install a dedicated network card probably an intel pro CT

There are probably things I may have missed concerning this project so please let me know. Thanks

Edit - meant to put raid 1 and not raid 0 my bad
Last edited by anotherengineer on Wed Sep 14, 2011 5:29 pm, edited 1 time in total.
Life doesn't change after marriage, it changes after children!
anotherengineer
Gerbil Elite
 
Posts: 504
Joined: Fri Sep 25, 2009 12:53 pm
Location: Timmins, ON Canada, Yes I know, Up in the sticks

Re: New home server/file storage/ftp

Postposted on Tue Sep 13, 2011 8:10 pm

RAID 1+0 works well with 4 drives. It has minimal processing requirements, so you could get away with a cheap (or on-board) controller or do it at the OS level.
i7-4770K, H70, Gryphon Z87, 16 GiB, R9-290, SSD, 2 HD, Blu-ray, SB ZX, TJ08-E, SS-660XP², 3007WFP+2001FP, RK-9000BR, MX518
JustAnEngineer
Gerbil God
Gold subscriber
 
 
Posts: 15153
Joined: Sat Jan 26, 2002 6:00 pm
Location: The Heart of Dixie

Re: New home server/file storage/ftp

Postposted on Tue Sep 13, 2011 9:37 pm

anotherengineer wrote:Hi people. I am currently in the research phase of this project ;)

I have some parts already, an old 780G gigabyte mobo, X2 3.1 Ghz (65nm), 4 gigs ddr2-800, probably use the onboard video and give away the HD3850 or sell it for 30 bucks and put it towards a psu, and I have 2 WD 640GB blacks. (I may add 2 more for 4 disks)

What I would like to do is basically have it set up for file storage. I was thinking of getting a decent NAS, but it's cheaper for me to just get a few parts and make a PC instead. Also I may setup an FTP on it in the future, so the relatives all over the world can access it.

Keep in mind that the power savings over the lifetime of the server may make a NAS more cost-effective over the long run, even though the PC is free (or nearly free).

anotherengineer wrote:So here is my list of questions.

What would be a good OS - I have a spare copy of Windows 7 Professional x64 - good enough?

Yes, that's certainly good enough for a light duty server.

anotherengineer wrote:What is an inexpensive raid card that is fairly reliable??

Inexpensive RAID cards are all essentially just software RAID using a special proprietary driver. If you have enough SATA ports on the motherboard, you're probably better off using those and running pure software RAID. If you need more ports, just get a non-RAID SATA card and use software RAID on that.

The problem with cheap (driver-based) RAID cards is that if the card ever dies, and you can't get another identical (or at least very similar) card, you may lose the entire contents of the RAID array. You're also at the mercy of the card vendor's driver and tools; if they're not robust, then neither is your RAID array.

anotherengineer wrote:Should I raid 0 the 2 drives or get 2 more drives and raid 5 them?

My recommendation would be "neither". RAID-0 improves performance at the expense of reliability, and RAID-5 has a subtle robustness issue commonly referred to as the RAID-5 write hole. The write hole can result in silent corruption of data on the RAID volume in the event of a system crash or power outage. Bottom line is, RAID-5 is probably not the best idea unless the server will be on a UPS, with the ability to do an automatic clean shutdown when the UPS battery gets low.

I don't think Windows 7 Pro's software RAID supports RAID-5 anyway...

I recommend setting up the two drives as RAID-1, or doing 4 drive RAID-1+0 as JAE suggests.

anotherengineer wrote:Is there any issue with having a boot drive on the mobo chipset and having a raid 0 and/or raid 5 card with file storage for the network/future ftp?

Nope, shouldn't be a problem. I *think* Win7 allows you to boot directly from a software RAID volume though, so there's probably no real *need* to set things up this way.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 36922
Joined: Tue Aug 20, 2002 9:51 pm
Location: Somewhere, having a beer

Re: New home server/file storage/ftp

Postposted on Tue Sep 13, 2011 11:28 pm

If you are really only just going to use it for a 'file sever' I would, and have for that matter, get a Flash USB Stick and boot up 'FreeNAS'...

Go to 'SourceForge.net' and search for 'FreeNAS'.

It's based on BSD Unix.

I have used it for several years now.

I have found it to be very stable and fast enough for my needs.
It has a small memory foot print and does a good job using your extra RAM for cache.

That is my recommendation...

Try it.
Maybe you will like it for your needs.
Jim552
Gerbil
 
Posts: 14
Joined: Wed Jan 28, 2009 7:33 am

Re: New home server/file storage/ftp

Postposted on Tue Sep 13, 2011 11:46 pm

Yeah, FreeNAS is a reasonable option. I kind of figured we'd try to get the hardware and RAID issues out of the way first though. :wink:

My own inclination would be to install Linux on it... but I've already come up the Linux learning curve, and he indicated he's got an extra Win7 license, so I figured Win7 was the path of least resistance. If he decides he wants to keep that Win7 license for a rainy day, FreeNAS is certainly worth trying.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 36922
Joined: Tue Aug 20, 2002 9:51 pm
Location: Somewhere, having a beer

Re: New home server/file storage/ftp

Postposted on Wed Sep 14, 2011 10:26 am

It seems, from the post, that cost is an issue.

For any activity the primary goals is always to 'maximize the effectiveness of the activities intent' while 'reconciling that with real-world constraints'. (Regardless of what those 'constraints are or where they originate' they exist.)

A 'Storage System' is for storing data, first and foremost. (There are of course other benefits and liabilities as well, but 'storing data' is where it starts.)

So the most important goal is 'being able to store data safely'.

That being said...

As had been noted in earlier posts the 'cheap RAID Adapters' are not really all that great of a choice for a variety of reasons. (Personally I consider Adaptec and LSI Logic to be the better 'cheaper RAID Adapters that have most professional capabilities'.)

I think that FreeNAS solves many of the issues associated with getting a 'Networked Storage System' working.

The use of RAID Cards is not an issue, because FreeNAS uses Software RAID. Since you have only mentioned about using 2 or 4 drives, you should have enough ports already on the Motherboard.

The 'RAID-5 Write Hole' shouldn't be the deciding factor on whether to use 'RAID-5' or not. It should be 'your requirements and needs' that determine that.

FreeNAS has been tailored specifically for 'Storage System' use.
Once booted it only has a minimal text only screen so you can sell your Graphics Card and put those funds towards a Power Supply. (Most of the configuration is done VIA the Network through a WEB Interface.)

It's footprint is very small, it puts as much of it's code into RAM as possible, and uses a SWAP File very sparingly if at all. If it's run on a cheap SLOW USB Flash Device there is no noticeable difference in performance versus a faster USB Flash Device.

Data Drives are identified by a signature written onto them when so it doesn't matter what port, or card they are connected to.

Compatibility could be an issue... If you know Linux/Unix well you could get virtually any Hardware to work, but for more 'casual use' you are limited to it's 'built-in Compatibility'. (Which for me has never been a problem.)

If nothing else, during your 'Research Phase' I think you should try it...

You may like it, and it may suite your needs, then again it may not. (By no means does everything work for everyone...)

All you need to try though, is a bit of time, a 'blank CD to write on', and a 'USB Flash Drive that can be booted from'. (The other requirements you already have...)

Looking forward to hearing the direction you end up going.
Jim552
Gerbil
 
Posts: 14
Joined: Wed Jan 28, 2009 7:33 am

Re: New home server/file storage/ftp

Postposted on Wed Sep 14, 2011 11:03 am

Jim552 wrote:The 'RAID-5 Write Hole' shouldn't be the deciding factor on whether to use 'RAID-5' or not. It should be 'your requirements and needs' that determine that.

I agree that it shouldn't be the *deciding* factor... but if data security is a requirement, and a UPS is not in the budget, I'd say the "write hole" is a non-trivial negative factor for RAID-5. All it really buys you over the RAID-1+0 arrangement is some extra storage capacity (you effectively get 75% of the raw capacity of your 4 drives instead of only 50%).
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 36922
Joined: Tue Aug 20, 2002 9:51 pm
Location: Somewhere, having a beer

Re: New home server/file storage/ftp

Postposted on Wed Sep 14, 2011 12:36 pm

just brew it! wrote:
Jim552 wrote:The 'RAID-5 Write Hole' shouldn't be the deciding factor on whether to use 'RAID-5' or not. It should be 'your requirements and needs' that determine that.

I agree that it shouldn't be the *deciding* factor... but if data security is a requirement, and a UPS is not in the budget, I'd say the "write hole" is a non-trivial negative factor for RAID-5. All it really buys you over the RAID-1+0 arrangement is some extra storage capacity (you effectively get 75% of the raw capacity of your 4 drives instead of only 50%).


I was under the impression modern software RAID5 implementations (like mdadm on linux) had background 'scrubbing' processes to prevent the 'write hole'. I can't find any supporting information with a 60-second google expedition, but I thought I had read that somewhere.
Desktop: FX-8350 | 32 GB | XFX Radeon 6950 | Windows 7 x64
Laptop: i7 740QM | 12 GB | Mobility Radeon 5850 | Windows 8.1.1.1.1 x64
SuperSpy
Graphmaster Gerbil
Gold subscriber
 
 
Posts: 1499
Joined: Thu Sep 12, 2002 8:34 pm
Location: TR Forums

Re: New home server/file storage/ftp

Postposted on Wed Sep 14, 2011 1:01 pm

Background scrubbing does not completely solve the "write hole" issue. The main point of scrubbing is to proactively find latent bad blocks *before* you have a drive failure and need to do an array rebuild. If a drive fails (causing it to drop out of the array), and any of the *remaining* drives has a latent bad block on it, the array rebuild will fail; at best you'll have a royal PITA on your hands, and at worst you may lose the contents of the array... but at least you *know* you have a problem. This issue has become more important as drive capacities have soared, since there are more likely to be latent bad blocks lurking out in the array somewhere.

The "write hole" results when the parity stripe is inconsistent with the data, due to an interrupted write (e.g. power failure). While a scrub pass could indeed *identify* these inconsistencies, there's no way to reliably repair it, since you don't know which stripe(s) are the "bad" ones -- it's a data error, not a sector that physically fails to read. Furthermore... if a drive fails and an array rebuild occurs *before* the next scrub pass, the rebuild will *silently* reconstruct corrupted data for the affected block, even though the rebuild appears to succeed. It is this last failure mode that is the problem.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 36922
Joined: Tue Aug 20, 2002 9:51 pm
Location: Somewhere, having a beer

Re: New home server/file storage/ftp

Postposted on Wed Sep 14, 2011 3:07 pm

And that's why RAID is a performance and disk space enhancement hack and not a backup solution. Subsets of RAID 1 are safer, but we're still talking about mechanical disks.

anotherengineer wrote:What is an inexpensive raid card that is fairly reliable??


The $500 one which puts you most of the way to a NAS device. :) Don't worry about hardware RAID. I would use software RAID 1 with a hot spare.

The server will be more flexible and probably faster, but a NAS is hard to beat for electricity and heat. Also, the NAS will have considerably less admin time associated with it, so that's something to think about.

FreeNAS or Openfiler would be my recommendations for an OS. They have Web based GUIs which are designed to make it easy to setup shares and such. You could probably invest in a small SSD for the boot drive with those two and leave the disks for storage. When I say small, I mean 8GB would be big and 16GB would be huge.

Windows has the advantage of software, if you want to do cloud backups, and probably familiarity, but it's pretty heavy.

What do you ultimately want to do with this NAS, and how are you going to back it up?
Flatland_Spider
Gerbil Elite
 
Posts: 727
Joined: Mon Sep 13, 2004 7:33 pm
Location: The 918/539

Re: New home server/file storage/ftp

Postposted on Wed Sep 14, 2011 5:48 pm

Thanks for all the replies. I meant Raid 1 and not raid 0 sorry.

Ok so it I raid 1 the 2 drives on the 780G mobo chipset, if I use windows as my OS then would I be correct in assuming that the OS would be on the Raid 1 drives?

Second - If I use FreeNAS I put that on a usb drive and boot from the drive and the raid 1 will have 100% disk space for backing up/storing files?

Main reason for doing this. Both hard drives are currently in my PC (JOBD). I want to move them to an always on server/storage so anyone can save or access the files from any pc.
I also want them out of my pc to cut back on noise (future ssd 1st time purchase)
I would also like to set up FTP on it so the relatives from all over the can look at my pics or upload some of their pics.

Hope this adds clarification.
Thanks
Life doesn't change after marriage, it changes after children!
anotherengineer
Gerbil Elite
 
Posts: 504
Joined: Fri Sep 25, 2009 12:53 pm
Location: Timmins, ON Canada, Yes I know, Up in the sticks

Re: New home server/file storage/ftp

Postposted on Wed Sep 14, 2011 6:21 pm

anotherengineer wrote:Ok so it I raid 1 the 2 drives on the 780G mobo chipset, if I use windows as my OS then would I be correct in assuming that the OS would be on the Raid 1 drives?

Yes.

anotherengineer wrote:Second - If I use FreeNAS I put that on a usb drive and boot from the drive and the raid 1 will have 100% disk space for backing up/storing files?

Also yes.

anotherengineer wrote:Main reason for doing this. Both hard drives are currently in my PC (JOBD). I want to move them to an always on server/storage so anyone can save or access the files from any pc.
I also want them out of my pc to cut back on noise (future ssd 1st time purchase)

Sounds reasonable.

anotherengineer wrote:I would also like to set up FTP on it so the relatives from all over the can look at my pics or upload some of their pics.

Rather than a basic FTP server, you may want to consider setting things up such that you can run some sort of photo gallery server (accessed via HTTP) instead. More user-friendly that way.

Do you know whether your ISP even allows publicly accessible servers? If they block common server ports (FTP/HTTP) this could be an issue. (Check your Terms of Service...)

Also, unless your ISP assigns static IPs (which is quite rare for residential broadband service) you will also need to set up dynamic IP tracking with a DNS service like dyn.com (formerly dyndns.org). As long as you use one of their sub-domains (i.e. you're not interested in registering your own domain name), the service is free.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 36922
Joined: Tue Aug 20, 2002 9:51 pm
Location: Somewhere, having a beer

Re: New home server/file storage/ftp

Postposted on Wed Sep 14, 2011 7:50 pm

Everything sounds good, despite the overkill some people go to.

I wouldn't bother with a dedicated NIC as most on board ones work perfectly fine with little to no real downside, unless you expect oodles of network traffic (which I sorta doubt). I do have a Intel Pro 1000 PT and have been using it for years, it works beautifully. It can do some cool things like Vlans and network teaming, but generally you don't need either.

As far as a raid solution I would actually recommend a dedicated card, specifically a perc 5i or a perc 6. They can be found on ebay for roughly $130 and there is a giant community that supports them.

http://www.overclock.net/raid-controlle ... -tips.html

I would suggest doing Raid 5 as well, as it offers performance as well as redundancy, despite having a write hole. The same could be said about Windows being a terrible operating system, everything has it's pitfalls... some are worse then others.
Bensam123
Gerbil Elite
 
Posts: 941
Joined: Wed May 29, 2002 11:19 am

Re: New home server/file storage/ftp

Postposted on Thu Sep 15, 2011 5:21 am

Bensam123 wrote:I would suggest doing Raid 5 as well, as it offers performance as well as redundancy, despite having a write hole. The same could be said about Windows being a terrible operating system, everything has it's pitfalls... some are worse then others.

If doing software RAID, the performance advantage of RAID-5 is mainly on reads; the overhead of calculating the parity stripe tends to partially (or even completely) negate the higher raw I/O throughput depending on the speed of the CPU.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 36922
Joined: Tue Aug 20, 2002 9:51 pm
Location: Somewhere, having a beer

Re: New home server/file storage/ftp

Postposted on Thu Sep 15, 2011 7:13 am

I'm surprised no one brought out the Netgear ReadyNAS DUO card yet... There is NOT a better NAS system in the world for the price you pay for one of these babys!

We're talking $175 for the shell (add 2x 2tb sata hdd's to it to save $ for pre-installed).

It comes with their own AWESOME X-RAID system. The drives are HOT swappable. It has it's own photo gallery built in. It's built on linux. It's hackable. You can also do torrents on it, not sure about the FTP, but it's linux!

From the overview:

"The RND2000 from Netgear is a 2-bay network storage enclosure that will grow as your needs do. The 2 hot swappable bays support SATA and SATA II hard drives. Add capacity beyond the hard drives with the 3 USB 2.0 ports, or, attach a printer and let the print server handle your network's print jobs. A 256MB memory module speeds up operations.

Take advantage of X-RAID technology to future proof your system. Add drives as necessary and swap out smaller drives for larger as your needs increase. X-RAID will handle it all automatically; no more buying a second enclosure full of drives and transferring all of the data manually.

The RND2000 is also a fully featured media server with UPnP AV and SlimServer support. It's also Windows MCE compatible so all of your PCs on the network can access the media files from a central location. HTTP/S and FTP/S support lets you access the device from anywhere you have internet access."


How can u possibly beat that? YOU CAN'T.
thegleek
Darth Gerbil
Gold subscriber
 
 
Posts: 7344
Joined: Tue Jun 10, 2003 10:06 am
Location: Detroit, MI

Re: New home server/file storage/ftp

Postposted on Thu Sep 15, 2011 9:44 am

just brew it!"]Background scrubbing does not completely solve the "write hole" issue.


While 'Scrubbing" does have many usages I am assuming for this that we are referring to some sort of RAID Integrity Checks. ('Scrubbing', 'Resync', 'Array Check', 'Integrity Check', 'Patrol Reads'. There are undoubtedly more but these are the ones that come to mind)

I have not ran across any recent Hardware/Software RAID Products that do not allow this.

YES, this does 'completely repair' any parity problems associated with the 'RAID-5 Write Hole' issue. (Technically is does not 'solve' the potential issue, but any damage would be repaired. Specifically if you are using 'mdadm' this can be done.)

The "write hole" results when the parity stripe is inconsistent with the data, due to an interrupted write (e.g. power failure).


Is an 'accurate representation' of the what the 'RAID-5 Write Hole' problem is.
But...

While a scrub pass could indeed *identify* these inconsistencies, there's no way to reliably repair it, since you don't know which stripe(s) are the "bad" ones -- it's a data error, not a sector that physically fails to read.


The 'RAID-5 Write Hole' specifically refers to the situation where 'RAID-5 Parity is incorrect for the data contents'. By definition 'scrubbing' would fix any 'RAID-5 Write Hole' damage.

Any situation where RAID Hardware/Software doesn't "know which stripe(s) are the "bad" ones" is 'data corruption'.

While 'data corruption' could cause a 'RAID-5 Write Hole' issue it is a much larger problem.

'Data Corruption' is a problem which will affect data regardless of how your storage is configured.

Various scenarios can always be devised where specific solutions are not valid, but scenarios can also be devised where only a specific solution is valid as well.


Ultimately do keep in mind...

"ALL THINGS FAIL, it's only a matter of time".
"ALL things critical need to be backed up".
"Always strive to use the simplest solution that fulfills your needs".
Jim552
Gerbil
 
Posts: 14
Joined: Wed Jan 28, 2009 7:33 am

Re: New home server/file storage/ftp

Postposted on Thu Sep 15, 2011 9:52 am

Jim552 wrote:The 'RAID-5 Write Hole' specifically refers to the situation where 'RAID-5 Parity is incorrect for the data contents'. By definition 'scrubbing' would fix any 'RAID-5 Write Hole' damage.

My understanding is that the "write hole" issue is more general than that, specifically: due to a power failure or system crash some stripes get written, and some do not. The bad stripe *might* be the parity stripe, but you have no way of knowing that. With a traditional RAID-5, all you know in this situation is that the parity check fails; there is *nothing* that tells you whether it is the parity that is incorrect, or one (or more) of the data stripes.

A scrub pass can't fix the problem, because it has no way of knowing which stripe to repair.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 36922
Joined: Tue Aug 20, 2002 9:51 pm
Location: Somewhere, having a beer

Re: New home server/file storage/ftp

Postposted on Thu Sep 15, 2011 10:17 am

Second - If I use FreeNAS I put that on a usb drive and boot from the drive and the raid 1 will have 100% disk space for backing up/storing files?


This has been answered.

For additional detail though, the newer versions of FreeNAS do not allow, without some hacking around, shares to be the device where it's 'System Volume' resides. (Which I do think is a really good thing. I have 'never been a fan' of mixing the two uses in more 'critical situations'.)

That was why I recommended the 'USB Flash Drive'.

I have used 8GB USB Flash Drives, and could easily downgrade to 4GB. (But the 8GB ones were only 9 bucks...) I have tested it on 64GB High Speed USB Flash Drives and for 90% of the Administrative Operations I do not see any significant difference, and for NAS Performance there is no difference that I have ever found.

For installation on the slower USB Flash Drives it takes about 20 minutes, on the faster ones that drops to about 3 minutes.

I would also like to set up FTP on it so the relatives from all over the can look at my pics or upload some of their pics.


FreeNAS does allow for FTP Services so it could still work for your purposes. (Other issues with Internet Access have been noted already.)

I have not ever used 'Dynamic DNS', so I do not know if FreeNAS would work with that.

The more things you add to what you want to do, the more likely it becomes that a 'General Purpose OS' would be a better choice for you. I do though think it is a very good idea to 'separate OS from Data' on your storage devices if you can though.

I want to move them to an always on server/storage so anyone can save or access the files from any pc.


In regards do this, that is even more reason to 'use the onboard video'.

My primary PC with a GPU Card and 4tb of storage runs about $16 per month for 'always on use'.

My little FreeNAS Server with 4tb runs about $5 per month for 'always on use'.

If it is 'always going to be on' then the electric costs may factor in. Not necessarily in regards to 'whether you can pay for it', but rather in just recognizing that using/not using specific components can make a difference over time.

Also keep in mind that the 'more Services/Capabilities that are exposed the more chances there are of getting hacked' across the Internet in a 'always on' situation.

That 'may or may not' be an issue, but something that should be thought about to some degree.
Jim552
Gerbil
 
Posts: 14
Joined: Wed Jan 28, 2009 7:33 am

Re: New home server/file storage/ftp

Postposted on Thu Sep 15, 2011 10:20 am

thegleek"]I'm surprised no one brought out the Netgear ReadyNAS DUO card yet... There is NOT a better NAS system in the world for the price you pay for one of these babys!

We're talking $175 for the shell (add 2x 2tb sata hdd's to it to save $ for pre-installed).


They are pretty good devices.

I think the reason Products like this have not been brought up is because the question was specific about 'reusing existing left-over hardware'.
Jim552
Gerbil
 
Posts: 14
Joined: Wed Jan 28, 2009 7:33 am

Re: New home server/file storage/ftp

Postposted on Thu Sep 15, 2011 10:27 am

Not wanting to continue turning this into another RAID argument...

but I'd be more concerned about getting a decent backup system in place before spending time and money on raid.

On a home server performance isn't much of an issue as the server isn't going to be loaded with simultaneous requests from hundreds of different users and most modern sata drives can saturate even a gigabit network connection.
Fernando!
Your mother ate my dog!
cheesyking
Minister of Gerbil Affairs
 
Posts: 2221
Joined: Sun Jan 25, 2004 6:52 am
Location: That London (or so I'm told)

Re: New home server/file storage/ftp

Postposted on Thu Sep 15, 2011 1:38 pm

just brew it! wrote:If doing software RAID, the performance advantage of RAID-5 is mainly on reads; the overhead of calculating the parity stripe tends to partially (or even completely) negate the higher raw I/O throughput depending on the speed of the CPU.


Yup! That's why I suggested a Perc 5i to the parent, you can even buy BBUs for them so it completely solves your write whole issue (battery backup unit). Sometimes you can even get them on the card for around $130, sometimes you have to pay extra. Perc 5i is a PCI-E SAS card that Dell makes specifically for their servers, but many people do not use them so they sell them on eBay. SAS cards can be used with SAS drives or with normal SATA drives.

They're quite amazing for the price you pay for them and are a little niche for hard drive enthusiasts. Most multport SATA controllers will run in software up to about $200 (which you noted) and then above that you can get some hardware based ones, but they get really expensive the more ports you have. The perc 5i is 8 ports and runs around $800 normally. You can get a Perc 6i around the same price, that is faster, depending on the auctions.


I will note my friend ran into this Write Hole issue you guys are throwing back and forth and had to scrap his entire array because of it. He brought it up at the time because it was a ridiculously isolated incident. Keep in mind his scenario was one of his hard drives failed and his array was rebuilding at the time and he suffered a power loss. So that makes for some very specific circumstances. He ended up switching to raid 0+1 after that and bought a UPS.
Bensam123
Gerbil Elite
 
Posts: 941
Joined: Wed May 29, 2002 11:19 am

Re: New home server/file storage/ftp

Postposted on Thu Sep 15, 2011 2:13 pm

Jim552 wrote:I have not ever used 'Dynamic DNS', so I do not know if FreeNAS would work with that.

The more things you add to what you want to do, the more likely it becomes that a 'General Purpose OS' would be a better choice for you.


There are a couple ways to use a DDNS service. The best way is to have router with a DDNS client baked in. The next way is to have a client on a frequently used desktop, and if nothing else, he could probably get a DDNS client from ports if he had to. I haven't played around with FreeNAS in a while, but I'm sure someone has gotten ports on it. It may be as easy as running portsnap; I don't know.

I should also be mentioned that FreeNAS can create ZFS volumes, and ZFS is designed to create storage volumes out of regular disks hanging off a non-RAID controller while safe guarding the integrity of the data. More into here: Wikipedia: ZFS

With "General Purpose OS" comes a lot of admin overhead: setup, updates, security, manual config. Playing sysadmin at home is something to think about. It really comes down to if the OP wants a pet or an appliance.
Flatland_Spider
Gerbil Elite
 
Posts: 727
Joined: Mon Sep 13, 2004 7:33 pm
Location: The 918/539

Re: New home server/file storage/ftp

Postposted on Thu Sep 15, 2011 2:24 pm

Flatland_Spider wrote:... It really comes down to if the OP wants a pet or an appliance.

I had to LOL when I read this. I would've probably used the word "hobby" instead of "pet," but "pet" fits too. Especially when your shiny new custom server build sh*ts the carpet for no descernible reason, and you have to clean up the mess! :lol:
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 36922
Joined: Tue Aug 20, 2002 9:51 pm
Location: Somewhere, having a beer

Re: New home server/file storage/ftp

Postposted on Thu Sep 15, 2011 6:26 pm

Thanks for all the replies. I think I will go raid 1 over raid 5. My disks are WD blacks and will more thn likely saturate a gigabit network anyway.

If I have a 2 disk raid 1 on the mobo chipset, could add 1 or 2 more hdd's to it later for extra redundancy? And if 1 drive fails, could I just replace the bad drive and will the software raid automatically rewrite everything from the good drive to the new blank drive?

I have heard of ZFS file system, but can it be accessed by windows? Say the server PC dies, if the file system is fat32 or ntfs I could drop the drives in my docking station and backup/access them from windows.

Thanks Brew it for the ISP info and static IP, I wouldn't have thought of it until I ran into a problem, so something to look into.

Also with FreeNAS, what file types does it support? (Fat32, ntfs?) I have zero experience with it and Linux, but it would allow me to save my win7x64 pro for something else.
Life doesn't change after marriage, it changes after children!
anotherengineer
Gerbil Elite
 
Posts: 504
Joined: Fri Sep 25, 2009 12:53 pm
Location: Timmins, ON Canada, Yes I know, Up in the sticks

Re: New home server/file storage/ftp

Postposted on Fri Sep 16, 2011 4:53 am

anotherengineer wrote:I have heard of ZFS file system, but can it be accessed by windows? Say the server PC dies, if the file system is fat32 or ntfs I could drop the drives in my docking station and backup/access them from windows.

Also with FreeNAS, what file types does it support? (Fat32, ntfs?) I have zero experience with it and Linux, but it would allow me to save my win7x64 pro for something else.


Freenas does support ntfs but you'd probably do better with a native filesystem. You can always access the drives from your main PC using a live CD for disaster recovery.

If you did go down the *nix route you won't want to mess around with the raid on the motherboard as it won't be properly supported. freenas has it's own software raid management built in. I'm a little out of date on this subject but getting the actual OS to boot from a raid volume is a bit tricky, it's more about "protecting" data.
Fernando!
Your mother ate my dog!
cheesyking
Minister of Gerbil Affairs
 
Posts: 2221
Joined: Sun Jan 25, 2004 6:52 am
Location: That London (or so I'm told)

Re: New home server/file storage/ftp

Postposted on Fri Sep 16, 2011 8:54 am

cheesyking wrote:If you did go down the *nix route you won't want to mess around with the raid on the motherboard as it won't be properly supported.

Just to clarify this: The RAID software provided by the motherboard vendor won't work on *NIX. But that doesn't mean you can't run pure software RAID on the motherboard ports. In general you don't want to run the "fake hardware RAID" provided by the motherboard vendor anyway, so this is basically non-issue.

cheesyking wrote:freenas has it's own software raid management built in. I'm a little out of date on this subject but getting the actual OS to boot from a raid volume is a bit tricky, it's more about "protecting" data.

I haven't personally kept up with FreeNAS so I can't comment directly on its RAID boot capabities. However, newer versions of Linux do indeed support software RAID boot volumes. The desktop I'm posting this from boots Ubuntu 10.04 from a software RAID-1 array connected to the motherboard's SATA ports. The motherboard RAID capability is disabled; the SATA controller is set for generic AHCI. While I agree that it is still somewhat less than straightforward to install to a software RAID-1, IMO it is now within the realm of "things mere mortals can actually do". :wink:
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 36922
Joined: Tue Aug 20, 2002 9:51 pm
Location: Somewhere, having a beer

Re: New home server/file storage/ftp

Postposted on Fri Sep 16, 2011 11:01 am

FreeNAS is FreeBSD rather then Linux. :)

There is some churn going on in the FreeBSD RAID stack, and booting off a RAID volume hasn't been implemented yet. FreeNAS Support Ticket

Windows doesn't support ZFS. NTFS and the FAT variants are the only filesystems that get really good support in Windows. Unices are pretty flexible about moving between hardware, so you would just need to boot to a live environment to recover the data.

FreeNAS supports all of the filesystems FreeBSD does. UFS and ZFS are the recommended filesystems for a running system, but it can read/write to anything that's fairly common.
Flatland_Spider
Gerbil Elite
 
Posts: 727
Joined: Mon Sep 13, 2004 7:33 pm
Location: The 918/539

Re: New home server/file storage/ftp

Postposted on Fri Sep 16, 2011 11:14 am

Flatland_Spider wrote:FreeNAS is FreeBSD rather then Linux. :)

Yeah, I knew that...

Flatland_Spider wrote:FreeNAS supports all of the filesystems FreeBSD does. UFS and ZFS are the recommended filesystems for a running system, but it can read/write to anything that's fairly common.

I fully admit I haven't kept up on what's going on with ZFS.

So the ZFS license is fully compatible with the BSD license? (The ZFS/GPL licensing incompatibility issue is apparently the reason ZFS currently has second class citizen status in the Linux space.)
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 36922
Joined: Tue Aug 20, 2002 9:51 pm
Location: Somewhere, having a beer

Re: New home server/file storage/ftp

Postposted on Fri Sep 16, 2011 3:57 pm

just brew it! wrote:Yeah, I knew that...


I couldn't remember if you did or not. A lot of people get confused....

Flatland_Spider wrote:FreeNAS supports all of the filesystems FreeBSD does. UFS and ZFS are the recommended filesystems for a running system, but it can read/write to anything that's fairly common.

I fully admit I haven't kept up on what's going on with ZFS.

So the ZFS license is fully compatible with the BSD license? (The ZFS/GPL licensing incompatibility issue is apparently the reason ZFS currently has second class citizen status in the Linux space.)[/quote]

It is. I don't think there is any license that's incompatible with the 3- and 2-clause BSD licenses. I think the GPL/CDDL incompatibility comes from the GPL requirement that no further restrictions be placed on the code.

The BSD devs also imported Dtrace from Solaris. It's too bad Oracle closed off Solaris development.
Flatland_Spider
Gerbil Elite
 
Posts: 727
Joined: Mon Sep 13, 2004 7:33 pm
Location: The 918/539

Re: New home server/file storage/ftp

Postposted on Sun Sep 18, 2011 9:38 pm

I haven't personally kept up with FreeNAS so I can't comment directly on its RAID boot capabities


I am not really sure why anyone is talking about the FreeNAS, (*nix stuff in geneeral...), being able to boot from RAID?

For FreeNAS spend the $8 on a decent, albeit slow, 8GB USB Flash Drive.
Load FreeNAS on that.

If the Flash Drive Fails, buy a new one, load FreeNAS on that and you're up again. (As long as you didn't go too wild with the various security stuff, or go into the Console to change things behind the WEB Interface...)

Simple...

FreeNAS has been optimized to not do too many writes to it's Boot/System Drive so it's not going to be too much wear on the USB Flash Drive.

Keep the System and the Data Drives Completely Separate.

As for the RAID...
Yes, ALL RAID in FreeNAS is software.

I haven't done extensive testing on it quite yet, but in general it does seem quite speedy.
It is my 'test NAS' and will be my 'primary NAS' before the end of the year.

I have zero experience with it and Linux...


Not even remotely an issue if you just stay what the FreeNAS WEB interface is setup to do.

For all intents and purposes FreeNAS is 'Appliance Software'...
Just provide necessary hardware, install, and you're up.
(Well you need to change a password, format hard disks, provide a bit of configuration, and a few other things. There is nothing else you need to 'hunt around for', 'nothing else to install', 'no *nix config files to deal with or figure out', etc...)

While Hardware Compatibility 'out of the box' is by no means 'guaranteed', I have not had any issues at all.

The FreeNAS WEB Interface is where the bulk of the configuration is supposed to be done across a network.

The FreeNAS Console basically has scripts to configure the Network Adapter.
(You can go to the Console and do whatever you want/know how to do, but I would recommend against that...)

Once you Login to the FreeNAS Server with your Browser change the password... (The default 'user/password' is 'admin/password' for ver6.x. v7.x is still in beta I think...)

If you already have a USB Flash Drive that you can use, permanently or temporarily, all you need is to go to SourceForge.net, download the software, burn it to a CD, and install.

Play around, see if it makes sense 'to you', as well as 'for your needs'. Start 'simple', see what you are comfortable with, see what you need...

I wouldn't worry about File System Compatibility with Windows.

FreeNAS implements SMB, ('Server Message Block' Protocol...), so that allows other Computers to Access your Data from Across the Network. (The underlying File System that the files are actually stored on, makes virtually no difference.)

It also implements various protocols for other network access as well.

One thing that is not readily obvious, well I do suppose if one read the Manual , is that you need to go to the 'Service Configuration' use the 'slider switch/button' to enable the network protocols that you want. (Always 'enable' the minimum number of Services that you need/want...)
Jim552
Gerbil
 
Posts: 14
Joined: Wed Jan 28, 2009 7:33 am

Next

Return to System Builders Anonymous

Who is online

Users browsing this forum: No registered users and 3 guests