Building a File Server

The network is the forum.

Moderators: Steel, notfred

Building a File Server

Postposted on Thu Aug 26, 2010 7:46 am

I have been running WHS version 1 and then later the next WHS Vail beta for about a year now but I was looking to try something different as I have been using more and more of Ubuntu. I really like WHS but I wanted to test some alternatives for the time being. I am now just starting to grasp the basics of Ubuntu and its powerful command space and so I want to experiment more. My current network is wireless N+Gigabit. Here's the current hardware I have and would like to run it on.

ZOTAC IONITX-A-U Atom 330 1.6GHz Dual-Core 441 http://www.newegg.com/Product/Product.aspx?Item=N82E16813500027
2gb of RAM
2x2TB WD Green HD (1 or both is of the EARS variety)

I currently only have about 2TB of space needed but I have enough space to backup using spare HDDs and remove them. (not listed above)

Here's what I would like to use it for:

File Serving - documents, videos and music
File Streaming on an Xbox 360 (and Wii???????)
Torrents
FTP server
I know this one is possibly a stretch but I have a TV tuner (ATI650 USB) that I would like to record TV with directly to the server rather than leave a HTPC on 24/7.
Remote Access to the files on the server (don't know if this could be considered FTP or not but guessing not)

I thought of two options and if anyone has other suggestions I would greatly appreciate it.

1) FreeNas - I like the simplicity of it and that it is mostly a GUI but I have some questions about it

Should I use the ZFS file system? It looks promising but I know people have had problems
Should I use a small USB drive as the OS?
Are file slow downs with transfers a real issue with Windows systems?
It recommends using an embedded system, is that what I should use even if I add and get rid of HDD's a lot?

2) A Linux system of some sort - More powerful but is it more complex than I need

Is the atom powerful enough to run everything?
Which variant would be the best to use?
Can this be ran headless?

Any thoughts and suggestions would greatly be appreciated.
To Start Press Any Key'. Where's the ANY key?
If something's hard to do, then it's not worth doing
You know, boys, a nuclear reactor is a lot like a woman. You just have to read the manual and press the right buttons.
mmmmmdonuts21
Gerbil Elite
 
Posts: 590
Joined: Wed Jul 16, 2008 9:09 am

Re: Building a File Server

Postposted on Thu Aug 26, 2010 10:46 am

Just curious why you wanted an Ion system. You could grab a atom D510 board for half that price.
Ahhhhhhh!
chasscF1
Gerbil Team Leader
 
Posts: 242
Joined: Thu Sep 29, 2005 1:18 pm
Location: Cincinnati, OH

Re: Building a File Server

Postposted on Thu Aug 26, 2010 10:49 am

i would go with the Linux server, easily gives you the most operation to operate the different type of server's you want. Even if you aren't super knowledgeable about the subject there are many people out there that are and are willing to spread the linux good-ness.

Also, I have been instructing people to stay away from Western Digital Green Drives. The parking issue has killed off 2 of my hard-drives and it seems that the third one doesn't have much longer to live. They are in warranty but you still don't want this happening....
StuG
Graphmaster Gerbil
Silver subscriber
 
 
Posts: 1457
Joined: Wed May 23, 2007 11:19 pm
Location: Florida

Re: Building a File Server

Postposted on Thu Aug 26, 2010 10:55 am

1) The current verison of FreeNas is based off FreeBSD 7.2, which supports ZFSv6. FreeBSD 7.3 and FreeBSD 8.0 upgraded to ZFSv13. FreeBSD 8.1 upgraded to ZFSv14. The version in Open Solaris is ZFSv24. These version numbers describe the various features that are available. Anyway, ZFSv6 support is considered to be an experimental feature in FreeBSD 7.2. ZFSv13 support is considered to be a stable feature in FreeBSD 8.0. Here is a relatively recent thread regarding its stability on the FreeBSD forums:

http://forums.freebsd.org/showthread.php?t=9116

If I were you, I would probably opt for a UFS filesystem on FreeNAS. You should not use a USB key as your OS drive because USB keys are raw flash devices and as far as I know, neither UFS nor ZFS have special facilities for doing wear levelling and FreeBSD has no other options in terms of file systems. I think Microsoft fixed the slow file transfers under Windows Home Server. Lastly, where is the recommendation to use an embedded system? You should be able to use whatever you like, embedded or otherwise.

Your Atom question is more of a generic question than something particular to Linux. The Atom should be overkill for everything but the DVR functionality. The DVR functionality can possibly have much higher CPU requirements; I suggest you read MythTV's documentation on the subject:

http://www.mythtv.org/docs/mythtv-HOWTO-3.html

2) Which Linux distribution is the best to use varies based on what your definition of best is. Best can be some a combination of several things, such as most customizable, most secure, least setup time, highest performance, etcetera. It really depends on what you consider to be the best. My personal favorite is Gentoo Linux, which has a package manager that was inspired by FreeBSD's ports system and can be customized to do anything any other system can do.

http://www.gentoo.org/

Gentoo Linux is a minimalist rolling distribution that compiles almost everything it installs according to your specifications. It is the only mainstream Linux distribution in that category and it has been remarkably innovative in minimizing the amount of work a user needs to do in order to gain such a high degree of customization. It is the complete opposite of more traditional full-featured static binary distributions, which from a design aspect, share a great deal in common with Microsoft Windows. A more traditional option for this application would likely be CentOS:

http://www.centos.org/

CentOS is a clone of RedHat Enterprise Linux. All of the source code for RedHat Enterprise Linux is available from RedHat. The people behind it took the source code, stripped it of RedHat's trademarks for copyright purposes and compiled it, producing CentOS. RedHat Enterprise Linux is the Linux distribution of choice for running servers, but it requires that you have to pay for an expensive service contract, which is why CentOS has become very popular. An alternative to RedHat Enterprise Linux is SUSE Linux by Novell. It is by no means as popular as RedHat Enterprise Linux and as far as I can tell, it is often not considered to be as good. Both RedHat Enterprise Linux and SUSE Linux suffer from often being out of date, even more so than other binary distributions. For instance, the version of the Linux kernel used in RedHat Enterprise Linux is almost 3 years old, although it has been heavily patched by RedHat over that time period, mostly for security fixes. As a consequence, I am not sure if driver support is available to support a TV Tuner.

A good compromise between the two differing types of Linux distributions might be Sabayon Linux, which should work on your system as long as it is Intel-compatible. With other distributions like Gentoo Linux and RedHat Enterprise Linux, you are not limited to Intel-compatible systems. I believe that Gentoo Linux supports more architectures than RedHat Enterprise Linux. If you do not like, how limited Sabayon is in terms of architecture support, some other good options would be Fedora Linux and Debian Linux. Fedora Linux is a distribution where RedHat experiments with features before making a new release of RedHat Enterprise Linux. Debian Linux is a distribution that as far as I can tell, is developed in a manner as similar to FreeBSD as is possible. It also has broad architecture support, roughly analogous to the architecture support provided by Gentoo Linux.

All Linux distributions are modeled after UNIX, which is designed for headless operation. As a consequence, both Linux and FreeBSD rely on the X Window System for graphical support. The X Window System was designed for people running on terminals connected to a big mainframe, which allowed them to use GUI applications that ran on the main frame, but appeared on their local computers. The end result was a somewhat laggy experience. Modern UNIX desktop environments that run locally still suffer from this, although since everything is done locally, the networked communication occurs inside the computer, making the lag difficult to perceive in most cases.

I think that Gentoo Linux would be the most educational way of doing this, although producing a production system with it will also take the most time. Very little of that time will require work from you, because the majority of the time will be spent compiling software. There are ways of making that take less time (e.g. ccache, distcc), but as a beginner, you will not know how to do any of them. Also, while almost any Linux distribution can in theory be installed on a headless systems without ever installing a graphics card with a monitor and a keyboard, such installations typically require expert level skills, so you will likely not be able to do them. As a consequence, you will want a graphics card, a monitor and a keyboard for the installation process with any Linux distribution. Then you could follow the Gentoo Linux handbook:

http://www.gentoo.org/doc/en/handbook/

If you must do a headless installation, you have a few options. You will need to ensure that the BIOS is configured in a manner that allows for whichever one you choose (which means that you might not be able to escape doing things 100% headlessly), although I think that typical BIOS defaults should allow you to do any of them without modifying the BIOS settings.

The first option is to install the system in a virtual machine on a virtual disk, making certain that drivers are installed for both your virtual machine and your target hardware (which is something I cannot stress enough). The virtual machine's hard drives should be configured to the exact configuration that you will have on your real system, because since this is a headless system, it is not a simple thing to take it offline and run some maintenance disk on it to modify the partitioning and filesystems to suit your target system. The key points are to make certain that you have the proper storage system drivers compiled into the kernel (not as modules!), that you have the proper networking drivers available (compiled as modules or as part of the kernel; my preference is as modules), that you have set the system passwords to be something you know (which is very important, even if you do not run headless), that you have your system configured to connect to the network using either DHCP to get an IP address or some static configuration to set an IP address at boot and that SSH is configured to run at boot. Then you could just copy an image of the virtual disk to your target system's hard drive, install the hard drive in the target system and start it. In theory, it should work and you should be able to SSH into it. If it does not work, then the likely problem is that your kernel does not have appropriate storage drivers (which means a kernel panic, the equivalent of a BSOD) or your networking (i.e. drivers, networking configuration, sshd) is not configured properly (which means that the system is running, but you cannot talk to it and it will not talk to you.

Your second option can be easier than the first option, but at the same time, it can also be more difficult. This method involves booting from a LiveCD that has a few special properties.
  1. It will automatically boot a Linux distribution
  2. It will automatically configure networking
  3. It will automatically start sshd
  4. It has all of the tools required to install Gentoo Linux available either by default or by some installation command that can be done while the LiveCD is running.
  5. It has a known username and password that can be used to obtain access to root via SSH.

The first 4 properties are not difficult to find (e.g. System Rescue CD has them), but the last property is something that I have not seen in the LiveCDs I have used. It might be necessary to make your own LiveCD, which is why this can be much more difficult than imaging a system you installed inside a virtual machine.

Your last option is to do a network installation via a PXE boot. In theory, you can combine this with the second option to produce a system that neither has a graphics card or optical drive at any point during the installation and it is often done with embedded systems. The only guide I could find on this was one that used PXE as a substitute for having an optical drive, which is not a headless installation per se, but if your LiveCD is a special LiveCD I outlined above, then a network installation via a PXE boot can be done headlessly. You will likely need to at the very least peek at the BIOS settings to know what the defaults are, which could be done either with a graphics card (which is cheating) or by calling your motherboard manufacturer's technical support. Although I do not know for certain, I would expect that level 1 technical support will not be able to answer this question for you, so you would need to escalate it to level 2 or level 3 before you can get the answer. If the default settings do not allow you to do a PXE boot by default (assuming that you purchased a motherboard that does support PXE boots), you will need to use a graphics card, a monitor and a keyboard to allow you to modify them, although any motherboard that supports PXE boots should by default attempt one if it cannot find a boot loader on the usual devices (e.g. optical drive, floppy, usb key, floppy drive). If your motherboard really does support PXE, but is not configured to use it by default, there are ways of changing the BIOS settings without a graphics card, but they likely require doing things that are done in the factory, which is outside of the scope of my ability to describe. Another possibility is just attaching a keyboard and making guesses about the internal state of the system to be able to modify the BIOS settings, but that is masochistic.

http://www.howtoforge.com/ubuntu_pxe_install_server

In theory, any Linux distribution can be used as a PXE server, although Ubuntu Linux is the one used in that guide. Also, in theory, any Linux distribution can be installed by these methods and not just Gentoo Linux, but then again, you seem to be starting out with Linux, so you should probably try more simple methods of installation (e.g. the Gentoo Manual's procedure) instead of something like this. Attempting something like this from the start will likely discourage you, because doing it correctly requires knowledge that you likely do not have and I can guarantee that obtaining it in the process of doing this will be an extremely frustrating experience.

I hope you find this helpful. It took me 2 hours to write.
Disclaimer: I over-analyze everything, so try not to be offended if I over-analyze something you wrote.
Shining Arcanine
Gerbil Jedi
 
Posts: 1717
Joined: Wed Jun 11, 2003 11:30 am

Re: Building a File Server

Postposted on Thu Aug 26, 2010 11:26 am

That has to be one of the longest responses on Tech Report. :o
StuG
Graphmaster Gerbil
Silver subscriber
 
 
Posts: 1457
Joined: Wed May 23, 2007 11:19 pm
Location: Florida

Re: Building a File Server

Postposted on Thu Aug 26, 2010 11:31 am

First of all thank you for taking the time and energy to write that post Shining Arcanine. That took me 25 minutes to read and understand. I found it extremely useful and informative.

So I think my best bet than is to run a full linux distribution for the server then.

Just curious why you wanted an Ion system. You could grab a atom D510 board for half that price.


I originally grabbed this board about a year ago when it was being used for a HTPC and the Ion was extremely helpful in the graphics department.

Also, I have been instructing people to stay away from Western Digital Green Drives


I have not had any issues yet and I got these two drives a while back so I will be sure to keep a careful eye on them and probably purchase different HDDs next time.

Shining Arcanine:

I will definitely check out Gentoo. It looks promising. I may run it in a VM first for a little bit just to get a hang of how the system handles.
I don't need to do a headless installation at all, but I would really like to run it headless after I got the system up and running. I believe that would be much easier to do.
To Start Press Any Key'. Where's the ANY key?
If something's hard to do, then it's not worth doing
You know, boys, a nuclear reactor is a lot like a woman. You just have to read the manual and press the right buttons.
mmmmmdonuts21
Gerbil Elite
 
Posts: 590
Joined: Wed Jul 16, 2008 9:09 am

Re: Building a File Server

Postposted on Thu Aug 26, 2010 12:01 pm

Shining Arcanine wrote:You should not use a USB key as your OS drive because USB keys are raw flash devices and as far as I know
They're not raw flash. Raw flash exposes a read/erase/write interface, whereas a USB key "looks" like any other USB mass storage device. I'm not suggesting using them as an OS drive either, but for different reasons.

mmmmmdonuts21 wrote:I will definitely check out Gentoo.
If you want your file server to be stable and not a lot of continuous upkeep and maintenance work, I'd suggest using CentOS or Debian stable (possibly plus the backports repository) or Ubuntu LTS rather than Gentoo.
bitvector
Grand Gerbil Poohbah
 
Posts: 3234
Joined: Wed Jun 22, 2005 4:39 pm
Location: Mountain View, CA

Re: Building a File Server

Postposted on Thu Aug 26, 2010 1:21 pm

What you wanna do for your greens is get HD Tune PRO and look at the Load Cycle Count. Most drive this will be low, but for Greens its always extremely high. Green Drives are said to last around 300,000 with anything above that being "lucky". I had two of them, the one I am RMA'ing to WD right now had over 1,000,000 of them on there. The other one I have has 200,000. I have had them for less than a year ><
StuG
Graphmaster Gerbil
Silver subscriber
 
 
Posts: 1457
Joined: Wed May 23, 2007 11:19 pm
Location: Florida

Re: Building a File Server

Postposted on Thu Aug 26, 2010 1:24 pm

Also you might want to consider Windows Home Server.
Carpe diem quam minimum credula postero
sid1089
Gerbil Team Leader
 
Posts: 290
Joined: Wed Jul 26, 2006 4:56 am

Re: Building a File Server

Postposted on Thu Aug 26, 2010 1:46 pm

A dual-core Atom should be able to handle all of your wants, except maybe the TV tuner. Depends on how the tuner works; if it is a hardware encoder supported by Linux, it might be fine, but a software encoder will probably be too much for poor little Atom.
FuturePastNow
Gerbil Elite
 
Posts: 601
Joined: Mon Nov 24, 2008 11:28 am

Re: Building a File Server

Postposted on Thu Aug 26, 2010 1:49 pm

mmmmmdonuts21 wrote:I have been running WHS version 1 and then later the next WHS Vail beta for about a year now but I was looking to try something different as I have been using more and more of Ubuntu. I really like WHS but I wanted to test some alternatives for the time being.
sid1089 wrote:Also you might want to consider Windows Home Server.
wibeasley
Gerbil Elite
Gold subscriber
 
 
Posts: 952
Joined: Sat Mar 29, 2008 3:19 pm
Location: Norman OK

Re: Building a File Server

Postposted on Thu Aug 26, 2010 2:27 pm

StuG wrote:That has to be one of the longest responses on Tech Report. :o


Depending on how you count, I have written longer posts on the subject:

http://www.techreport.com/forums/viewto ... 9#p1031649

mmmmmdonuts21 wrote:First of all thank you for taking the time and energy to write that post Shining Arcanine. That took me 25 minutes to read and understand. I found it extremely useful and informative.

So I think my best bet than is to run a full linux distribution for the server then.

Just curious why you wanted an Ion system. You could grab a atom D510 board for half that price.


I originally grabbed this board about a year ago when it was being used for a HTPC and the Ion was extremely helpful in the graphics department.

Also, I have been instructing people to stay away from Western Digital Green Drives


I have not had any issues yet and I got these two drives a while back so I will be sure to keep a careful eye on them and probably purchase different HDDs next time.

Shining Arcanine:

I will definitely check out Gentoo. It looks promising. I may run it in a VM first for a little bit just to get a hang of how the system handles.
I don't need to do a headless installation at all, but I would really like to run it headless after I got the system up and running. I believe that would be much easier to do.


I started with Gentoo Linux last September by installing it in virtual machines and have learned everything that I know about Linux (and UNIX systems in general) between then and now. Up until that time, I had never been impressed by any Linux distribution I had tried in virtual machines, because their virtualized performance was awful, but Gentoo Linux's performance impressed me, so I spent some of my spare time experimenting with it in virtual machines after I first tried it. Then last January, I migrated from Windows 7 to Gentoo Linux on my laptop. Last May, I switched from Windows 7 to Gentoo Linux on my desktop as well. I plan to build my a file server that will run Gentoo Linux after I rewire my house with Ethernet at some point later this year, so I am in a similar situation. My plans are to have it function as a private rsync mirror, store backups of my systems' SSDs and also store other stuff that I do not have room to keep on my systems' SSDs.

Since I assume that you plan to run Gentoo Linux inside a virtual machine on Windows, I recommend that you try using VMware Player. When I used to run Windows 7, I found that virtualization on Windows worked the best with VMware Player. It was actually my first choice when doing virtualization because I had read good things about it and that worked out fairly well in my opinion.

You might be interested in a tweak that Damage did to enable Time-Limited Error Recovery on his Western Digital drives:

http://techreport.com/discussions.x/19491

While I do not know what Damage did, someone here might know. I am sure if you send him a private message asking how he did it, he would share. I assume it involves some sort of offline utility that is designed to tweak hard drive parameters like Automated Acoustic Management, but I have not done anything with anything like that in years, so I am out of the loop. I would likely be interested in knowing what he did when I build my own file server later this year.

bitvector wrote:
Shining Arcanine wrote:You should not use a USB key as your OS drive because USB keys are raw flash devices and as far as I know
They're not raw flash. Raw flash exposes a read/erase/write interface, whereas a USB key "looks" like any other USB mass storage device. I'm not suggesting using them as an OS drive either, but for different reasons.


The USB keys are raw flash devices in a way. I have read about people who installed full blown Windows installations on USB keys only to have them die within a week. There are no guarantee of wear leveling with USB keys, so it is best to assume that there is none. As long as you are not running an OS on a USB key and are using it for moving the occasional file around, you should be fine, because even if files are written to the same physical location in the flash when you write files to it, the cells have >10,000 write cycles such that you will never kill a USB key in ordinary usage.

bitvector wrote:
mmmmmdonuts21 wrote:I will definitely check out Gentoo.
If you want your file server to be stable and not a lot of continuous upkeep and maintenance work, I'd suggest using CentOS or Debian stable (possibly plus the backports repository) or Ubuntu LTS rather than Gentoo.


Gentoo Linux is not much work in terms of upkeep if you know what you are doing. The work involving most upgrades is done for you by the package manager. The package manager must be manually told to perform upgrades and while there are occasional hiccups, you can expect hiccups from many other Linux distributions. The only Linux distributions that are free of such hiccups are ones whose components are allowed to become incredibly out-dated, to the point where they begin to lose their usefulness. As I pointed out in my above post, the kernel on CentOS is roughly 3 years old, so it might not have driver support for his TV tuner. Debian would be better in this regard, because it was released as Debian Lenny in Feburary 2009. The current stable kernel is 2.6.35.x while the kernel in Debian Lenny is version 2.6.26, which is fairly old. ext4 support in Linux had experimental status until kernel version 2.6.28, so using Debian Lenny will require that the original poster use ext3. There are other options on Linux, but ext2, ext3 and ext4 are the filesystems typically used on Linux distributions. Being stuck with ext3 would mean that **** following power outages will take extremely long periods of time. It also will mean that several other modern features will not be available, although their importance will likely not be as great as the improved **** times. For anyone unaware of what **** is, it is the UNIX equivalent to Windows' chkdsk.

An advantage of Gentoo Linux is that it can support any hardware that any other Linux distribution can support and while other distributions can wait months or even years before they receive new hardware support from new code that has entered the Linux kernel, Gentoo Linux is one of the first to receive the new hardware support, because it always has an up to date kernel available (specifically, one that is typically made on the end-user's system). It is also an ideal distribution for learning how to do things. No mainstream distribution (other than itself) can touch it in that respect.

The only area in which Gentoo Linux requires more maintenance than other Linux distributions is the kernel. While the package manager can be configured to install new kernel sources for you, it does not do kernel upgrades for you. Gentoo Linux gives users two options for installing kernels. The first requires that users create a .config file, which can be done with a graphical utility that the kernel provides that is accessible with the command "make menuconfig" inside the kernel source directory. There is an extensive guide on what to do inside of "make menuconfig" at www.kernel-seeds.org, which was created by a Gentoo Linux user, and it is a good complement to the section in the Gentoo manual that describing the procedure for manually installing your own kernel. After the .config file is created, you need to compile the kernel, copy the new kernel image to your boot directory/partition, install the kernel modules, install any firmware that was not bundled with the kernel binary (I think keeping it separate is best), modify your system's boot loader to specify the location of the new kernel, update the /usr/src/linux symbolic link to point to the new kernel source directory and recompile any external kernel modules that were not compiled as part of the kernel tree. The second option is to use genkernel, which creates a generic kernel like the one used on LiveCDs. Both options are described in the Gentoo Manual, although some of the extra steps I mentioned above are only done when upgrading your kernel, and they are mentioned in a separate kernel upgrade guide:

http://www.gentoo.org/doc/en/kernel-upgrade.xml

The first procedure is best, because it allows you to produce a kernel that is optimized for your system's intended application and hardware configuration. Gentoo Linux is the only mainstream Linux distribution that supports per system kernel customization. All other Linux distributions strongly discourage it. Kernel upgrades are a bit more involved than other system software upgrades, but they can be done relatively quickly if you know what you are doing. If you have an .config file from an older kernel, you can usually use it with a newer kernel by running "make oldconfig", which will then ask you a series of questions regarding what you want to do about new options that were not present when you configured your previous kernel with default recommendations for each one. While I think there is an option that can automate even that (such that all defaults are selected), going through that does not take much time and it is usually a good idea in case any of the changes pertain to your system. I usually run a command like "KERNEL=2.6.35.3; cd /usr/src/linux-2.6.35.3 && cp ../linux/.config . && eselect kernel set linux-$KERNEL && make -j5 oldconfig && make -j5 && make -j5 modules_install && make -j5 firmware_install && cp arch/x86_64/boot/bzImage /boot/kernel-$KERNEL && module-rebuild rebuild && nano -w /etc/grub/grub.conf", which semi-automates the process for me. That command is custom tailored to my system (i.e. quadcore processor, vanilla-sources, grub boot loader), but it shows basically everything that needs to be done. The original poster would likely want to use the gentoo-sources to build his kernel, because kernels built using the vanilla sources are not supported by Gentoo Linux, although usually you can get help using them anyway, because the patches that Gentoo Linux does to the kernel sources have such rare applications that the two are typically the same thing anyway.

Kernel installations and upgrades do not need to be done very often, but the fact that Gentoo Linux allows users to take full control of the process is one of the reasons why people learn more from Gentoo Linux than they do from other Linux distributions.

Edit: Apparently, **** is considered an expletive by TR's word filter. **** stands for File System ChecK. You can get the wikipedia article on it via Google:

http://lmgtfy.com/?q=file+system+check

I sent Kevin a private message explaining the issue, so hopefully it will be removed from TR's word filter soon. It is annoying to see a bunch of asterisks in place of ****.
Disclaimer: I over-analyze everything, so try not to be offended if I over-analyze something you wrote.
Shining Arcanine
Gerbil Jedi
 
Posts: 1717
Joined: Wed Jun 11, 2003 11:30 am

Re: Building a File Server

Postposted on Thu Aug 26, 2010 3:29 pm

Shining Arcanine wrote:As I pointed out in my above post, the kernel on CentOS is roughly 3 years old, so it might not have driver support for his TV tuner.
And you're wrong about the age because you're just looking at the base number. The enterprise kernels start out with an older, stable base version but they are patched with literally thousands of patches backported from recent kernels, including drivers. For example, you can see in their release notes lines like "netxen: driver updates from 2.6.31 and 2.6.32 BZ#516833," despite the fact that the base version remains at 2.6.18. Now, as far as TV tuners go in particular, RHEL removes some drivers they don't want to support (such as Firewire and some TV capture cards), but CentOS often puts them back in their CentOS plus kernels.

Shining Arcanine wrote:Debian would be better in this regard, because it was released as Debian Lenny in Feburary 2009. The current stable kernel is 2.6.35.x while the kernel in Debian Lenny is version 2.6.26, which is fairly old. ext4 support in Linux had experimental status until kernel version 2.6.28, so using Debian Lenny will require that the original poster use ext3.
No, the lenny backports.org kernel is 2.6.32. My fileserver runs lenny with ext4.
bitvector
Grand Gerbil Poohbah
 
Posts: 3234
Joined: Wed Jun 22, 2005 4:39 pm
Location: Mountain View, CA

Re: Building a File Server

Postposted on Thu Aug 26, 2010 3:31 pm

wibeasley wrote:
mmmmmdonuts21 wrote:I have been running WHS version 1 and then later the next WHS Vail beta for about a year now but I was looking to try something different as I have been using more and more of Ubuntu. I really like WHS but I wanted to test some alternatives for the time being.
sid1089 wrote:Also you might want to consider Windows Home Server.


haha. I was testing the forum on my new iPad when I was making that post.
Carpe diem quam minimum credula postero
sid1089
Gerbil Team Leader
 
Posts: 290
Joined: Wed Jul 26, 2006 4:56 am

Re: Building a File Server

Postposted on Thu Aug 26, 2010 3:44 pm

bitvector wrote:
Shining Arcanine wrote:As I pointed out in my above post, the kernel on CentOS is roughly 3 years old, so it might not have driver support for his TV tuner.
And you're wrong about the age because you're just looking at the base number. The enterprise kernels start out with an older, stable base version but they are patched with literally thousands of patches backported from recent kernels, including drivers. For example, you can see in their release notes lines like "netxen: driver updates from 2.6.31 and 2.6.32 BZ#516833," despite the fact that the base version remains at 2.6.18. Now, as far as TV tuners go in particular, RHEL removes some drivers they don't want to support (such as Firewire and some TV capture cards), but CentOS often puts them back in their CentOS plus kernels.

Shining Arcanine wrote:Debian would be better in this regard, because it was released as Debian Lenny in Feburary 2009. The current stable kernel is 2.6.35.x while the kernel in Debian Lenny is version 2.6.26, which is fairly old. ext4 support in Linux had experimental status until kernel version 2.6.28, so using Debian Lenny will require that the original poster use ext3.
No, the lenny backports.org kernel is 2.6.32. My fileserver runs lenny with ext4.


There is a limit to the amount of stuff that is backported. There are literally thousands of patches between two kernel versions (e.g. 2.6.32 and 2.6.33). If RedHat Enterprise Linux has backported thousands as well, then that means that they have only backported a small fraction of the number of patches available and if they did backport all of the patches available, then they might as well upgrade people to the latest kernel.

Anyway, the important question here is, if RedHat Enterprise Linux refused to backport TV tuner drivers that were introduced after Linux Kernel 2.6.18, would CentOS backport the drivers to their CentOS Plus kernel and will they support their CentOS Plus kernel? Another important question is if the 2.6.32 kernel is available via the lenny backports, is using it on a production system supported by the Debian project?

Note that when I say that software is "supported" I mean that they will not tell you to run a supported version if you ask for help with issues you encounter and will instead try to help you in some way.
Disclaimer: I over-analyze everything, so try not to be offended if I over-analyze something you wrote.
Shining Arcanine
Gerbil Jedi
 
Posts: 1717
Joined: Wed Jun 11, 2003 11:30 am

Re: Building a File Server

Postposted on Fri Aug 27, 2010 6:36 am

Shining Arcanine wrote:
bitvector wrote:
mmmmmdonuts21 wrote:I will definitely check out Gentoo.
If you want your file server to be stable and not a lot of continuous upkeep and maintenance work, I'd suggest using CentOS or Debian stable (possibly plus the backports repository) or Ubuntu LTS rather than Gentoo.

Gentoo Linux is not much work in terms of upkeep if you know what you are doing. ...

The issue is, he's just now learning what he is doing. If his goal is to learn the ins and outs of a niche distro like Gentoo, and get a working file server out of it as a side benefit at the end, then great. If his goal is to learn a mainstream distro (and gain some skills which are more likely to be useful out in the "real world"), and get his Linux server up and running relatively quickly, then Gentoo would not be my first choice.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37493
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Building a File Server

Postposted on Fri Aug 27, 2010 7:12 am

Thanks for all the further suggestions. The good part of my day yesterday was spent playing with different VM machines (fairly new at this as well) with different builds of Linux (CentOS, Debian, Gentoo). I spent the most time on trying to setup Gentoo (still not complete) and I am learning a lot in trying to do that. I guess my goal for today is to continue testing different varieties to try and see which one I like the best and how to integrate a VM Linux distribution with an actual Windows share and vice versa. Thanks again for all the help.
To Start Press Any Key'. Where's the ANY key?
If something's hard to do, then it's not worth doing
You know, boys, a nuclear reactor is a lot like a woman. You just have to read the manual and press the right buttons.
mmmmmdonuts21
Gerbil Elite
 
Posts: 590
Joined: Wed Jul 16, 2008 9:09 am

Re: Building a File Server

Postposted on Fri Aug 27, 2010 10:21 am

just brew it! wrote:The issue is, he's just now learning what he is doing. If his goal is to learn the ins and outs of a niche distro like Gentoo, and get a working file server out of it as a side benefit at the end, then great. If his goal is to learn a mainstream distro (and gain some skills which are more likely to be useful out in the "real world"), and get his Linux server up and running relatively quickly, then Gentoo would not be my first choice.


I think you are a bit confused, as Gentoo Linux is not a niche distribution. It is not only a mainstream distribution, but it is considered a "major" distribution by distrowatch.com:

http://distrowatch.com/dwres.php?resource=major

I believe that all of the distributions I recommended are major distributions according to that list.

Furthermore, I did say that Gentoo Linux would require the most time to install. While only like 10% of the installation time will require he do something, Gentoo Linux will require that he learn how things work during that 10%. If he wants a file server, he will need to learn how to do things and Gentoo Linux is an excellent teacher because it teaches people to do things on UNIX systems both well and quickly.

mmmmmdonuts21 wrote:Thanks for all the further suggestions. The good part of my day yesterday was spent playing with different VM machines (fairly new at this as well) with different builds of Linux (CentOS, Debian, Gentoo). I spent the most time on trying to setup Gentoo (still not complete) and I am learning a lot in trying to do that. I guess my goal for today is to continue testing different varieties to try and see which one I like the best and how to integrate a VM Linux distribution with an actual Windows share and vice versa. Thanks again for all the help.


I have a bit more advice. It is probably better to install Gentoo Linux with System Rescue CD. It is a fork of Gentoo Linux that exists only as a LiveCD and most Gentoo Linux users prefer it to the official LiveCD. You can substitute it for the official live CD without having to deviate from the manual:

http://www.sysresccd.org/Main_Page

Also, finding guides for anything on Gentoo Linux typically only requires googling "<topic> gentoo", the answer usually appears in the first results:

http://www.gentoo.org/doc/en/quick-samba-howto.xml
https://trac.transmissionbt.com/wiki/He ... sageGentoo
http://en.gentoo-wiki.com/wiki/Pure-ftpd
http://en.gentoo-wiki.com/wiki/MythTV

Gentoo-wiki.com is not an official site, but it was made by a Gentoo Linux user and it is a good supplement to the official manuals. For running Gentoo Linux in a vmware virtual machine, you will likely want to read the following guide:

http://en.gentoo-wiki.com/wiki/VMware_Guest

The documentation at kernel-seeds.org probably could have helped you find the kernel options that you had to enable, but that is much more explicit. Keep in mind that in "make menuconfig", you can do text searches by hitting the '/' key, which helps in finding configuration options. The other option as I said in another post would be to use genkernel, which should just work out of the box, but while it reduces the time you need to get things running, it is not nearly as educational.

If you have any problems installing Gentoo, feel free to ask here, on the Gentoo forums or in the #gentoo IRC channel at chat.freenode.net. There are plenty of people that are willing to help.
Last edited by Shining Arcanine on Fri Aug 27, 2010 10:30 am, edited 1 time in total.
Disclaimer: I over-analyze everything, so try not to be offended if I over-analyze something you wrote.
Shining Arcanine
Gerbil Jedi
 
Posts: 1717
Joined: Wed Jun 11, 2003 11:30 am

Re: Building a File Server

Postposted on Fri Aug 27, 2010 10:29 am

just brew it! wrote:[The issue is, he's just now learning what he is doing. If his goal is to learn the ins and outs of a niche distro like Gentoo, and get a working file server out of it as a side benefit at the end, then great. If his goal is to learn a mainstream distro (and gain some skills which are more likely to be useful out in the "real world"), and get his Linux server up and running relatively quickly, then Gentoo would not be my first choice.

Furthermore, even if you know what you're doing, it's more work. I know what I'm doing, and yet I don't care to spend my time shepherding a system through things that could be handled for me (or waiting for updates to compile so I can perform whatever post-install tasks are required).

Slackware is another minimalist distro with a tinkerer approach like Gentoo. I used Slackware from 1995 until about 2001; when I switched to Debian, it was a revelation -- I realized how much of my own time I'd been wasting compiling tarballs and doing things by hand that could have been automated. And the tasks that I wasted the most time didn't really even transfer into learning anything useful that I'd ever have to do again with more automated, non-minimalist distros.
bitvector
Grand Gerbil Poohbah
 
Posts: 3234
Joined: Wed Jun 22, 2005 4:39 pm
Location: Mountain View, CA

Re: Building a File Server

Postposted on Fri Aug 27, 2010 10:33 am

bitvector wrote:
just brew it! wrote:[The issue is, he's just now learning what he is doing. If his goal is to learn the ins and outs of a niche distro like Gentoo, and get a working file server out of it as a side benefit at the end, then great. If his goal is to learn a mainstream distro (and gain some skills which are more likely to be useful out in the "real world"), and get his Linux server up and running relatively quickly, then Gentoo would not be my first choice.

Furthermore, even if you know what you're doing, it's more work. I know what I'm doing, and yet I don't care to spend my time shepherding a system through things that could be handled for me (or waiting for updates to compile so I can perform whatever post-install tasks are required).

Slackware is another minimalist distro with a tinkerer approach like Gentoo. I used Slackware from 1995 until about 2001; when I switched to Debian, it was a revelation -- I realized how much of my own time I'd been wasting compiling tarballs and doing things by hand that could have been automated. And the tasks that I wasted the most time didn't really even transfer into learning anything useful that I'd ever have to do again with more automated, non-minimalist distros.


Slackware lacks dependency resolution. Gentoo Linux has dependency resolution. It also handles compilation of tarballs for you. It is much easier to use than Slackware and automates a great deal of the time that is wasted on slackware systems.

Slackware is another distribution that is highly educational and I had forgotten about it. It and Gentoo Linux are probably close to each other in terms of being educational, although Gentoo Linux is definitely the easier of the two.
Disclaimer: I over-analyze everything, so try not to be offended if I over-analyze something you wrote.
Shining Arcanine
Gerbil Jedi
 
Posts: 1717
Joined: Wed Jun 11, 2003 11:30 am

Re: Building a File Server

Postposted on Fri Aug 27, 2010 10:48 am

Shining Arcanine wrote:I think you are a bit confused, as Gentoo Linux is not a niche distribution. It is not only a mainstream distribution, but it is considered a "major" distribution by distrowatch.com:

http://distrowatch.com/dwres.php?resource=major

I believe that all of the distributions I recommended are major distributions according to that list.

That list also has this to say about Gentoo:
Cons: Occasional instability and risk of breakdown, the project suffers from lack of directions and frequent infighting between its developers

:roll:

Also, if you go to this page on their site, their own statistics contradict the claim that Gentoo is one of the 10 most popular distros. Depending on how long a period of time you average over, it ranks anywhere from 17th to 24th. Even if we're generous and use the 17th place ranking, I'd say that still counts as "niche" by most definitions of the word.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37493
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Building a File Server

Postposted on Fri Aug 27, 2010 10:50 am

just brew it! wrote:
Shining Arcanine wrote:I think you are a bit confused, as Gentoo Linux is not a niche distribution. It is not only a mainstream distribution, but it is considered a "major" distribution by distrowatch.com:

http://distrowatch.com/dwres.php?resource=major

I believe that all of the distributions I recommended are major distributions according to that list.

That list also has this to say about Gentoo:
Cons: Occasional instability and risk of breakdown, the project suffers from lack of directions and frequent infighting between its developers

:roll:

Also, if you go to this page on their site, their own statistics contradict the claim that Gentoo is one of the 10 most popular distros. Depending on how long a period of time you average over, it ranks anywhere from 17th to 24th.


Wikipedia's article on Linux distribution talks about that and says that that is done on page views, which is no secret in the Linux community:

http://en.wikipedia.org/wiki/Linux_dist ... tributions

The page hits on distrowatch are by no means an indicator of install base or users managing that install base.

Furthermore, distrowatch.com claims that Gentoo Linux is one of the ten major distributions, not one of the ten most popular distributions. While Gentoo Linux certainly might be one of the top 10 most popular distributions, it could very well depend on how you count. Do you count by installations? Do you count by actual users managing those installations? Do you count installations of derivative distributions as being installations of the base system? Gentoo Linux is one of the most influential Linux distributions of all time. Patches are made to a multitude of open source projects by Gentoo Linux users who notice that things are wrong. Parts of Gentoo Linux are reused by other distributions, with its eselect utility having found it way into Ubuntu Linux. Gentoo Linux is the root distribution of a tree of child distributions, which is a distinction only 5 Linux distributions have:

http://en.wikipedia.org/wiki/List_of_Li ... ntoo-based
Last edited by Shining Arcanine on Fri Aug 27, 2010 11:06 am, edited 1 time in total.
Disclaimer: I over-analyze everything, so try not to be offended if I over-analyze something you wrote.
Shining Arcanine
Gerbil Jedi
 
Posts: 1717
Joined: Wed Jun 11, 2003 11:30 am

Re: Building a File Server

Postposted on Fri Aug 27, 2010 11:02 am

Shining Arcanine wrote:Furthermore, my claim is that Gentoo Linux is a mainstream distribution

Well, to be very blunt, I disagree. If you can provide any credible data which shows that it has more than oh... say... 2% of the Linux market, maybe I'll change my mind.

You are the only person I know who actually runs Gentoo as their primary OS. I think that says something. :wink:
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37493
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Building a File Server

Postposted on Fri Aug 27, 2010 11:13 am

just brew it! wrote:
Shining Arcanine wrote:Furthermore, my claim is that Gentoo Linux is a mainstream distribution

Well, to be very blunt, I disagree. If you can provide any credible data which shows that it has more than oh... say... 2% of the Linux market, maybe I'll change my mind.

You are the only person I know who actually runs Gentoo as their primary OS. I think that says something. :wink:


I edited my post above to be more clear. Anyway, would you split this discussion into another thread? This has little to do with the original poster's decision to build a file server.

By the way, Titan uses Gentoo Linux too:

http://www.techreport.com/forums/viewto ... 4#p1004304

While mindshare in a Windows dominated community is hardly indicative of Linux marketshare in general, I can make broad generalizations about the Linux community based off mindshare in a Windows dominated community too. There are less than 20 people on these forums that even use Linux while there are two Gentoo Linux users on it. That gives Gentoo Linux at least a 10% marketshare.
Disclaimer: I over-analyze everything, so try not to be offended if I over-analyze something you wrote.
Shining Arcanine
Gerbil Jedi
 
Posts: 1717
Joined: Wed Jun 11, 2003 11:30 am

Re: Building a File Server

Postposted on Fri Aug 27, 2010 11:51 am

Shining Arcanine wrote:I edited my post above to be more clear. Anyway, would you split this discussion into another thread? This has little to do with the original poster's decision to build a file server.

The OP was also asking for advice on what OS to use, so IMO the tangent was still (marginally) relevant. Unless you feel a need to continue (I don't), I don't think a split is really warranted.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37493
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Building a File Server

Postposted on Fri Aug 27, 2010 7:03 pm

I favor Ubuntu for ease of use.
JustAnEngineer
Gerbil God
Gold subscriber
 
 
Posts: 15331
Joined: Sat Jan 26, 2002 7:00 pm
Location: The Heart of Dixie

Re: Building a File Server

Postposted on Fri Aug 27, 2010 7:24 pm

JustAnEngineer wrote:I favor Ubuntu for ease of use.


Ubuntu's strengths do not extend to the realm of headless file servers.
Disclaimer: I over-analyze everything, so try not to be offended if I over-analyze something you wrote.
Shining Arcanine
Gerbil Jedi
 
Posts: 1717
Joined: Wed Jun 11, 2003 11:30 am

Re: Building a File Server

Postposted on Fri Aug 27, 2010 7:29 pm

Shining Arcanine wrote:
JustAnEngineer wrote:I favor Ubuntu for ease of use.


Ubuntu's strengths do not extend to the realm of headless file servers.
I have to disagree, just install the Ubuntu Server and it will run very well as a headless file server. I also like Debian and run that on my server which is headless and sitting in the basement.
notfred
Grand Gerbil Poohbah
 
Posts: 3712
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: Building a File Server

Postposted on Fri Aug 27, 2010 7:30 pm

Ubuntu does seem to be the flavour of the month and is what I use myself. There is also a server edition put out by Canonical. But don't forget nice distros like OpenSUSE and Fedora. And if you want to dig deeper then check out Distrowatch which also includes the stats on the BSD projects.
FDISK /MBR
Dirge
Gerbil Jedi
 
Posts: 1543
Joined: Thu Feb 19, 2004 3:08 am
Location: New Zealand

Re: Building a File Server

Postposted on Fri Aug 27, 2010 7:31 pm

I favor Ubuntu for ease of use.


After extensive testing of four flavors of Linux today (Gentoo, CentOS, Debian, and Ubuntu) I decided I am going to probably go with Ubuntu for the reason JustanEngineer just stated. Right now I don't have the time to learn Gentoo and I liked Debian and CentOS but ultimately in the end I was more familiar with Ubuntu and its interface. I have used the desktop variant of Ubuntu for a while now and have gotten used to it and like it a lot. The latest release also being a LTS is also a great thing in my mind for stability.

Now for testing purposes I was using the server edition. Is this the edition I should be using? I am relatively new to the command line in Linux but I am fairly comfortable using it in Windows so should I just go ahead and configure the server without a GUI? I also was looking into ebox and SSH and trying to determine the advantages of each to try and access the server remotely. Any other suggestions for using Ubuntu as a server especially as I described my uses in the original post? Thanks again.
To Start Press Any Key'. Where's the ANY key?
If something's hard to do, then it's not worth doing
You know, boys, a nuclear reactor is a lot like a woman. You just have to read the manual and press the right buttons.
mmmmmdonuts21
Gerbil Elite
 
Posts: 590
Joined: Wed Jul 16, 2008 9:09 am

Next

Return to Networking

Who is online

Users browsing this forum: Bing [Bot] and 1 guest