I had heard podcasts and news surrounding the idea of the HP Home server platform. This always seemed like a good idea to me, especially considering the idea around making and restoring backups of data and the operating system install. Either I’m cheap, or perhaps just because I don’t need another windows machine in the house, I decided to setup my own personal Linux server that performed some, if not all, of the functions as the equivalent HP home Server would.
Some of the concepts I’m interested in with a server platform are the following:
- Data Backup
- Database Server
- File Server
- Personal Wiki server
- Test web server
This post will focus on the data backup portion of the home server config. I’ve been interested in configuring a RAID setup ever since I heard the Stack Overflow podcast from Jeff Atwood. My interests are purely for data redundancy at this point, so I researched creating a mirror configuration. Jeff and Joel make an observation that the hardware controllers seem to be the point of failure for hardware RAID controllers due to the fact that hardware guys are not programmers. This made sense to me, so I was set to create a software RAID such that theoretically I utilize the strength of both the hardware guys and the software guys. In order to configure the discs in any configuration I choose, I’d need a hardware / software combination that will accept a JBOD (just a bunch of discs) setup.
I decided to go with the Sans Digital 5 bay external enclosure. Newegg must have had a sale when I bought it, because I remember buying it cheaper than it currently is. ($229) For now, I just purchased 2 1.5TB drives for my RAID mirror, realizing that I have 3 bays that I can fill with more drives later. A quick note about this enclosure: while I eventually got it to work well, apparently the drives need to be inserted into the enclosure starting at the bottom, rather than at the top. None of the instructions that came with the device mentioned this, only one diagram of installation pointed toward the bottom of the enclosure, so I decided to give it a try, and it worked. (After I had had the drives inserted into the top of the enclosure for about a week, and asked a question on ServerFault trying to get either of them detected by the operating system)
The Sans Digital enclosure came with a PCI-express card to plug into the motherboard, that would allow the enclosure to connect to it via an eSata connection. This card supported port multipliers, a concept that I hadn’t heard of before this project, and am now well acquainted with. Since the hardware I was using was not the latest and greatest (read: hodge-podge of old hardware I had lying around) the motherboard in question did NOT have a PCI-express slot, only PCI. So, blindly I thought that all eSata ports were equal, so I bought the least expensive PCI eSata controller card that newegg had. It serves me right, I got what I paid for. Just a quick note to anyone buying this, or any eSata device thinking it’ll work in any eSata port, ensure that you’re computer’s eSata port will support port multipliers. If it doesn’t, as it didn’t in my case, only one device was seen by the operating system. This is fine if the external drive is a single drive, but since I had an enclosure of discs, it would not work. In addition to only one drive being detected, the eSata controller card was detected as the primary drive controller for the computer. So, when Ubuntu booted, it numbered (lettered?) the drives starting with the controller card, rather than with what was connected to the operating system. Since I had only an IDE drive plugged into the computer internally, this was causing issues, which was noted by the quick responders on the server fault link I posted earlier. This didn’t matter as long as the enclosure was turned off during the boot, but if I left it on and restarted Linux, then the enclosure would be the drive where the OS expected to see the main OS files. Since I hadn’t installed the OS to the drives in the enclosure, this caused a bit of a problem with booting Linux, so I bought a 250GB Sata drive from Fry’s to plug into the internal SATA port of the PCI controller card, and mirrored the previous IDE drive onto the new SATA drive, and everything worked fine booting from the new drive.
So, now my problem was getting the PC to recognize the two drives in the enclosure so that they could be mounted as normal. So, back to newegg.com I went. This time I purchased a USB 2.0 PCI controller card, (because the USB ports onboard the motherboard were only USB 1.1) a USB 2.0 to eSata converter, and a new heat sink / CPU fan, because the fan I had on the CPU was the factory fan that came with the CPU, and from the sound of it, it was on it’s last legs. (The sound was similar to a jet during take-off. The noise would subside after a time, but it was obvious that it’s days of operation were numbered.) This new fan I can’t speak highly enough about. The Rosewill RCX-Z100 fan is super quiet. So much so that I hardly even realize that the computer is on. Instead of the jet take-off from before, it’s now just a gentle hum. The USB 2.0 to eSata converter worked like a charm, and now… finally Linux showed me /dev/sdb and /dev/sdc devices that I could configure using fdisk.
RAID Software configuration:
Now that I had my new drives (/dev/sdb and /dev/sdc) detected by the operation system, I ran fdisk to create a single primary partition on each drive. A prerequisite of software mirroring in Linux is the mdadm tool. Simply running apt-get install mdadm as root installed the packages necessary. Now for the creation of the mirror:
mdadm --create /dev/md0 --level=mirror --raid-devices=2 /dev/sdc1 /dev/sde1
Will create the new "hard drive" device that is used to mount the file system. Ensure that the device names on the end match the letter and number of the paritions you created on the individual drives. To view the status and details of the array one can run the following:
mdadm --detail /dev/md0 cat /proc/mdstat
Now that we have a single device created, we can work with it as if it were any other hard drive:
mkfs.ext3 /dev/md0
I added the below to my /etc/fstab file to auto mount this drive on subsequent boots
/dev/md0 /home/raid ext3 relatime 0 2
Alternatively one can mount the drive manually:
mount -t ext3 /dev/md0 /home/raid
So, that's my new data redundancy storage device created with Linux. In addition to the LAMP (Linux, Apache, MySQL, PHP) stack already in place, this system is quite nice as a Linux home server. Granted, I don't have automatic backups at the operating system level, so that restoring takes less than an hour, but at least I have a file, ftp, ssh, print, and SVN server already setup.
Oddly, one of my biggest problems with Linux is that it works too well. Once I have it setup, I rarely if ever have to mess with it, so I forget how to do everything. Hopefully this blog, and adding content to my mediawiki page that I have will help going forward.