I seem to be sinking under a pile of spare hard drives at the moment - typically 2.5" 500GB ones. It would be nice to find a way of making use of them *cheaply*. It would be nice to build a NAS platform for use as a backup repository, and for perhaps archiving stuff like films.
Performance is not that critical, but I would like fault tolerance. Not too fussed about uptime. So it needs to be a RAID setup of some sort that can survive any individual drive failure (e.g. RAID 5 or 6), but it can be shutdown for maintenance etc without any worries - so I don't need to worry about hot swap or redundant components.
A small low power mobo in an old PC case could be a starting point, or for that matter, even as RaspPi 2 B or similar level single board computer, but that will soon run out of sata ports (or not have any to start with). One option that springs to mind would be a powered USB hub, and a bunch of drive caddies which would be a cheap way of adding lots of drives if required.
That then raises the question of software to drive it... How workable would the various MD style RAID admin tools and file systems be at coping with drives mounted on mixed hardware interfaces - say a mix of SATA and USB? Has anyone tried multiple budget SATA cards on stock PC hardware?
My immediate thought is that, by the time you've bought the hardware needed to achieve this (caddies, etc. especially, as you mention them) it would be cheaper to buy a few big drives. Buy just three and you can do RAID5 (=fault tolerance).
Look for a cheap NAS on fleabay, or stick then in a cheap case. Use a separate small drive or SSD for the OS, then you can upgrade/maintain that without affecting the data on the RAIDed disks. I take them out of my Microserver when doing OS upgrades, etc. so no "accident" can befall them.
Yes and no (but probably more no as a solution for you). ;-)
My current server is an Dual Core Atom running 3 x 500G 2.5" SATA drives on Windows Home Server V1. Two SATA are on board and there are another 3 on a PCI card (1 used).
They used a technique that sounds like it would suit your needs (as it still suits mine) in that you simply add drives to a pool (it's easy to do because it's Windows ) and they can be any size or interface. So, an old Mobo with 4 SATA and 2 PATA ports could use 6 drives straight away. It was called Drive Extender and the good thing was the drives were just running straight NTFS and so each could be read independently if required (unlike a single drive from a RAID 5 array)
As soon as the system detects a new drive it asks you how you want to use it, either by adding it to the / a pool or as a backup drive. If you have an existing pool of say 1.5TB and then you add another 500G drive your pool then becomes 2TB.
Data redundancy is provided by folder mirroring where a mirrored folder will be mirrored across two separate drives.
If a drive starts to play up (or you want to replace it with a bigger one), you just remove it from the pool (all the data will automatically be migrated to the remaining drives), you take it out, fit the replacement and join it back into the pool.
Mine has been running every day for 1812 days:
formatting link
However, it is woken up by the first (Windows / Mac) client that turns on and goes to sleep after the last client has shut down, assuming there is no ongoing network or CPU activity above a certain threshold or an unfinished torrent etc.
It also backs up all the Windows clients every day to the point where if a client had drive fails catastrophically, I can replace the drive and re-image the machine over the LAN in less than an hour and a few clicks (and have done so a couple of times so far).
Or you can browse the list of backup available for each machine and get a single file from say six months previous, depending on your backup settings.
The backup system knows when it's backing up identical files from different machines so only stores them once.
MS dropped WHS V1 (based on WS 2005) for WHS V2011 (based on WS 2008) but you can still find copies on the likes of eBay.
I initially tried making a Linux server but gave up long before I even considered the automatic (image) backup or all of the other features that were so easy for me to install on WHS. I just wanted a NAS / file server, not a new geeky hobby and was happy to pay the 45 or so quid for the privilege.
Cheers, T i m
p.s. You can still get drive pooling on other Windows solutions (I don't know which, I'm not a Windows fanatic) and you can also get independent solutions like Drive Bender:
I can get basic USB2 caddies for about £3.50, so say £25 for 3 gig. For the drives you shove in the main box, one does not even need to use the box that comes with them, just the SATA header.
I have about 15 gigs worth of new 500G 2.5" drives, and perhaps 3 or 4 gigs worth of 3.5" ones. That won't buy you that much time if recording all the muxes - especially the HD ones.
That's apparently single-figures TB but the tricky thing is you need at least 6 tuners for all the muxes.
As to the HDD stash, you can probably cut costs with a homemade mounting system. For SSDs (light and no vibration issues) I just self-adhesive velcro them inside the case somewhere. HDDs want a bit more sturdy mountings and ventilation. I'm sure John could knock up a rack system with a few bits of wood and maybe a nice fan at the back.
For connections, you can do one of: A pile of USB-SATA adaptors (and a hub maybe) A multi-port SATA card A SATA port multiplier
This is the first port multiplier hit on ebay, for instance:
formatting link
(no idea if it's any good)
If you want to rack them up, you can use an ex-server backplane board:
formatting link
(I haven't tried this one but there are standard SATA connectors on the back) and make some kind of rails to slide the drives in and out.
There are also ex-server port multiplier backplanes about if you can find one:
formatting link
(most recent sale was $16 on ebay.com)
You can probably knock something up for a fraction of the price of a multi-bay commercial JBOD.
As for setting up the array, I suspect MD/ZFS/whatever will cope fine with discs on divergent interfaces. Obviously grinding everything over USB 2 to a Raspberry Pi is not going to be exactly fast, but I can't see any showstoppers.
Yes, that's a though - a port multiplier is something I had not considered... a 5 to 1 mux on a six port mobo would swallow a load of drives. You can get loads of 2.5" 7mm drives into the space of mini tower PC ;-)
Indeed, but its not primary storage, so it does not need to be that quick (although quick enough to stream video would be handy)
Linux MD (RAID) does not care what the interfaces are - it works at the block device level, so as long as your devices appear, it will work find.
But of course you are going to have to balance the sizes for RAID1,5,6 which might not be totally convenient. MD RAIN can work with partitions (it's just another block device) so that is one get out.
However, ZFS might be a more interesting choice.
For something like this, I would (and do) use Debian. ZFS is pretty solid on Debian (but is not "native: you need to include an extra repository
Just to point out (and I'm sure Tim knows this anyway), you don't normally use MD/LVM with ZFS - it has its own volume management integrated in. You can mix disk sizes in zpool, but you ideally use the same size in each top level vdev (stripe), as within a top level vdev, it will treat all disks as though they are the size of the smallest one.
Watching with interest as I am slowly collecting 2.5" SATA HDDs as I upgrade to SSDs.
I use to fit two 2.5" drives into a 3.5" hole in a chassis.
A recommendation for a SATA3 add on card for PCIe would be good, as I am running out of SATA3 ports on my main chassis.
For an old box, I assume SATA 2 would be sufficient because the bus wouldn't provide SATA3 transfer speeds anyway (I have some recollection of being told this on uk.comp.homebuilt some time back).
ZFS is your friend ! Here's what I'd do in your situation:
Supermicro Atom (Rangely) board - with plenty of ECC RAM and lots of SATA ports FreeNAS on a USB stick All the drives in a massive ZFS pool with redundancy.
Supermicro boards aren't cheap but they will serve you well. Don't be tempted to skip on ECC RAM though.
HomeOwnersHub website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.