(Slightly) OT:Well chuffed with myself

After finding we're relying on my (Linux based) media server for more and more (have nearly all our CD collection on it, plus downloads, videos, and photos), it seemed a good idea to have some backup.

Currently we've got about 300Gb on a 1TB USB drive used up.

So I bought another (identical, although it wasn't a requirement) 1TB USB drive, plugged it in, and after reading up on the mdadm software RAID management tool, I managed:

1) to create a RAID1 array, with the new disk, telling mdadm that the 2nd disk was "missing" 2) Copy data from existing disk to RAID array 3) Wipe partitions on old 1TB disk 4) Add old 1TB disk to RAID array

There was a slight warm feeling of satisfaction, when I saw the mdadm monitoring pick up the addition of the disk, and start rebuilding the array. And a quick test shows that even while rebuilding the array, the media server can happily pump out an HDTV file with no problem for the media player.

The only wrinkle is that it's not hot-swappable RAID AFAICS. If a disk goes belly up, I'd have to manually partition and add the new disk to keep things going. Still, not a bad result ...

Reply to
Jethro
Loading thread data ...

Why would you want to use RAID1 as opposed to a backup for home use?

Reply to
brass monkey

That occurred to me, too.

I do nightly incremental backups..the disadvantage is a main disk crash could lose me a days data..the advantage is if I wipe my whole database accidentally, I have up to 24 hours to get it back..

Reply to
The Natural Philosopher

RAID is NOT backup...

As another poster has already indicated, should you accidentally delete or otherwise screw up the data, the damage is instantly on the other drive also, so no protection. RAID1 does protect from a disk failure however.

'Backup' should offer point-in-time recovery, and ideally be located elsewhere to provide fire and burglary protection too. Loads of ways of achieving this - which has been covered several times in the past.

Alan (who works in enterprise data backup - but also applies the principles to my own data).

Reply to
AlanD

More to the point (unless I'm missing something here) why would he go for USB? Why not a NAS drive that sits on the network and can stream or serve to any device on the network (simultaneously if required) and is also available over the internet?

Reply to
John

well, it is available over the internet. It's evolved, and I'm too tight to get a separate NAS system.

Reply to
Jethro

I appreciate the distinction between backup and protection from failure. And true, if there should be a fault in the RAID software, I could be looking at 2 paperweights ... I could have just as easily setup a backup job to the new drive, running once a day, and hell, could have put the new drive on another machine off site.

It was more a learning exercise in RAID. And it's the first time I have had matching sets of disk drives !

Reply to
Jethro

I use RAID1 for my home server, but that is to ensure I can carry on working if a disk fails, rather than having to drop everything and fix/restore the data at that instant.

For backup, I snapshot the filesystems every midnight, which means I can go back and look at older versions of the files. Yesterday, I looked back through snapshots of a file, and found what I was looking for in an 18 month old snapshot of the file. I have daily snapshots online going back to March 2007, so I can look at the state of the filesystem every day back over the last 4.5 years.

For disaster recovery, I periodically attach a third mirror disk, let it resilver, and then store off-site. I am intending to change that to be an off-site system to which I send the incremental snapshots over ADSL now that ADSL speeds can cope.

Reply to
Andrew Gabriel

I just use "viceversa" periodically to update my NAS to 2 other USB drive enclosures on a different machine.

Reply to
brass monkey

What medium allows you to store such a vast amount of data.

Reply to
Andy Wade

Actually, April 2007 (I bought the system in March 2007).

The disks are 2TB (just upgraded from 1TB, so used capacity is currently just under 1TB).

The snapshots contain only the files that changed (or more accurately, just the blocks that changed as it works at the block level), and the amount that changes each day is tiny compared with the filesystem size.

So looking at the stats, the current home directory filesystem image is

26.3G, but including 4.5 years of daily snapshots, it's 216G. In the early days, daily snapshots typically occupied 3-10Mb, nowadays around 100Mb.

$ zfs list -t all -r export/home NAME USED AVAIL REFER MOUNTPOINT export/home 216G 1.09T 26.3G /export/home SNAPSHOTS: export/home@20070428 10.4M - 2.44G - export/home@20070429 3.93M - 2.45G - export/home@20070430 3.90M - 2.46G - . . . export/home@20110904 132M - 26.2G - export/home@20110905 95.6M - 26.2G - export/home@20110906 89.9M - 26.3G - export/home@20110907 90.4M - 26.3G - export/home@20110908 91.1M - 26.3G - $

To be honest, keeping snapshots that far back is just because I can, and I use it as an example in presentations I give. For most cases, keeping them back a few months would be more than enough, and I think retrieving an 18 month old version of a file is the furthest back I've gone so far. (It was a configuration file for an application which I'd had to make a temporary change to, and had then forgotten about, so when I noticed this, I searched back for the original configuration file.)

One of my colleagues with school-age children uses a similar scheme for their files, but snapshots every minute. Then, when one of the kids says "Daddy, I just lost my homework file", it's back within seconds.

I have separate filesystems for things like music and photos, because they tend to only grow with new files over time, and I almost never modify or delete anything. That makes their daily snapshot sizes normally zero and I don't keep them for more than a year (the snapshots in those cases really only serve to recover a file I accidentally delete/overwrite, and if I don't notice that for a year, I probably never will).

Reply to
Andrew Gabriel

Hmmm, not a great preparation for the real world ...

Well this RAID is intended *solely* for media, so any deletions will be occasional and deliberate (or so I am gambling). However, with the price of disk drives coming down all the time (I paid £60 last year for the same 1TB drive I paid £40 for on Monday) I might end up with the RAID array for continuous uptime, and a 2TB or 4TB drive for occasional backups.

Reply to
Jethro

OK, dock them £1 out of pocketmoney and make them wait an hour ...

Reply to
Andy Burns

I think this question is largely answered by:

I'll miss it :-(

Darren - migrating to RedHat (for some servers at least - so far...)

Reply to
D.M.Chapman

Depending on your taste in licences of course. I suspect btrfs will eventually provide more of what zfs does ...

Don't look for info at kernel.org at the moment though!

Reply to
Andy Burns

HomeOwnersHub website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.