Glass Maze Every jumbled pile of person

Another Hard Drive Dies

Fuck. Fuck fuck fuckity fuck fuck. My hard drive just crashed.

But not just crashed. Crashed and took two weeks of data with it. Including my taxes (unsent, of course) and a story I was actually managing to make some progress on, a rare thing in these fallow post-Clarion days.

And the killer is, I could have stopped it. Not the hard drive crash, which is an unavoidable hazard of the trade (although this is the second catastrophic failure in the last three months, god damn it Apple), but the data loss. Over the past year, I’ve cobbled together a backup strategy that seemed, to me, pathologically anal. It includes:

1. Full, bootable backups to two separate hard drives.
2. Incremental backups to a Time Machine drive.
3. A remote subversion repository for the stories
4. A remote IMAP respository for my email

And yet I still lost data. Why? Because I — idiotically — haven’t actually used any of these lovely mechanisms for a long time. Haven’t done a full backup for a month. Haven’t plugged in for a Time Machine backup in two weeks. Haven’t been checking my stuff into subversion. About the only thing I have been doing is using my IMAP store, and that’s only because there’s nothing to actually do there. The protocol does it all for you, automatically.

I guess that’s the problem here. It’s not that my backup strategy is insufficiently anal: it’s insufficiently automated. If there’s one thing I’ve learned in my 37 years on earth, it’s that it never pays to count on me. I have this habit of letting me down.

So how to automate? Here’s what I’m thinking:

1. Get that new Time Capsule router, which does incremental backups over your home network network, automatically and wirelessly.
2. Bite the bullet and subscribe to .Mac. I’ve heard nothing but terrible things about its hyper-shitty WebDAV-based filesystem, but it does give you automated remote backups wherever you are, and it integrates effortlessly with Leopard. Also, it’s perdy.
3. Make TextMate automatically commit to Subversion when you close a file. Not sure how to do this, but TextMate seems infinitely customizable. There has to be a way.
4. Burn CD backups of static, archival data — old stories, financial data, etc.
5. Set up repeating alarms that prompt me to do full weekly backups, with SuperDuper¹. This is the weakest link in the strategy, as it requires me to actually do something — but I tend to respond to alarms.

Anything I’m missing? That seems like it should cover it, as long as I’m at least occasionally plugged into a network. It has to, really. Hard drives expire with alarming regularity in our household, and every iota of lost data is just searingly painful. I love my data.

Actually, step 5 can be automated. SuperDuper can be scheduled to run automatically.

I do this as well; I have an USB harddrive for that reason, which is usually turned off. SD is scheduled to run every Friday at 16:00, so I’ve set up an iCal reminder that hits me each Friday at 15:45 w/ a sticky, high priority, red flashing reminder to turn on the drive.

It’s working really fine.

If you want more info, you have my contact details. :)

Cheers, Carlo

Time Machine over wifi is probably the one thing you need, the rest is just redundancy.

Skip .Mac. It’s all crap, both from experience and from what I’ve read and heard. You don’t need automated wireless backups with .Mac — you have Time Machine for that.

If you commit automatically, you will be committing broken code without commit messages. I’d recommend against that :P Just get into the habit of committing for every atomic thing you implement, like for every complete step of a feature as long as it leaves trunk in a usable state.

Get a Drobo for archival. It’s redundant so you won’t lose anything if a disk fails; you just have to copy the file over so it’s not at tedious as burning CDs and keeping either very good labels or very good indices; it’s spotlighted so finding the files later will be a piece of cake; and it’s basically infinitely large (since you can swap disks for bigger ones while it’s running) so it’ll hold everything you throw at it. I use a drobo for movies and archives; I have basically every file I’ve ever created since the eighties in one big spotlighted blob :)

And SD! backups might be useful, but TM’ll probably keep you safe anyhow :)

I actually just caved and bought a TeraStation Pro II. A whole 2TB at, that running RAID5 (loss of 25% of data capacity). Of course you don’t need the rack mount version (I had to have it; just because I could use bolts to hang it up), but a wonderful product. So, I can loose a hot swappable drive without loosing data. I then hung two old USB drives off its two USB ports and have it doing automatic backups via its web-based software. Priceless.

Z: Don’t you need identical drives to be able to hotswap? The cool thing about a Drobo or a ZRAID setup is that you can just replace a drive with a bigger one and thus basically never having to reformat.

Thanks for the tips, all — I’ll check all that stuff out — Drobo and the Terrastation in particular, both of which seem very cool. Though the Terrastation is a little scary — it looks like it’s getting ready to unfold into a angry giant robot from the 50s.