Friday
07Sep2007
Please backup your hard drive now... twice!
Friday, September 7, 2007 at 14:18
There is a tightness in my chest, and I am crying right now. I have just suffered a catastrophic data loss for the second time in my life. Fool me once, shame on, shame on, fool me can't get fooled again, or something like that.
In college, a freak transformer explosion and subsequent power surge killed my hard drive. From that point on, I swore to always back up my data, and mostly I did just that. As of mid this summer, I had a ridiculous mirrored RAID drive setup with external SATA drives and all sorts of doohickies. I had about 1 terabyte of data backed up locally and had started to upload it offsite to a service called Mozy. But then I started selling off my desktop in preparation for my move from Boston to NYC. I purchased a LaCie 1TB Big Disk and put all my media files and documents from my "Atlas" drive on it. That drive literally held my world on its shoulders.
I reasoned that after the move, I would re-establish my redundant data setup. I was not given the time. Two days after moving in, the drive started clicking. I knew that sound from my college crash, and raced to B&H Photo Video in Midtown. I purchased a Drobo storage device (a redundant storage array), hoping to save my Atlas drive. I was too late. I took the drive to Tekserve on 23rd St. It would cost $2,000, but can you put a price tag on your memories and thousands of hours of media production? They couldn't recover it. They sent it on to DriveSavers who said it may cost up to $6,000.
I had recently closed out my Discover Card, but decided it was worth going back into nasty credit card debt. Then today, I got the phone call. "We have some bad news."
They could recover nothing. They will just charge $400 for the attempt. It's funny, I struggled with the decision to send them the drive considering the cost but it is so clear now that I would rather have paid $10,000 to get my data back. On the technical side, here is what happened. That LaCie big disk is actually two 500GB drives "striped" together in an array. One of those drives failed and because the data is stretched across both, you can get nothing even from the good drive.
Fortunately, I managed to get some of my data uploaded to Mozy as of late May 2007. So I've managed to recover all my digital photos as well as my "Documenz" folder which includes my books, jokes, financial filings, scripts and everything else a digital paper version of a file cabinet would have. Over the past year, I have been using Google Docs for most of my day to day creative documents with columns, joke ideas, etc, so that's all good. Unfortunately, I have lost much, much, much more, so much that I cannot even be sure how much.
I certainly blame Lacie for the drive that failed, but my data is my responsibility. I will mourn this loss forever, and I really will never let it happen again. I'm trying to be open minded about this. It's the most aggressive "spring cleaning" I've ever done. Even with my mother's memories, I have thousands of photos and a bit of video. Mostly I have her in my heart, and if I think about it, I just happen to live in an era where it's possible to capture image and sound in such high fidelity. Most of the people that ever lived had no such technology to remind them of their lost loved ones. The best memories are always going to be with me.
Now, here's the plan
I urge everyone reading this to backup your most important files right now. Not tonight. Not tomorrow. But right now. Do a local backup. Upload files to a server. Email them to yourselves. Print things out and put them in a lockbox.
If you're interested in the Drobo, I have a discount code you can use for $25 off. It's EVBARATUNDE, and yes I get some money out of it. Mostly, I want you to avoid what I'm going through.
Update: September 10, 2007 @ 11:13am
Wow, I never expected such a massive response to this, of all my posts. Most of yall found me through reddit, it seems. Thanks for dropping by and thanks for all the very useful suggestions. I wanted to provide a few more details of what went down and why I wrote this in the first place
If you're interested in what I do when I'm not lamenting the loss of my digital existence, here are a few posts to give you a flavor for what I'm about. If you like what you see/hear/read, subscribe to the feeds or join the email list (both at the top of the blog page in the left and right columns)
Update: September 11, 2007 @ 8:50am
First up, welcome digg users who put me on the front page. I am so glad this story is making people back up their stuff. This is unbelievable. A few more updates
Update: September 11, 2007 @ 1:05pm
Another update. I've been reading the comments further and want to point people to a few more resources
In college, a freak transformer explosion and subsequent power surge killed my hard drive. From that point on, I swore to always back up my data, and mostly I did just that. As of mid this summer, I had a ridiculous mirrored RAID drive setup with external SATA drives and all sorts of doohickies. I had about 1 terabyte of data backed up locally and had started to upload it offsite to a service called Mozy. But then I started selling off my desktop in preparation for my move from Boston to NYC. I purchased a LaCie 1TB Big Disk and put all my media files and documents from my "Atlas" drive on it. That drive literally held my world on its shoulders.
I reasoned that after the move, I would re-establish my redundant data setup. I was not given the time. Two days after moving in, the drive started clicking. I knew that sound from my college crash, and raced to B&H Photo Video in Midtown. I purchased a Drobo storage device (a redundant storage array), hoping to save my Atlas drive. I was too late. I took the drive to Tekserve on 23rd St. It would cost $2,000, but can you put a price tag on your memories and thousands of hours of media production? They couldn't recover it. They sent it on to DriveSavers who said it may cost up to $6,000.
I had recently closed out my Discover Card, but decided it was worth going back into nasty credit card debt. Then today, I got the phone call. "We have some bad news."
They could recover nothing. They will just charge $400 for the attempt. It's funny, I struggled with the decision to send them the drive considering the cost but it is so clear now that I would rather have paid $10,000 to get my data back. On the technical side, here is what happened. That LaCie big disk is actually two 500GB drives "striped" together in an array. One of those drives failed and because the data is stretched across both, you can get nothing even from the good drive.
Fortunately, I managed to get some of my data uploaded to Mozy as of late May 2007. So I've managed to recover all my digital photos as well as my "Documenz" folder which includes my books, jokes, financial filings, scripts and everything else a digital paper version of a file cabinet would have. Over the past year, I have been using Google Docs for most of my day to day creative documents with columns, joke ideas, etc, so that's all good. Unfortunately, I have lost much, much, much more, so much that I cannot even be sure how much.
- My iTunes music and video library. (~300GB) I estimate I had about $1500 worth of purchased music and videos in there plus hundreds of gigs of ripped CDs. The good news is I saved all the original CDs and can re-rip them. I had also "acquired" a massive music collection from a friend which ended up creating more problems than it solved. There was a lot of music I never really wanted to own permanently. I can repurchase the iTunes music at far less than the cost of the data recovery, though I'll see about begging Apple for a restoration. I've head that happens sometimes.
- My video projects (~500GB). This includes imported MiniDV footage and many edited and rendered Final Cut and iMovie projects made since January 2005. The good news is I have all the original MiniDVs and I can download the most valuable rendered projects back from YouTube (I hope) and blip.tv which hosts a bunch. The bad news is video is the most time intensive, high learning curve activity I have ever engaged in. Much of my knowledge in those project files has to be relearned.
- My audio projects (??GB). This includes raw audio for my podcast, including dozens of unedited, unreleased interviews. I've often felt bad that I never got to many of these. Now I have a pretty good excuse.
- My old computer files. About two months ago, I extracted data from my old college computer hard drives and put them on the Atlas drive. This had emails, papers, mp3s, etc. I was so excited to have found this time capsule, but now it's gone.
- My mother. At the end of it all, I am pained by the loss of the above items, but nothing can represent the sense of anguish I feel at having lost audio of my mother who passed away two years ago. We had taken a cross-country drive together, and I recorded hours of conversation. I only got to podcast a little bit of it (which can be redownloaded from my webhost) but the unedited stuff is beyond valuation. It's like losing her all over again.
I certainly blame Lacie for the drive that failed, but my data is my responsibility. I will mourn this loss forever, and I really will never let it happen again. I'm trying to be open minded about this. It's the most aggressive "spring cleaning" I've ever done. Even with my mother's memories, I have thousands of photos and a bit of video. Mostly I have her in my heart, and if I think about it, I just happen to live in an era where it's possible to capture image and sound in such high fidelity. Most of the people that ever lived had no such technology to remind them of their lost loved ones. The best memories are always going to be with me.
Now, here's the plan
- I have the Drobo with 1.3 terabytes of capacity to be the home of New_Atlas. This drive will also be mirrored on a 1TB external Glyph and online via Mozy or a similar service. Any recommendations?
- My MacBook Pro internal drive will be mirrored on the Drobo/Glyph/Mozy setup as well
- I'll keep a smaller subset of high priority files for more frequent offsite backup
I urge everyone reading this to backup your most important files right now. Not tonight. Not tomorrow. But right now. Do a local backup. Upload files to a server. Email them to yourselves. Print things out and put them in a lockbox.
If you're interested in the Drobo, I have a discount code you can use for $25 off. It's EVBARATUNDE, and yes I get some money out of it. Mostly, I want you to avoid what I'm going through.
Update: September 10, 2007 @ 11:13am
Wow, I never expected such a massive response to this, of all my posts. Most of yall found me through reddit, it seems. Thanks for dropping by and thanks for all the very useful suggestions. I wanted to provide a few more details of what went down and why I wrote this in the first place
- The more technical side of the failure is that the "master" drive is fine but the "slave" drive suffered a series of head crashes. Basically, a major mechanical failure happened, and the platters inside the drive collapsed. It does not appear to be due to physical impact but just a mechanical malfunction
- I appreciate just about all the comments except for those telling me I'm an idiot. I know that. This single-point-of-failure system was temporary during my move. The odds of the drive failing in such a short period of time are low, but it happened. Remind me to drop in on your house and mock you when you suffer your own tragedy
- I wrote this post to 1) provide an emotional outlet for me 2) see how others might be handling their own data backups in this era of digital memories but mostly 3) to scare people into backing up their stuff as soon as possible in one way or another. I really don't want this to happen to others. It costs too much in time and emotional energy.
If you're interested in what I do when I'm not lamenting the loss of my digital existence, here are a few posts to give you a flavor for what I'm about. If you like what you see/hear/read, subscribe to the feeds or join the email list (both at the top of the blog page in the left and right columns)
- Facebook Follies (or the Dangers of Investing in Someone Else’s Platform)
- gcc 004: I Hate the Smugness of Apple (VIDEO)
- Iraq is to Vietnam as Dubya is to WTF!?!? (Huffington Post)
- Jesse Jackson Really Needs to Read a Book… explaining what satire is
- Me. Public Radio. Monday Apr 24. Racism In America.
- and my standup set on YouTube
Update: September 11, 2007 @ 8:50am
First up, welcome digg users who put me on the front page. I am so glad this story is making people back up their stuff. This is unbelievable. A few more updates
- Apple's iTunes folks restored almost all my purchased items, 83GB worth in over 900 tracks.
- A 1TB Glyph drive has arrived which I will use to back up the drobo in a "spanning" setup. It's two drives, but in this case the Glyph will fill up one drive then the other, sequentially. I'll store this in a fire-proof box in my home. I'm also gonna store this all on Mozy, so that's three places (two on-site and one remote) with ALL my data. I'll make smaller backup sets of really important stuff
- Today is my birthday, and getting Dugg is the best web gift ever... way better than a $1 Facebook "gift" :) And yes, it really is my birthday. Check the vid...
Update: September 11, 2007 @ 1:05pm
Another update. I've been reading the comments further and want to point people to a few more resources
- The Infrant ReadyNAS is a tool many have mentioned for hard drive backup. I studied it vs. the Drobo and chose Drobo, but that may not be right for everyone or even for me.
- An eBook recommended by Macworld called Take Control of Mac OS X Backups, 2nd Edition. Here is a free article in two parts which goes through a lot of the same material. There is some great content here even if you don't have a Mac
Technorati Tags: backup, data, hard drive failure
tagged Apple, Backup, Computers in Personal, Technological
Apple, Backup, Computers in Personal, Technological 
Reader Comments (251)
Jesus man. My chest is tight too now. And my eyes are watering. Thank goodness you still have memories to recall. I'm so sorry.
So sorry to hear about this man. I am a huge advocate of external hard drives. Not only for the obvious reasons, if you are heading out of town, you can stash the external hd to protect your photos (that's my main concern).
A thief will rip off your computer, but never your photo albums. Food for thought.
Same applies in the event of a fire, it's much easier to rip out the USB cord and carry it out of the door.
So sorry about the loss of your mom's voice. That truly is tragic dude.
Peace
Dave
Thats super-harsh, dude. I've been following this story on twitter. I'm amazed they couldn't save anything. I recently set up a monster machine and set up RAID across the drives. This makes me really nervous about it now - that's why RAID5 is the way to go, I guess.
There are scenarios where I could lose a lot of data - my most vulnerable is my laptop - I really should do something about that. The new box I mentioned above also needs something better. My linux servers, though, all share the load and back each other up. I've got 2 boxes in one location and another offsite and they all have a decent amount of storage. I do nightly rsyncs among all three boxes, so I feel pretty good about those.
But the two Windows machines really make me nervous. As does the one external drive I used to use for video projects before I built the big desktop. Externals/poretables are just subjected to too much abuse to be trusted much.
Thanks for the reminder to double-check my backup routines to make sure stuff is really doing what I expect.
Now, you better never let that happen again!
Wow, B. I'm sorry to hear about your loss, and thank you for the reminder. I plan to back stuff up the very second I get home. I'll also keep those Mini DV tapes the way they are as a second backup.
P.S. One reason I like eMusic over iTunes is that I can re-download everything I ever bought for free.
i've learned to accept that all data is fragile and temporary. electronic data most of all. still sucks royal booty crack (understatement) that you've lost so much, especially those recordings of your mom.
for online backup, may i suggest amazon S3? you'll probably have to learn some programming magic, but it's pretty darn cheap. http://aws.amazon.com/
I'm really sorry to hear about your loss. I've had a couple of drive crashes in my days and they are always painful. The saving grace for me has always been that I've never quite been sure what is gone forever and what is buried in one of the giant backup archive files that I never bother to sift through. It hurts hearing about your audio and video that you know is gone.
S3 does sound wonderful, but uplink speeds on modern broadband connections make it challenging for media, especially if you're editing.
Best,
Leo
Hey man,
That really sucks. I don't mean to minimize your loss in any way, and I feel for you -- I've lost stuff, too (one of my Maxtor drives failed), but I thought I'd point out a few things:
1. Drives fail. It sucks. But they do. Drives are rated with a mean time between failures (http://en.wikipedia.org/wiki/MTBF). It's a MEAN time. Some drives will fail before that, some will last longer.
2. I'd point the finger of blame at the manufacturer of the actual drive mechanism. LaCie doesn't manufacture the actual hard drives. If the housing or the firmware or the external interface failed, I'd blame LaCie. My LaCie 1TB NAS device has two 500mb drives and both are Seagate ST3500630AS. As far as I know, the only companies manufacturing the actual drives these days are Fujitsu, Hitachi, Seagate, Maxtor and Western Digital. This reminds me of the time when people were blaming Ford for the bad Goodyear tires. Ford doesn't make the tires. LaCie doesn't make the drive.
Thanks for letting me know about Mozy. Seems like a good service.
oh my gosh, Baratunde - I meant to write to you to let you know how much I enjoyed hearing the podcast with your Mom. but, you are right, she lives in your heart, deeper than any electronics can go...I am so sorry, and hope that whatever fixes can happen, do. as painlessly as possible...
I'm so sorry to hear about the loss of data but the loss of the audio with your mother on it is even worse.
I think you can get a 1 time redownload of your iTunes music. Thanks a lot for turning me on to Mozy and Drobo.
Sounds like you need to discover spinrite. http://www.grc.com/spinrite.htm
Spinrite is a low very level disk utility, and has recovered litteral thousands if not 100's of thousands of hard disks. It's saved my skin a few times and is well worth buying. I'll let you research it yourself, if you want to here more about it, listen to a Security Now podcast from twit.tv. The co-host Steve Gibson is the author and found of grc.com and spinrite.
Good Luck
Get a second opinion.
Any reputable hard disk recovery set-up will offer a cost free assessment.
Secondly, the striping does make it more difficult. However, it is by no means impossible to recover data from a striped drive. Hell, a competent engineer can pull back stuff even if they only have 1 part of the array.
Personally, I've heard from people who have had drives recovered after water immersion, fire damage, dog attacks. Mechanical failure should be no issue for a reputable set up.
Mozy Unlimited is starting to look like a better deal every day...
stay away from lacie. they use the cheapest drives they can get and resell them at a huge premium. their service is abysmal. there's an inverse relationship between quality of support and quality of product, once again verified by lacie.
Sorry but why in the heck would you blame LaCie? Things break from time to time; cars, planes, microwaves, etc. I find it completely sad that from your last data lost you obviously didn't learn a lesson.
So in hopes you learn your lesson here is what I do. First I have an NAS device with 1 TB, this allows me to make complete copies of my MAC/Windows machines. Then I also copy my regular data to these machines. On top of this I use Mozy and Amazon S3 both to upload my data. I have no single point of failure so that if any of the four places my data is contained breaks I will always have a copy.
Have you tried SpinRite? Magical program for hard drive recovery. Worth a shot.
http://www.grc.com/spinrite.htm
Sorry to hear it, man, but thanks for reminding others that this can happen to anybody.
You would have happily paid $10,000 to have your data restored, yet if someone had asked you if you would pay merely a few hundred dollars to get a SECOND external drive as a backup, you would have declined.
You sir, with all due respect are an idiot.
That really sucks. Especially that you thought you were covered in case of catastrophic data loss.
I am really paranoid about the type of thing you said. I've heard too many stories of people's backups being bad and all-is-lost type of situation. With pretty much all of my life's memories being digital (I've gone digital a long time ago, storing all receipts, photos, CDs, etc digitally) if I were to suffer a data loss I agree - it would be devestating and $10k would be cheap to recover it.
My backup plan is as follows:
1. My main home PC hosts most of the active data. I have a normal HD - no RAID as that can sometimes make data recovery more impossible - not to mention if you do striping and not mirroring it doubles the chance of one hard drive taking out all of your data. If I had a lot of money I would probably do RAID mirroring of some sort.
2. I have a linux server in-house that acts as a file server. I back up all music, documents, and anything else I can think of to this server hourly or nightly depending on the type of data. Backup is unencrypted.
3. I have a portable hard drive I plug in once a month and back up most things to. This is then stored in a fire-proof safe in my residence. This backup is compressed and encrypted in case it is stolen.
4. I rent a dedicated server to which I back up to. This backup is done daily and is encrypted; and is located fairly far away from me to give some separation for any sort of natural disasters.
5. I backup my photos manually maybe twice a year onto DVD. This is in the off-chance that someone sets off an EMP which manages to wipe out all magnetic storage devices. I told you I am paranoid.
6. I backup all other information such as USB Key Drive and Laptop data to my main PC which is then eventually moved to the other backups.
The main software I use to do all of this is called SyncBack SE
http://www.2brightsparks.com/syncback/sbse.html
I highly recommend to all - you have not only scheduled backups but also backups based on when you insert a certain drive or other criteria.
I had a similar problem (clicking drives, write/read errors after some time) with my LaCie drive, a 500GB version, 2*250 hard drives. It also stretched the data across both disks but NOT like a RAID0. Rather, it just had a chip that simulated one big partition. I was able to disassemble it, connect the two disks to my PC and restore pretty much everything using R-Tools NTFS and scanning the drives. I am not sure how the newer LaCie drives work, but my guess is they also don't handle the data like a RAID0. If your drives are actually mechanically broken, it is probably too late. But if you still have the drive and have not tried it yet, I would recommend you do ;)
I know the feeling, I am sitting in a hotel room right now waiting for "Kroll Ontrack" to open up so I can drop a drive off with them. If you do still want to try to recover it, I would check with them. They supposedly can recover raid drives.
Ryan
So sorry to hear this. Losing hard drive data is the same thing as as having your memories wiped out of your brain. Thanks for the post I need to start backing up again soon!
As others have said, you should try spinrite on your drive. There is a good chance that spinrite can save your data. Go to http://www.grc.com/spinrite.htm and give it a try.
Use CBL (cbltech.com). They've recovered drives from me damaged by almost everything you can imagine and they just managed to recover a few TB of data for a friend which was spread over a huge software RAID that was incorrectly configured and which the software manufacturer couldn't help with at all. The drives couldn't be seen by the software. CBL got every last bit back.
Their slogan is "No data, no charge" and they've lived up to that for the 11 years that I've done business with them. I don't work for them and I don't get paid for referrals. Ever drive I or any acquaintance of mine has ever sent them was fully recovered but if they can't recover -- unlike the other scum out there who charge $200-2000 just to look at the problem -- there's no bill.
http://services.seagate.com/ seagate data recovery is much cheaper than what you listed. i have usedthem often, they have never been able to not repair a drive, and they do not charge if they can't recover any data. they are among the best in the business, and since seagate has bought them out, it might be worth hanging on to the drive and sending to them for a free assessment. no need to explain the situation or past attempts, if they can get the files they will send you a file list of your drive.
Damn, that sucks. This might not be feasible for all your files, but here's a tip: Spread them around. And I don't mean as in several backups - That's inherently risky since truly redundant backups are almost impossible to achieve (what happens if your house burns?). Instead, use BitTorrent (there's a fair chance any interesting content will be downloaded by lots of people), YouTube, Flickr, Gmail, etc..
Personally, I'd also recommend a version control system for settings and small personal files - Not only can you have one copy on each machine you use (same settings everywhere X backups), but you won't lose stuff you've deleted from the sandboxes.