“Perhaps you know that sinking feeling when a single keystroke accidentally destroy hours of work. Now imagine wiping out a disc drive containing an account worth $38 billion.”
Eeek! It’s a pretty horrifying prospect, even if very few people reading this e-zine are likely to get their hands on $38 billion.
It’s not the money that was destroyed, of course. Both the destruction and the creation of money are pretty much matters of consensus, or perhaps of fiat. But without the proof that the money represented by zeroes and ones in the bank is yours, it might as well not exist for all the good it’s going to do you.
The money in question belongs to the Alaska Permanent Fund, of which I’d never heard before this. Part of the fund’s mission is to ensure that Alaskans receive timely dividends. That’s just a bit difficult to accomplish when you don’t have any records of who gets what.
The story continues:
“While doing routine maintenance work, the technician accidentally deleted applicant information for an oil-funded account—one of Alaska residents’ biggest perks—and mistakenly reformatted the backup drive, as well.
There was still hope, until the department discovered its third line of defense, backup tapes, were unreadable.”
I’m not sure how one accidentally reformats a backup drive, and I haven’t seen anything online (most sources just seem to reprint the AP story) to explain it. I suppose if it were an internal drive, one might choose the wrong partition to reformat—easy enough to do if they’re all the same size and their drive letters have disappeared or been rearranged (something that happens with DOS-based Norton Ghost, so that if your main and backup drives are the same size, you have to look at their contents to be sure you’re making an image of the right one). But it’s really hard to imagine accidentally reformatting an external drive.
Unreadable tapes are no big surprise. Blogger and system architect Payton Byrd uses the story to illustrate his main point that “Tape sucks as a backup medium…Even the most unreliable of today’s hard drives will be much less susceptible to failure as [sic] tape.”
In the end, the fund was able to get its data back—because it still had the original paper forms it had scanned in to create the database. Re-entering the data cost them $200,000.
Back in 2005, I wrote about a similar thing which happened to a client of mine on a smaller (though proportionally just as significant) scale. While the business owners were overseas, the backup server stopped working. Then the main server messed up. They came home to find a month’s worth of customer transactions gone up in smoke, and spent $10,000 on data recovery, hardware replacement, and re-entering the data from the paper printouts the CEO had taken it into her head to start collecting in the previous year.
In my client’s case, the problem was one of hardware failure—compounded by the fact that no one in the office thought to report the problem with the backup server to anyone who could have done something about it.
The single biggest cause of data loss is human error. Is there anyone reading this who hasn’t accidentally deleted a file, or copied over something they meant to keep, or even thrown an important paper into the recycling bin? The Windows recycle bin and the Mac trash can give us at least some opportunities to retrieve items we didn’t mean to get rid of, but just as there’s no such thing as a child-proof container, there’s no such thing as an error-proof system.
And if there’s anything the least bit wonky about the hardware or software, it just makes it that much easier for us to mess up. I love my computer, but for some reason the “Shift” and “CTRL” keys tend to stick. This caused me no end of bafflement and trouble until I realized what was going on. I would end up actually rebooting my computer because I couldn’t do anything and all my keyboard commands were messed up. Now I’ve clued in, and if the keys I press do something funny, I whack the “Shift” key and see whether that fixes it, and if it doesn’t, I hit the “CTRL” key. Usually one or the other will set things to rights.
Probably the only thing more dangerous to our data than we are ourselves are our pets and children. The cat once managed to shut my laptop down by walking across it. I’m still not sure how she managed it, because she was nowhere near the power button. And I’m sure the inside of the machine is full of cat hair, which no doubt will combust one day. (Well, maybe not, because the drive doesn’t get all that hot, but my computer repair guy is decidedly reproachful on the subject.)
So what can we do? There’s only so alert and careful we can be, but there are a few precautions to take. Set any program you work with regularly to autosave on a frequent basis. (And practice hitting that CTRL-S (Windows) or Command-S (Mac) key combination to save manually, until it becomes a reflex.) Set up automated backup systems and check them to make sure that they’re working. Use good-quality backup media and store them in protective containers away from heat, dust, moisture, and cat fur. Send or put copies of your most important data in a secure off-site location on a regular basis. (It’s much easier if you set up a routine, e.g. “On the first of each month I have to put all this onto CD and take it to the safe-deposit box.”)
And if anyone asks you to do hard drive maintenance for a fund worth $38 billion, just say no.