The Ur-Guru pointed me to this sad post on Refreshing News:

A Calgary man is desperate to get his stolen laptop, with years of work on it, back.

It was stolen from the trunk of his car while he went for a run in Edworthy Park Wednesday night.

John Boldt is pleading for the return of his hard drive, which contains research and notes for the thesis he was writing for his master’s degree in history. He only had three chapters left to write before his paper was complete.

Unless he gets it back, Boldt will have to abandon his dream and quit at the University of Calgary.

“It’s so many years of my life just thrown away,” he says. […] Boldt says if he doesn’t get the data back, he won’t be able to return to the U of C when classes start next week.

I feel for John Boldt. My car was broken into in February, and it’s a terrible experience even when nothing of value is taken. Even if the laptop had been in plain sight on the passenger seat and not in the trunk, that wouldn’t have given anyone the right to steal it.

But how does someone get into, and nearly through, a master’s program without learning to back up his work, especially when it’s so important to him?

Back when I was in graduate school, in the late Eighties and early Nineties, I had all my papers, and then the draft chapters of my dissertation, on floppy disks. This was partly because I didn’t own the computers I worked on, and partly because computers didn’t have much internal storage. But there were always two sets of disks, the main set and the backup set. Always. I can’t even remember who taught me to do that, but the lesson was an old one.

Heck, even when I was an undergraduate using the mainframe as a glorified typewriter, they told us we could get a tape with our files on it. (I never did, but I can’t imagine what computer I would ever have been able to use it with, anyway. It would be easier for me to run my undergraduate honors thesis through a scanner and OCR it.)

Ah—according to Gawker, there was a backup drive in the trunk along with the laptop, and that got taken, too. This additional info makes Boldt look a bit more clueful. Some people do habitually store their backup drives in the car, in order not to have them in the office where the desktop computer is. I’m not sure that would be the best option if you’re a laptop person carrying your computer around a lot, but at least it suggests he was doing something to protect his work.

Many of the commenters on the Gawker post are protesting that the situation seems fishy to them, pointing out numerous ways that the chapters of the thesis could have been backed up and asking why Boldt’s advisor doesn’t have a copy of it. It’s a good question: I certainly gave chapters of my dissertation to my advisor as I wrote them, though I think I had to deliver them in hard copy. Having a printout would be a lot better than not having anything.

Others point out that Boldt had plenty of opportunity to use an online backup service like Mozy or Carbonite or even a tool like Dropbox. A history thesis would almost certainly be a Word doc, probably using a template provided by the university or the department. It might be large as Word docs go if it had illustrations in it, but it would be unlikely to be larger than the 2 GB limit for free accounts at Mozy.

But, as I suspected, Boldt would have had no need to resort to a commercial service if he wanted online backup. The University of Calgary offers all students a service called Webdisk File Storage:

Webdisk is a remote file storage system that allows you to store your files on a IT server making them easily accessible from any computer with an Internet connection. […] Webdisk files are automatically backed up every day, so they can be recovered if they are inadvertently deleted or changed.

So where was Boldt when Webdisk accounts were being handed out?

It’s easy to get sloppy about backups, to say you’ll put another system in place tomorrow, to forget to re-establish levels of protection when you get a new machine. But when you have really critical data, you can’t back it up too many ways or in too many places, and you can’t do it too soon.

And that will be true even if John Boldt turns out to be just as much a fiction as Whiteboard Jenny.

Literally.

On February 20th, there was an an accident during the annual inspection of the fire-suppression system at WestHost’s data center. If you rewrite WestHost’s account of the incident in the active voice, it amounts to “The vendor forgot to remove an actuator before the inspection, and this triggered a release of Inergen all over the data center.”

According to Wikipedia, Inergen is non-toxic…to humans. It is, alas, highly poisonous to computer servers. And John’s blog just happened to be on one of the worst-affected machines.

Bye-bye blog.

The good news is, WestHost actually had backups of its clients’ sites. Not all hosting companies actually back up your website for you. (You can usually make your own backups through the control panel, but you may or may not be able to automate this process. Of course, if you have a traditional HTML site and edit the files on your computer before uploading them, you should be able to back up your local versions easily.)

The better news (theoretically) was that John, being a smart guy with lots of IT experience, had recently made his own backup of his blog. That meant he was starting out in better shape than Jeff Atwood over at Coding Horror, who had to rely on other people to piece together bits of his lost blog for him.

But the first attempt to restore Ronin Marketeer left a bit to be desired. When I sent a sympathetic inquiry to John after hearing the podcast, he sent me a link to a post he had titled “When Even a Backup Is Not Enough.”

As you can see, everything is all f’d up here.

Over a week ago disaster struck at my hosting company, during a fire alarm test the suppression system was triggered, hosing all the servers. This blog was dead for a full week.

We were offered to move our hosting from the version 3 infrastructure to v4, and I took up the offer since it got my domain back 2 days earlier. Unfortunately the new environment is not the same – even though I have a full backup of my Database that supports this blog, the new system does not allow you access to the directory where that data is kept.

I’m no expert in MySQL, but it looks like I’ve gone from having my own instance to sharing one on the server with everyone else.

The end result is that all my archives are gone for now and my Google juice vanishing as there’s no access to any of my archives. It looks like my only path is to install WP and MySQL on a box of my own, then do a WordPress export so I can then import it back in. I cannot believe that having the actual files is not enough for me to do a restore.

“My god,” you may be thinking. “If having the backup is no good, why bother making one?”

But if he hadn’t had the backup, the story would not have had a happy ending, and it does. John had to do some heroically geeky things, but he was able to get the blog back up and running. He did lose some comments, probably due to the nature of the restore process, but everything else seems to be intact. John started Ronin Marketeer in November 2006, and he’s a pretty prolific blogger. It would have been a serious loss, and no fun to try to reconstruct from the Google cache and the Wayback Machine.

I’m betting John will be especially interested in the WordPress backup plugin I’m going to be writing about next week. Everyone else certainly seems to be, and I’m very impressed so far.

Here’s the story. Jeff Atwood, geek extraordinaire, hosted his blog on a virtual machine (VM) on a server at a web hosting company. VMs are great for developers, because you can simulate different operating systems in order to test your software on them, and you can take snapshots and re-create them easily.

Unlike Jeff Atwood or the Ur-Guru, I’ve never worked with a VM. To use them you need more RAM than I have at my disposal. I can see how it might be handy to have one to test backup software on without cluttering up the registry of my main machine, though. But apparently backing up the contents of a VM doesn’t work quite the same way as backing up your ordinary operating system and files. This is a setup for a very dangerous situation: when you think you have backups, but don’t. (Have I said “Test your backups” lately? Test your backups.) And this, it seems, is exactly how Jeff lost his blog:

1. The server experienced routine hard drive failure. (Ed. note: hard drive failure is described as “routine.” In data centers, where drives are spinning 24/7, that’s exactly what it is.)
2. Because of the hard drive failure, the virtual machine image hosting this blog was corrupted.
3. Because the blog was hosted in a virtual machine, the standard daily backup procedures at the host were unable to ever back it up.
4. Because I am an idiot, I didn’t have my own (recent) backups of Coding Horror. Man, I wish I had read some good blog entries on backup strategies!
5. Because there were no good backups, there was catastrophic data loss. Fin, draw curtain, exeunt stage left.

Now, I don’t know what blogging platform Jeff was using. Given that he’s one of those extreme geek types, it could be something really obscure, even something he created himself. (Power geeks are like that; they’re as likely to insist on developing their own tools as to use anyone else’s.) I don’t even know what operating system his VM was running. But I know that there were ways to back up this blog when I was using Blogger (published by FTP to my own website), and there’s no excuse for not backing up WordPress blogs, since there are handy plugins to make it easier. And any offline blog editor like Windows Live Writer or Ecto will save local copies of your posts, so you can back them up along with the rest of the data on your hard drive. (Back in the days before Windows Live Writer, I used to write my blog posts in Microsoft Word, but you don’t actually want to paste from Word into anything that uses HTML. It did, however, mean that I had local copies.)

Jeff was able to re-build the text portion of his blog in HTML thanks to a fellow extreme geek who’s been archiving the Internet, but lost many of the images (which are not, apparently, on his hard drive, or not readily identifiable from among those on his hard drive). I shudder to think just how much work this must have been—and how much more work it will be to convert it back into blog format if he chooses to do that.

The lessons Jeff Atwood learned from the demise of Coding Horror are not just for geeks.

What can we all learn from this sad turn of events?

1. I suck.
2. No, really, I suck.
3. Don’t rely on your host or anyone else to back up your important data. Do it yourself. If you aren’t personally responsible for your own backups, they are effectively not happening.
4. If something really bad happens to your data, how would you recover? What’s the process? What are the hard parts of recovery? I think in the back of my mind I had false confidence about Coding Horror recovery scenarios because I kept thinking of it as mostly text. Of course, the text turned out to be the easiest part. The images, which I had thought of as a “nice to have”, were more essential than I realized and far more difficult to recover. Some argue that we shouldn’t be talking about “backups”, but recovery.
5. It’s worth revisiting your recovery process periodically to make sure it’s still alive, kicking, and fully functional.
6. I’m awesome! No, just kidding. I suck.

So when, exactly, is International Backup Awareness Day? Today. Yesterday. This week. This month. This year. It’s a trick question. Every day is International Backup Awareness Day. And the sooner I figure that out, the better off I’ll be.

Have you checked your backups lately? Now might be a really good time.

The adventure began on October 10th. The headline in TechCrunch read “T-Mobile Sidekick Disaster: Danger’s Servers Crashed, And They Don’t Have A Backup.” Jason Kincaid, author of the TechCrunch article, was absolutely scathing on the subject:

This goes beyond FAIL, face-palm, or any of the other internet memes we’ve come to associate with incompetence. The fact that T-Mobile and/or Microsoft Danger don’t have a redundant backup is simply inexcusable, especially given the fact that the Sidekick is totally reliant on the cloud because it doesn’t store its data locally.

I’ve never used a Sidekick, but a mobile device that doesn’t store phone numbers, etc locally at all seems bizarre, and in fact I’m not sure that statement is quite accurate, given suggestions in other articles that if you keep the Sidekick charged and turned on, you would at least save anything in its current memory.

But then, I still have a “dumbphone,” so what do I know about how these things work?

By October 11th, T-Mobile had posted the following discouraging notice on its user forums:

Regrettably, based on Microsoft/Danger’s latest recovery assessment of their systems, we must now inform you that personal information stored on your device — such as contacts, calendar entries, to-do lists or photos — that is no longer on your Sidekick almost certainly has been lost as a result of a server failure at Microsoft/Danger.

Not surprisingly, the media has been all over the story. “Microsoft has said that the hardware failure that caused the problem took out both the primary and backup copies of the database that contained Sidekick users’ information,” Ina Fried wrote on October 12th. “But the question remains, why wasn’t there a true independent backup of the data?”

That would certainly be my question. Rafe Needleman, also writing for CNET on October 12th, concluded that you can’t trust the cloud because you can’t trust the people running it. The problem, in other words, is not one of technology. Tech support staff often refer to problems that start “between the keyboard and the chair.”

If it’s possible to create independent, redundant backups in your own data center, it’s possible to do it in the data centers used by cloud computing companies. The only difference  is that you can’t walk down the hall and see that they’ve done it. Some people will slack off when you aren’t there to hold them accountable, but that’s not true of everyone. As Lance Ulanoff concluded in his October 13th article, “Don’t Blame Cloud Computing for the T-Mobile Mess,”

Obviously, something went very, very wrong with T-Mobile and Microsoft’s Sidekick data set-up, but let’s not throw out the baby with the bathwater (or the cloud with the rainwater). The cloud isn’t the problem. Instead, I blame the people—as always.

But the Register, with typically British enthusiasm for a pun, declared “Danger Lurks in the Clouds” on October 18th. The danger is that all mobile devices will rely increasingly on a working connection to provide any functions at all. Nevertheless, author Bill Ray concludes:

Cloud-based servers are still more reliable than most of the kit knocking around users’ homes – the life expectancy of an Apple Time Capsule, for example, is just over 17 months according to the Time Capsule Memorial Register, so even those who are backing up locally shouldn’t be too smug.

That article concludes, in the Register’s usual tongue-in-cheek fashion, that paper is the only safe storage medium.

By October 20th, Microsoft and Danger had in fact been able to restore some of the data, as reported in CNET and on T-Mobile’s user forum. That’s a happier ending than Sidekick owners had been led to expect. I’m glad they got their data back, but if I’d been affected, I’d want more.

I’d want to know what the company was going to do differently from now on so that this wouldn’t happen again. And I’d want a free application that would let me back up all my contacts, calendar entries, etc, onto my computer. It wouldn’t even have to sync with Outlook or Google or Mac-whatever, as long as I’d be able to restore the data to my mobile device.

Finally, as an occasional naming consultant, I want to see Microsoft Danger rebranded. What incentive do you have to entrust something valuable to a company called Danger? What incentive do employees of a company called Danger have to be careful? Danger is a fun name for a company that makes games, but for data storage, it just sounds unreliable.

By about 10:30 AM I was nearly finished creating a mind map for the website redesign for a corporate client, based on their suggestions and another site they thought was a good example. I use MindJet’s MindManager Pro 6 software for this. The latest version of MindManager is 8, but I never bothered to upgrade. The program already has more features than I know how to use.

Anyway, I had opened an earlier map to check something that I wanted to add to the current map. I was finished with the earlier map and went to close it, and got prompted with one of those “Do you want to save changes?” dialogs. I hadn’t made any changes to that file, so I said “No.”

And then discovered that I’d just closed the whole program, and lost the last 10 minutes of work I’d done. Four hours of sleep, yeah. Does wonders for the brain function.

I save my work often, and I have most of my programs set to autosave at least every ten minutes. But it turns out I can make a lot of changes to a document in ten minutes, at least if it’s a mind map.

Now, I have Mind Manager set to automatically create .BAK files the way I have Word set to automatically create .WBK files. In Word, the .WBK files preserve an earlier version of the file; you can return to it if the current version gets corrupted somehow. I assumed the same idea held true with MindManager, but I couldn’t even get it to open the .BAK file, so I had no way to tell whether it had preserved my new changes.

I also checked Free Agent Sync on my F:\ drive, but that copy of the file, like the one on my C:\ drive, had not preserved the changes. So I had to go back and re-create all the work I’d just done. And in the process of doing so, I misspelled someone’s name, a thing I normally pride myself on not doing, and probably left a few other things out.

I’ve since gone back and set the autosave in MindManager (it’s under Tools | Options | Save) to 5 minutes. That might keep me out of trouble. But only if I get enough sleep.

If your hard drive fails and you don’t have up-to-date, functioning backups, you call in the data recovery experts. It’s kind of like sending your drive in for emergency surgery—down to the sterile environment the engineers have to operate in. Chances are, your drive is dead for good, but it might be possible to give your data a new lease on life.

On February 16th I received an invitation to interview Bay Area data recovery company DriveSavers, accompanied by a press release announcing their introduction of industry standards for data recovery. I’ve written about DriveSavers and their Museum of Bizarre Disk-asters in the past, and happily took the opportunity to interview Chief Information Security Officer Michael Hall on February 19th, 2009.

While the issue of “securing data, even during data recovery,” to quote the first message I got from Margie Schaffner at BLASTmedia, is certainly an important one, I knew that my readers would want to hear about more than just industry standards, so I put out a call on LinkedIn for questions to ask Michael. I organized the questions into four basic categories and consolidated the ones that overlapped.

Note: I have edited Michael’s responses (which I recorded) just slightly, mostly to make them more appropriate for the written form of this blog. In some cases, where he provided the answer to a later question earlier in the discussion, I have relocated what he said. I sent him a draft of this post to check for accuracy before releasing it, and a few responses have been filled in or corrected through those e-mail discussions.

Questions About DriveSavers

What differentiates you from Ontrack Data Recovery? Both companies have nearly identical taglines of being the world leader in data recovery services. (Note: when I asked the question, I expanded it to include competitors in general. When Michael answered the question, Michelle Taylor, Director of Communications at DriveSavers, edited it slightly, so the style of the response is more formal than the rest of Michael’s answers.)

The primary difference between us and any other data recovery company is our certified secure environment and unparalleled customer service. We adhere to very strict and stringent government security protocols, and we are also certified SAS 70 Type II compliant. We have the fastest standard service available and we meet those turnaround times about 99% of the time. Our actual facility has certified ISO 5, 6 & 7 cleanrooms, expert engineers and advanced technologies which enable us to maximize the success of every data recovery. We defend and protect our customers’ information from security breaches here. No one else is going to be able to hack into our network and gain access to our customers’ information. I’m not sure what Ontrack’s network infrastructure is like; I know that what differentiates us from all other data recovery companies is that we adhere to very high security standards and we have met or exceeded all those standards and we have certifications to prove it: they’re available for viewing on the website.

What determines your prices?

It’s based upon the capacity of the device itself and the turnaround time that the customer is requesting. We have a number of different options there. We have what we call the economy 5-7 day turnaround; we have a standard service, which is a 2-3 day turnaround; and we also offer priority service, which is an immediate turnaround. That means an engineer is going to be dedicated to that job from the time it hits our facility until it’s completed, to expedite the recovery process for them and get the dataset in their hands as quickly as we possibly can. So it’s basically capacity and how quick they want their data.

How are you going to make data recovery affordable for the average Joe or Jane who didn’t make a backup? (To which Sallie adds: how much slack do we want to cut people who don’t make backups?)

What we see with individuals and small businesses is that they almost never have an on-site IT person. They have a consultant who comes in and configures their backup, or their network topology, or their security, or their entire infrastructure. They put it in place, tell people how to use it, and then they walk away. They don’t come back and periodically check things. What we see more than anything is that when something’s been configured as a backup, no one’s ever taken the time to actually validate the configuration by doing a test restore to another device or checking the integrity of the data. What we see oftentimes is that whoever put the backup program in place, it worked fine for what the customer had at that point in time, but say they go from a simple database program to a SQL server, or they change their e-mail program to an Exchange server. Those files are open files; they’re constantly changing.

When they established their backup system initially, they didn’t have to worry about open files. So now they’re doing backups with open files, but they don’t have “backup an open file” option in their backup program. So they’re backing up an Exchange server and they’re backing up everything except the two open files that they need.  The same thing holds true with SQL. You can back up the entire SQL directory—except for the database. And then they have some sort of natural disaster—hardware failure or an earthquake or a power surge—and their hardware goes down. They think they’re fine until they try to do a restore, and then they realize that the whole system was configured incorrectly.

We see innumerable data recoveries that come to us because of that same scenario right there.

How do you know when the cost of recovering data is greater than the value of the data?

That’s something you have to determine yourself. The easiest way to look at it is, how long will it take you to re-create the data yourself manually? How many man-hours is that going to take? How many temps will you have to get all the paper trail that you’ve got back into electronic format—if you have a paper trail at all? How much are you willing to pay to keep your business running?

Are you hiring?

Not at the moment.

An Ounce of Prevention…

Isn’t prevention (e.g. Business Continuity Planning) a better investment than data recovery?

A Business Continuity Plan is imperative to any business, no matter how small or large. You have to have contingencies in place and have a pre-set plan: “If this happens I can do that. If that happens, I can do this.” That’s part and parcel of doing business. Is it more important to do that than pay for data recovery after the fact? Absolutely. If your Business Continuity Plan is written properly and it’s comprehensive and inclusive, you’ll probably never need to use us. Why wouldn’t you be proactive on the front end to take care of that. What we see, though, is even with Business Continuity Plans, there’s that 2% you can’t account for. Natural disasters. Simultaneous catastrophic hardware of the main device and the backup device. There are corporations that have us at the very bottom of their Business Continuity Plan. If all this does not work, here’s your last stop, and it’s a data recovery company.

What are you doing to integrate a “prevention” mode so that people can do encrypted, compressed off-site backups via the Internet (automatic of course), so that data recovery is more easily accomplished should it ever be needed?

We don’t offer offsite backup solutions for customers, but we are huge proponents of customers having that in place, and we can point people to different companies that handle that kind of program. We’re not going to offer it. Our primary focus is only on data recovery.

The Data Recovery Process

What is the most common reason for needing data recovery services?

About 80% of what we see is electromechanical failure. The reason we have the museum of Bizarre Disk-Asters is that it’s an unbelievably great visual representation of what can happen. But 98% of the time, that’s not what happens. You don’t have a fire, you don’t have a flood. A hard drive is a mechanical device. It’s not a question of whether it’s going to fail—it’s when it’s going to fail. A hard drive has a Mean Time to Failure rate attached to it; that’s the life expectancy for the device. 98% of the time either the drive dies on its own, or it’s been fried in a power surge. That will cause your drive to fail a whole lot quicker than being run over by a bus.

What kind of data is hardest to recover?

That’s a very open question. We work on all platforms. Any operating system. Any type of electronic device. Our rule of thumb is, if you can write a 1 and 0 to it, we’ll take it off of it. Some are more difficult than others, but I don’t know that I could definitively say “This is the hardest thing to recover.” Sometimes the hardest things to confirm [recovery of] are proprietary software applications that have been written specifically to a type of business or to an individual, where they’re not off-the-shelf applications that we can easily access and figure out. When we have situations like that, we try to work with the person who wrote the application or the customer to gain access to the application in order to confirm the data set for them.

What percentage of the time do you have success with recovery?

We have an overall success rate, but I’m not sure what it is.

The website says “the highest in the industry,” but doesn’t give any numbers. A later e-mail discussion with Michelle Taylor produced the following answer:

One of the most telling reasons we know we have the highest data recovery rate in the industry is that the majority of the drives we see at DriveSavers have visible signs of previous data recovery attempts. In some cases, these attempts have caused so much damage that the data is unrecoverable. But, in most cases we are able to retrieve data that others could not.

What state is the data in when you recover it? For instance, if you recover a Word document, do you get the whole thing? Paragraphs? Sentences?

Our intentions are to get the dataset back to the user in the state it was when they were using it. Sometimes that’s not physically possible. Usually that occurs when there’s damage on the device itself that renders a portion of it completely inaccessible. If the data has been physically scraped off the platter and it’s just dust in the bottom of the drive, we’re not going to get that back—no one is. Our rule of thumb is to get back the original data set in its original form.

What’s your opinion of online backup systems (like Carbonite) and how difficult is it to resurrect information if one is backed up in that manner?

Most online backup companies have step by step instructions for restoring the backup set back to the customers system. It is a good idea to test the procedure ahead of time so that you know exactly what is required on your end to complete the restoration.

And how easy is it for DriveSavers to recover data if the online backup service suffers a loss?

If an on-line backup service needs to use our services we still have the ability to recover the data. Usually they will be utilizing a raid configuration to store data. We have an enterprise division that is dedicated to performing data recovery on multi drive raid systems.

Is it easier or harder to recover data from the new solid-state disks in netbooks and laptops than from traditional hard drives?

Any time a new technology comes out, we spend a tremendous amount of R & D on it to make sure that we can recover information from solid-state devices, and we are able to do it.

Data Recovery Standards

Security standards are nice; do they map to an ISO standard?

Since this information was on both the press release and the DriveSavers website, it didn’t seem necessary to ask it again. The standards, and the certifications, fall into several categories:

• ISO 5 (Class 100) Cleanroom Certification
• Cisco Self-Defending Network Certification
• SAS-70 Type II Compliance Certification
• Encryption Training Certification
• Manufacturer authorization

There have been numerous cases of recovered data being sold or released without the owner’s consent. You have other cases of Geek Squad employees making private copies of sensitive information when they repair hardware for a customer. How do companies like DriveSavers talk to this? “Trust me” only goes so far.

All the certifications mean you don’t have to take their word for it. In fact, the page listing them is entitled “Demand Proof.” In addition, according to Michael, “We perform background checks on all our employees. They have to sign a security policy; we have everything in place to inhibit that from happening. Only certified cleared engineers have access to the customers’ information. We hire the most qualified and credible people.”

Explain the standards for the SOHO user who doesn’t understand what those certifications mean:

An ISO-certified cleanroom increases the chances of a good recovery, because we’re not introducing any kind of foreign objects to the media as we’re going through the recovery process. So you’re going to maximize your bet right off the bat. Secondarily, we’re in compliance with the international technology control audits. We have everything in place to ensure that integrity of their data is not compromised while it’s at our facility. We monitor our facility and our network 24/7. We’re certified to handle any type of encryption recoveries and we have the manufacturer authorizations to be able to work on the devices themselves.

For an individual customer or small-business user, even though it’s your individual drive, you have to bear in mind that even though it’s your personal drive, there’s a high likelihood that there’s information on that drive that you don’t want shared. How many people use an accounting program? If you’re using Quicken or QuickBooks or any program of that type, your credit information is on that program. Your bank account numbers are in that program.

If you’ve set up your computer properly, it shouldn’t be able to be hacked at your house, but if you send it off-site, who do you trust, and why? Anybody can say they have a cleanroom; can they show you the certification? Anybody can say they have a secure network; can they show you the certification?

Here’s another classic example. How do other data recovery companies handle recycling customers’ drives? How many times have you seen something in the news about “I bought this drive on eBay and it had another person’s information on it when I got it.” If a customer sends us a hard drive and it is completely physically done, it’s of no value to them, it’s out of warranty from the manufacturer, and they don’t want it anymore, and they tell us to recycle it, we’re going to physically degauss the hard drive with a Department of Defense-approved degausser to render it 100% inaccessible before we recycle it.

That’s as opposed to throwing a whole bunch of drives in a pile and taking them to a recycling center. Those drives get bought in bulk and then sold on eBay.

The security criteria and protocols that we have in place are just as important when dealing with individuals as with corporations. You hear of innumerable instances of laptops with 50,000 social security numbers getting stolen. If mine was on that laptop, I would be upset. But at the same time, my social security number is on my hard drive, and it wouldn’t matter if it was one of 50,000 or one of one. A lot of people have a file that shows their passwords, or their PIN number for their ATM machine.

Special thanks to everyone on LinkedIn who provided these great questions.

So what happened? Someone—or something—erased all the data on the journalspace server. There were no backups. One of my Scary Statistics posts mentions that 60% of companies that lose their data go out of business within 6 months. Journalspace was forced to shut its doors much sooner than that.

The story has been all over the net—just look at my Backup Bookmarks posts over the past few days for a collection of links. Feel free to go read them for more details, or look at the journalspace blog. (Since they’re putting their domain name up for sale, that link may not be good for long.)

What I want to concentrate on here are two lessons we can all learn from this catastrophe.

RAID Is Not Backup

I once compared some form of backup or another to RAID, and some kind(?) person was quick to point out to me the error of my ways. Now that I own three different RAID devices (the Maxtor Shared Storage II, the Buffalo LinkStation Mini, and the Buffalo Quattro), I understand RAID a bit better. By using a “mirroring” array, RAID can help protect you from data loss. Because all data is written to two or more disks simultaneously, you’re still safe if one of those drives fail. That was the case with my first Maxtor Shared Storage II, Teras. One day the drive started making horrible clicking noises. Fortunately for me, I had Teras in RAID 1 (mirroring) configuration, and only one of the two drives inside the box was damaged. The wizards at Seagate/Maxtor were able to retrieve the data from the other drive for me without trouble, and I restored that data to the replacement MSS-II (named Teratides, which means “son of Teras”) fairly easily from the USB drive Jay sent it back to me on.

If you’re running a server, you definitely want RAID—the higher level, the better. Servers are generally on 24/7 and that kind of wear and tear makes it more likely one of the disks will go.

But RAID is no substitute for backup. It doesn’t protect against viruses, theft, human error, or natural disaster. Anything deleted from the first drive automatically disappears from all the mirroring drives. Journalspace had two drives in a mirroring (RAID 1) configuration, and no backups. They learned the limitations of RAID the hard way.

Don’t Count on Your ISP for Backups

Some hosting companies make backups of their clients’ data, and some don’t. Even if yours does, remember that it’s your data and therefore your responsibility to protect it. Make a backup onto your own computer. I don’t know what provisions the journalspace platform made for its users to back up their blogs, but one reason I like WordPress is the ease of backing up my blog content. Just install the WP-DB-Backup plugin and tell it how often to e-mail your backup to you. Restoring the data or moving it to another blog is easy.

Store Backups Off-site

There’s a possibility that what happened to journalspace was the result of sabotage by a disgruntled former employee of their datacenter. When you keep your backups in your office, anyone who has access to the physical location of your computer also has access to your backups. While you want some backups close at hand, you also need to keep copies of your most critical data somewhere else.

My heart goes out to all the people who have lost their jobs. I hope none of journalspace’s former customers goes out of business because of it. And I don’t want anyone who reads this blog to have to experience that kind of pain.

Back up. Back up. Back up.

We also fixed my father’s wireless router, so there’s wi-fi in his 45th-floor apartment again, but I had to use webmail for outgoing messages because RCN (Dad’s cable Internet provider) appears to block any outgoing traffic from non-RCN senders.

Anyway, I’m back home with another guest post for you, this one from Ken Colburn of Data Doctors.

Data Disasters: did you forget your mobile workforce?

A hard drive crashes every 15 seconds…

2,000 laptops are stolen or lost every day…

1 in 5 computers suffers a fatal hard drive crash during their lifetime…
31% of PC users have lost all of their files due to events beyond their control…

60% of companies that lose all their data will shut down within 6 months of the disaster…

The overall average failure rate of disk drives is 100% – all drives eventually fail…

And another hard drive just crashed while you were reading this.

Will one of yours be next? Are you prepared?

If you ask your IT department, they will assure you that the primary servers are being backed up every day and that an off-site data storage component is in place, so no matter what happens, the company is covered.

What most IT departments fail to recognize is that as much as 60% of a company’s mission critical data resides on hard drives that are not being backed up.

Your mobiles sales team, your CEO’s laptop, remote users or offices; the list can go on.

The assumption by the IT staff that all the users are following the company policy for backing up critical data is generally flawed.

In reality, getting 100% compliance from all users is virtually impossible because of a single hurdle; human nature. Everyone knows that a hard drive could fail at any moment, but no one thinks it’s going to happen to them, so they will do it tomorrow.

In providing data recovery services for over a decade, a pattern has emerged as more companies rely on computers; critical data is being lost on a regular basis.

The proliferation of the laptop computer as well as the increase in remote workers and even the digital camera (of all things) are primary drivers.

I regularly run into mobile salespeople throughout the country and in every discussion I hear the same thing: “I would be totally screwed if my laptop crashed or got stolen” and it’s usually followed by “my IT guys just don’t understand”.

Digital cameras have also increased the need for data recovery because digital images are not thought of as “data”… until they are gone! We routinely see a drive in for data recovery that has thousands of mission critical images on it that no one thought to backup or were so large that they did not fit into traditional backup procedures.

Even with the realization that their future is in jeopardy, statistically only 1 in 4 users will regularly back up their files. Why? It’s generally too technical or time consuming.

Another common mistake that some IT departments make is assume that if the critical data is being backed up and we can replace a laptop with an image of the corporate software, we have everything covered.

We routinely see customers pleading for help because they installed a special program that only they needed and no one took this into consideration during their disaster planning.

In a perfect IT world, everyone is using the exact same software on every remote or mobile system, but the reality for most is that no two computers are exactly the same.

Some of the biggest offenders of not following the IT department standards are upper management and they often times have some of the most mission critical data on their systems.

100% of all Data Loss is PREVENTABLE!

There are a number of personal backup solutions that IT departments should consider implementing as an additional layer for their mobile workforce.

We have been working with folks on backup procedures for long enough to understand some of the biggest roadblocks…users don’t know how to backup and even if they do, they don’t have any idea where on the hard drive this data resides, much less taking the time to actually do it.

The best chances for success and a huge time saver for the IT department for when (not if) a hard drive crashes is an automatic whole drive imaging system. (The expense of one data recovery will usually pay for 4 or 5 personal backup systems.)

If you can reduce the point of failure down to “can I get my users to plug this device in” your chances of success are much higher.

By eliminating all of the technical aspects of the backup process you can expect non-technical mobile and remote users to be much more successful in protecting themselves.

One solution is to install an automated imaging program that automatically fires whenever the external backup device is plugged in and/or setting a scheduler to automatically backup (and pester the user when it has not been done) to an external device.

Another great option for field personnel is an automated Internet based backup service. Once the client software is installed, it can automatically push copies of critical data up to a secured Internet server and be setup to pester users whenever it does not occur.

The bottom line on covering your bases is to really cover all your bases, so don’t forget your mobile and remote users!

External backup solution:
http://www.datadoctors.com/products/datavault

Online backup solution for businesses (Free 30-day trial): http://www.rdbup.com/partner/?id=datadrs

Last week TechTarget sent me a link to an “e-zine.” I don’t know why they called it that; there’s no sign that you can subscribe and get new issues. It’s essentially a white paper and probably the reason I got a phone call and an e-mail message from a hapless salesperson at ASEMPRA whose marketing department doesn’t know enough to put a “Don’t contact me” checkbox on its download forms.

Anyway, the white paper is called “New Tools for Better Backups,” and, like most white papers, it focuses on enterprise technology: deduplication, storage resource management, VM (that’s Virtual Machine) Backups. The kind of thing that gives most of the folks who read this blog a serious case of My Eyes Glaze Over.

But in the midst of these articles was a full-page ad for the latest installment from the Backup Trauma Institute:

First, if you’ve never visited the Institute for Backup Trauma, go check it out. This award-winning campaign for LiveVault’s Continuous Data Protection services appeared in April 2005. It makes two major points: how much trouble a company can be in without reliable backups, and how problematic tape is as a backup medium.

Three years later, Iron Mountain owns LiveVault and John Cleese is dispensing advice of dubious friendliness regarding some important questions about compliance, security, and whether there is, in fact, a mountain of iron. My favorite question is “How can I get our executives in trouble?” but my favorite answer is “How do I keep Mr. Wiggles from destroying electronic evidence?” That one expounds on some really creative ways to destroy a hard drive.

Even though Iron Mountain’s solutions are aimed at the enterprise and may not be immediately useful to you, the Friendly Advice Machine is an entertaining diversion—and one that makes me glad I don’t have to worry about compliance, discovery, and managing millions of e-mails.

The specific problem, however, involved an intersection of factors I don’t know enough about to explain: Vista x64, Office 2007′s .docx format, and Mozy, or rather why Mozy didn’t seem to want to back up .docx files from a Vista x64 PC.

I don’t use Vista, and don’t plan to, at least not for as long as I have this particular machine. The Ur-Guru says that the x64 version isn’t bad now that Service Pack 1 is available, but the overall Vista adoption rate is so low that the only pressure to “upgrade” comes from Microsoft. (Even the Ur-Guru only has it installed on one system, and that’s only because the software he develops has to work on it.)

As for Office 2007, while the Ur-Guru has been using it happily for some time, none of my clients use it, and I would be creating more problems than I was solving if I switched now. So I don’t know much about the new .docx format for Word files which Office 2007 for Windows shares with Office 2008 for Mac, except that it’s based on XML. And while I found a number of articles and blog posts talking about the difficulty people with older versions of Office have opening .docx files, I didn’t find anything that would explain why uploading them through an online backup service should be a problem.

Likewise, I found some “don’t use Mozy” stories from a few dissatisfied customers who had experienced file corruption or other problems, I didn’t notice anything specific to Vista. So that was the end of that idea.

This morning, however, while catching up on my C|Net newsletters, I saw an item in Gearlog that I couldn’t pass up mentioning: EDR’s Hard Drive Crusher, billed as “a new spin on destruction.”

Though this is by no means a data security blog/e-zine (blogzine?), I have mentioned before that if you are giving away a computer or a hard drive, you want it thoroughly erased. There have been special shredders for CDs and DVDs at least since I wrote about destroying outdated backups in 2003. And computer recycling facilities have powerful electromagnets designed for completely wiping the data off any magnetic drive.

The Hard Drive Crusher is not a home-office solution. For one thing, it weighs 85 lbs. For another, it costs $11,500. Even the Ur-Guru doesn’t go through enough disks in a year to make it a sensible purchase. But it’s the kind of thing your local electronics recycling center or data protection service might want to invest in, and let you use for a small fee if you don’t think a magnetic wipe or repeated overwriting of the drive is sufficient.

And it has to be a pile of fun to operate if you’re suffering from computer-induced frustration.