A couple of years ago, an associate of the Ur-Guru’s in Europe bought a 1 Tb Buffalo DriveStation Duo to back up his business data. Last week it stopped working. “I’ve been using it in a RAID 1 array for the past two years, and it’s just failed,” he wrote. “Three different Windows PCs here see it as unallocated space.”

Bad news. And while Buffalo offered to replace the hardware, since the DriveStation was still under warranty, there was still the question of the data.

You see, our European colleague had made the mistake of confusing RAID with backup. The DriveStation was configured in RAID 1, meaning the two 500 GB drives inside mirrored each other’s content. If one drive failed, the data would be safe on the other drive.

But if something else happened to the data—corruption by a virus, say—the damage would also be duplicated on both drives. And in this case the problem was not that awful grinding clicking noise that indicates a dead drive, but something else, perhaps a problem with the circuits in the box that tell the drives how to communicate with the PC. And there was no backup of the backup.

The down side of these convenient, reasonably-priced, consumer RAID boxes and NAS drives like the Buffalo DriveStation or the Quattro and the Maxtor Shared Storage II that I’m looking at on my own computer table is that, unlike traditional enterprise and power-user RAID, you can’t just swap out disks yourself when one goes bad. (With the MSS-II you really can’t; I couldn’t have opened it if I’d tried. With the Buffalo models you just aren’t supposed to.)

And the problem with sending your entire drive back to the manufacturer for repair or replacement is the vulnerability of your data. In this case, there was proprietary business data on that drive, but almost everyone is going to have some kind of data in their backups that shouldn’t fall into the wrong hands.

Even if you trust the manufacturer of your drive to restore your data without actually looking at or copying it, there’s the problem of getting the drive to them. There have been several big scandals involving stolen backup tapes with financial information on them. The tapes don’t get stolen while they’re at the banks, or while they’re at Iron Mountain. They get stolen out of the trucks while they’re in transit. You don’t want your drive lost or stolen while it’s on the way to the data recovery specialists. You especially don’t want it to show up on eBay or end up in the hands of your competitors.

So RAID Troubles in Europe (as Dear Abby might call him) got permission from Buffalo to remove the drives from the case in order to attempt to recover the data. This meant that he had to go out and buy a dock to mount the drives in. At first they showed up there as “unallocated space,” too, which was baffling. The Ur-Guru suggested some software tools that might help him. (I know, I know—DriveSavers and other data recovery professionals say you should never use them.)

In the end, it was a product called Active@ UnDelete that did the trick. According to its website, the product does a good deal more than file recovery, including making system partitions. The particular feature that must have attracted R.T.E. is “Damaged RAID data recovery and reconstruction.” There was much rejoicing in the offices of R.T.E.’s business when this message appeared at the end of the restore process:

Much chastened by his experience, R.T.E. has realized that he needs to develop a real backup plan, preferably automated, with more redundancy than just RAID. Your backups need backups, and you need to make them consistently.

But you should also think about what will happen if you do need to get a drive replaced or send it in for data recovery. Delivering the drive personally would eliminate one level of risk, but that may not be feasible. It’s possible, of course, that if you can’t get data off your drive, thieves can’t, either. But you do want to work with data recovery and computer repair people who have a reputation for integrity, security, and trustworthiness.

And if you don’t need to get your data off the drive, then you should have the drive degaussed before recycling it. (That means subjecting it to really strong electromagnetic fields that completely erase any data remaining on it.) There’s no reason your business should be the next one making headlines because of data leaks.

Disclosure: this post contains Amazon affiliate links. I bet you’re smart enough to figure out which ones they are.

As we were driving back from Palm Desert to Pasadena earlier this month, the Ur-Guru pointed out a problem with the current craze for cloud computing. No, not the problem I’ve talked about before—slow broadband or no broadband at all making it impractical if not impossible to store or transfer large files online—though that problem is a long way from being solved. It’s the reason the “prosumer” photographers with their 10-megapixel cameras are all lining up to buy the DroboPro. The Ur-Guru will be going home with about 70 GB of photos from this trip alone. (The RAW files are sitting on my Maxtor Shared Storage II drive until he tells me it’s safe to delete them; he’ll have more than a few backup routines to run when he’s back in his world-famous home office.) You try uploading that over your home cable or DSL line. I dare you.

But in spite of the bandwidth bottleneck in getting our data into the cloud, more and more of us are either storing our documents and photos online or creating them there in the first place. It’s not just the online backup services, but the photo sharing sites (the Ur-Guru likes Flickr, whereas my mother prefers Picasa), the social networks, the blogs, and the file sharing and collaboration tools like Google Docs, Box.net, and Huddle Workspaces. And then there are all those Kindle books hanging out on Amazon’s Whispernet servers waiting to be downloaded—and maybe sucked back up again when Amazon finds out it’s made a mistake. (That last episode points out another problem with online storage: if your only copy of the data lives on someone else’s server, is it really yours?)

But there’s a down side all this online storage, as connected as it makes us all, and as useful as it can be to get our critical data off-site in case of theft, fire, or natural disaster. The data you back up online, the photos you post to Flickr, the PDF of your resume that you just dropped into Box.net so it would show up on your LinkedIn profile, even the website you can’t conduct your business without—all of those live on servers in data centers. This is the good news and the bad news. Your website and mail server have to be up and running 24/7, of course. That’s just as true if you run them out of your home office as if you hire a web hosting company.

But what about the rest of it? Sure, some of it is there because we want anyone to be able to see it at any time, whether we’re around to show it off or not. That means it has to be on a computer that’s using power. But what about the data that really is just backup? Or even reference or archival material? My colleague Eve Abbott says that 80% of paper that gets filed is never referred to again. (And after a year has passed, it goes up to 90%.) I’m pretty sure that statistic holds true for digital documents, too. It’s easier to search through our electronic detritus, but if you go back and look at all the articles and white papers you’ve saved over the years, I bet you’ll be amazed at how many of them there are—and how few you can remember anything about.

That means that not only don’t we need immediate access to a lot of what’s going onto those 24/7 RAID-whatever servers in the high-security data centers with the enormously high power bills, we probably don’t need it at all. Most backup software is configured for lazy people: it backs up everything, automatically, so you don’t have to think about or do anything. Anyone who has experienced data loss is likely to err on the side of caution anyway, so unless there are severe limitations on that online storage space and harsh financial penalties for overrunning the quota, very few people are going to sit down and triage the contents of their hard drives before sending their data up into the cloud—never mind afterward.

The Ur-Guru was all set to go look up statistics for the cost of operating a data center so he could figure out the math of exactly how much it really cost (and how much power it used) to store everything online. Me, I started avoiding math as soon as I left high school—once was enough to take calculus. Besides, I don’t think we need the actual figures in order to reach some sobering conclusions.

Data stored on tape, CD, DVD, or a disconnected hard drive requires no power at all. The storage media aren’t particularly green—and never have been—but they aren’t using up electricity. When I shut down my computer, my USB drives power off. The Rebit that I use to back up my netbook spends most of its time unplugged on the computer stand, because the netbook itself spends most of its time there, except when I’m traveling or out at a business meeting. There’s no need to have it running all the time. Likewise the four Western Digital drives that the Ur-Guru brings with him on his visits wait patiently, unpowered, for him to return with a full SD card and offload new photos.

Because these external drives are not spinning all the time, we can expect them to live much longer than the drives in the RAID boxes in data centers. Those are designed to be easy to replace without interrupting the machine’s function or causing any data loss, and it’s a good thing, too: constant use wears them out. They have to be replaced by the boxload. That means more toxic electronic components to dispose of than if you were backing up at home.

Is there a way to get the benefits of online backup and storage without increasing the need for power at a time when we’re all trying to save energy? Possibly. In most cases, it’s not really necessary for backup to be continuous or for restoration to be instantaneous. In theory, then, data could be stored on removable drives and handled in much the same way the automated tape libraries in large corporation handle tapes. Retrieving that data would take a short while, since the appropriate disk would have to be found and inserted into a drive bay, but restoration would still be faster than with tape. But to utilize this kind of storage, service providers would have to invest in entirely different facilities and infrastructure. FalconStor could see a real boom in its sales of Virtual Tape Libraries if the world of online storage decides to go green.

I don’t think you’ll see everyone and his brother rushing into that, however. Setting up a VTL requires serious capital outlay, much more than renting space in a data center. That cost would inevitably get passed on to the user. It would take some serious dedication to reducing your carbon footprint to choose an expensive service that doesn’t use power just to store your data over a low-cost—or even free—service that does. Especially when you could be equally “green” by just making your backups at home and storing the drives in your safe-deposit box.

The Ur-Guru thinks the real solution probably involves a storage medium that hasn’t been invented yet, but says that all conversions to alternative power sources and more efficient systems require an outlay of capital. If we take that as given and are willing to invest in some R & D, then it would probably be possible to reduce the world’s collective power bill for data substantially even before we invent that new storage medium:

My idea for a VTL was not about using disks as tapes and having robotic arms load them, but rather using entire arrays of disks and having a robotic arm move entire sections in and out while some Drobo-like software makes duplicate copies when needed, caches data for users who are using restore more often, or use some statistic saying which user might be restoring in the next x amount of time, etc. It’d be quite a cool thing to work out and make work but with logic applied you could still design it.

What I want to take out of the equation is “physical storage units allocated to x users” and turn the data into a dynamic movable chunk so the data for one or more users can “move around” in the storage units that make up an entire grid, offloading any disks that contain data that is hardly ever accessed or updated and basically powering them off.

Maybe instead of robotic loader arms some kind of electronic solution might also work where entire arrays can be powered down or run on low power, only to be powered up when needed (to move “offloaded” data to “cache”).

That sounds like just the sort of mathematically complicated problem that the Ur-Guru would enjoy and that I went into the humanities to get away from. But it’s an intriguing idea. Any enterprising entrepreneurs out there interested in making it work?

Okay, be honest: who forgot to back up last week? Anyone?

That’s the “Does anyone actually need me?” question. Because if you’ve all got your backups automated by now, does it matter whether I actually write this reminder? Do people even notice if I don’t produce one?

Not that I’m trying to let myself off the hook for not producing a post last week, mind you, but no one on my less-than-enormous mailing list (backups are never going to win the e-zine popularity contest) wrote to ask where I was last week. (I was at the Podcast Expo in Ontario, California, having a great time but with scarcely time to check my e-mail, never mind compose a Backup Reminder.)

Just because I was on the road doesn’t mean I didn’t back up. I took my new FreeAgent Go drive with me in a pocket of my laptop case, and it worked just fine. The Ur-Guru took all six of his Western Digital XHDs–and a lot of pictures, which you can see on Flickr if you search under the tag “newmediaexpo2007.” We heard more than a few people mention what they did to back up their photos and their audio and video files, but mostly it came down to DVDs (which they were trying to get away from) and external hard drives. Nothing really new there.

Nothing new from Seagate about my dead Shared Storage II, either. Oh, I had a bit of back and forth with the tech support team, who were baffled by the serial number I gave them and asked for a photo of the label and a proof of purchase. Um, guys? My point was that I didn’t purchase it at all. After I explained that and sent them everything they asked for, they decided to pass the buck to another division and told me they’d get back to me, which they haven’t.

Suggestions for more reliable NAS drives to replace the late Teras are welcome.

I also did a comparison test of Norton Ghost 8 versus TrueImage 9. The Ur-Guru has been using TrueImage on all his systems (actual and virtual) for some time now, and he suggested I try it. First I did a Ghost backup of my hard drive. It worked the way it always does, and took a good while but performed as expected. The TrueImage backup projected that it would take somewhat less time, but I got a strange error message in the middle, even though everything thereafter appeared to work normally and TrueImage told me it had completed the backup successfully.

“I wouldn’t trust that backup,” the Ur-Guru said. Well, no, not if I had to try to restore my whole system from it. So I’ll stick to using Ghost 8 until I find something that works better.

But why TrueImage should work for him–and many others–and not for me, I haven’t a clue.

I’ll conclude on a humorous note. The Ur-Guru and I attended the Halo 3 launch party in Silicon Valley on September 23rd. The only game I play on my computer is Solitaire, but I knew the guy organizing the party from the National Youth Leadership Forum on Technology.

In addition to the gaming competitions, Microsoft was showing off some of its other projects, including the new improved Hotmail and the Silverlight/Popfly combination. We were talking to the Hotmail guy and I asked him what they did about backups.

There was a long, uncomfortable pause. “I know the answer to that question,” he said, “but I’m not sure I’m allowed to tell you.”

Last week I mentioned that I’d seen an intriguing post on Lifehacker entitled “Copy and Paste your Entire Hard Drive with Two Clicks with GParted.” This sounded rather improbable to me, but I wanted to check it out.

There was a small problem with doing so, because the link on the Lifehacker site is broken, but I figured out how to fix it and was able to go over to Sourceforge to download the GParted Live CD .iso file. (“GParted” stands for “Gnome Partition Editor,” but that’s not of particular interest to the non-geek, and in particular the non-Linux geek.)

At first it seemed a bit strange to be using a Linux tool to back up a Windows drive, though at second thought it actually makes sense. I automatically distrust anything that tries to back up the Windows operating system from within Windows, because the system state keeps changing. I normally make drive images using Ghost 8 after booting my machine from the non-Windows Bart-PE CD. (PE stands for Pre-Environment, another thing you probably don’t want to know about. Bart is just the guy who assembled this useful CD.)

Assembling the Bart-PE CD was beyond me; the Ur-Guru made mine. But the GParted Live CD is ready to burn: just start up your CD-burning software, open the .iso file, and burn. Presto: a bootable CD.

So I shut down my computer and booted from the CD, and after selecting the appropriate language/keyboard layout, I was in GParted and it was showing me the partitions on my C drive. Not that it actually said “C drive,” mind you. It said “dev/hda1″. “Dev” stands for “device,” and there’s a menu at the top where you get to choose devices. My choices were hda1 (my C drive), hda2 (my D drive), and sda1 (the FreeAgent Go drive). My Maxtor Shared Storage II network drive was conspicuous by its absence.

That was the first problem, because if you copy a partition with GParted, you have to copy the whole thing, and it doesn’t get compressed. I don’t have 80 GB free on either the D drive or the F drive. So there was no place to put the drive if I copied it.

And even if I’d been able to see the network drive (which has about 200 GB free at the moment), copying and pasting with GParted doesn’t just fill in empty space, but reformats the whole drive, thus wiping out anything on there. (It does give you a big warning prompt to prevent you from doing so.)

So yesterday I pulled out my not-yet-recycled X drive, which was completely empty and is an 80 GB drive like my C drive, hooked it up to the USB port, and rebooted with the PArted Live CD. I was then able to select the partition representing the C drive (hda1), hit the “copy” button, select the partition representing the X drive (sda2), hit the “paste” button, and apply the operation.

I was a little surprised that I had to do something more than hit the “paste” button to start the drive copy, though when you’re partitioning drives and potentially removing all the data on them, it’s not a bad idea to have to take an extra step or two. But even without that, it’s definitely more than two clicks, because you have to navigate between partitions.

Still, once you get past the unfamiliar-to-Windows-users (and probably even less familiar to Mac users) interface, it is a straightforward procedure. I hit the “apply” button and off it went.

Copying 80 GB is not a speedy activity even over USB 2.0 hi-speed, but I was about to leave for a client appointment anyway. I started the copying around 9 or 9:30 AM, and when I got home at about 1:30 PM it was finished. I exited GParted, selected “eject and reboot” from the shut down menu, and took a look at the X drive in Windows Explorer.

It was all there: everything on the C drive, including the label “local disk.” I’m presuming I could boot from the X drive, now, as long as I set the BIOS on my machine to look for a USB device before checking the local hard drive. And I could use GParted again to copy the partition onto a new machine—though it would have to have identical hardware for me to be confident that there wouldn’t be weird issues with drivers for things like sound and graphics cards that would mess it up.

I’m not likely to start using GParted instead of Ghost, if only because I can do the Ghost backups onto my network drive and keep several of them there. But it’s free and it requires only a moderate degree of geekiness to use. It also handles just about any conceivable file system, so you should be able to use it on Macs as well as PCs and Linux boxes. And restoring files from the GParted backup doesn’t require any proprietary software. (That’s the good news and the bad news, as there’s no way to encrypt the files and anyone could take that drive and have access to everything in my machine.)

Two clicks is definitely an exaggeration, though. Perhaps I should talk to Lifehacker about truth in advertising.