Twice in the past month people have asked me what the “best format” is for permanently archiving data that they want to keep forever, like family photos and tax records. Sadly, the short answer to that question is that there isn’t a “best format”. Technology keeps changing, and there’s absolutely no way to predict which current technologies (if any) will still be around in the next 50 or 100 years. Things can change an awful lot even in 20 years: somewhere in storage I have a box of 5.25″ floppy disks from my old Apple II+. If I suddenly had a desire to see what’s on those disks, I’d have my work cut out for me: while it’s certainly not impossible to find a 5.25″ floppy drive these days, it’s not a trivial matter, either. And 10 years from now, I reckon it’d be near impossible.
There are dozens of different ways to store data, and each one has their pluses and minuses. Magnetic media (like hard drives, tapes and proprietary disks like Zip disks) can often store huge amounts of data; however, magnetic media are also susceptible to damage from environmental issues (moisture, heat, shock, magnetic fields) and are also the most mechanically complex of the various backup types (and thus, prone to failure). The various types of “flash memory” like USB drives, Compact Flash (CF) and Secure Digital (SD) cards are renowned for being robust – indeed, the Internet abounds with stories of CF cards being run over by cars or washed in a jeans pocket and surviving. However, until only recently flash media had somewhat limited capacity, and flash still suffers from limited write-cycles and the unknown of future support.
So this leaves optical media like CD-R and DVD-R discs. Commercial CD and DVD discs (the ones you buy from Sony Music or Microsoft) are “pressed” much like vinyl records. A “master” is made containing the various “pits” on the disc (much like a record’s grooves) and thousands of copies are “stamped” from that master. CD-R and DVD-R technologies work by using a layer of dye which is heated by a laser to mimic those pits. It works well, but there are certainly some things you can do to make sure that your CD-R or DVD-R discs last as long as possible:
1) Buy Quality. As mentioned above, “burnable” media rely on a dye layer which is burned with a laser to mimic the “grooves” of a commercial disc. If the dye fades over time, the contents of your disc will slowly fade away and become unreadable. To keep this from happening, buy only quality discs from reputable manufacturers. Unfortunately, this is more difficult than it sounds. Major brands like Verbatim and Fuji don’t make their own media. They just slap their logo on discs made by a third-party. What’s worse is that they’ll use whichever disc manufacturer has the best price at the time, so you have no idea if the Verbatim discs at Best Buy are made by the same company as the Verbatim discs at Office Depot. Some batches might be made from top-notch materials; others might be of barely passable quality. As a general rule, if the package says “Made in Japan”, you’re OK; if it says “Made in Taiwan” or (worse yet) “Made in Mexico”, put it down and try to find a Japanese one. This is because almost all blank Japanese discs in the US market are made by Taiyo Yuden, world-renowned as the “Mercedes-Benz of blank media”. If you have some data that you’re deadly serious about keeping for as long as possible, you can skip searching local stores and buy Taiyo Yuden discs directly (here’s one online retailer just for an example). You’ll probably find that Taiyo Yuden discs are more expensive than the discs you get at Office Depot, but quality media is something you should *never* skimp on. And to that end, never, ever, ever buy the generic “100 blank CDs for $5” bundles that office supply stores like to run in their Sunday ads. Although these discs will probably be fine for short term use – like moving huge files from office to office, or sending patches to customers – these discs will almost certainly fail in the long run.
2) Use DVD-R if you can swing it. All types of compact discs – be they audio or data, commercial or homemade – have one very annoying “feature” – their data layer (the part that holds the information) is located directly underneath the label that you write on. This is why all blank CD discs say to never use a ballpoint pen to label a CD – it’s very, very easy press down too hard and corrupt the data layer, destroying the CD. DVD discs are more like a “sandwich” in that their data layer is protected on both sides by a thin layer of plastic. And aside from being more physically robust, DVDs can store a whole lot more data on them (4350MB vs. 700MB for a CD).
3) Burn at slow speeds. I’m not sure that I totally believe in this one, but you’ll find this little bit of “wisdom” all over the Internet. Basically, the theory is that burning at slower speeds reduces the number of possible errors in the burn. On the face of it, it makes sense, but it still smacks of an urban legend to me. Having said that, if you’re burning something really important to disc, why not burn at a slower speed? The only thing it’ll cost you is time.
4) Don’t encrypt it. It may be tempting to encrypt sensitive information like financial records before burning them to disc, but I see two possible problems in doing so. For one, security relies on your passphrase (password). If you encrypt some important stuff and die before you can tell someone your passphrase, your data is gone forever. Well… probably: chances are that computers will be powerful enough in 50 years to crack it in a reasonable amount of time, but I say “why take that chance?” Secondly, you run the risk of whichever encryption scheme you use becoming obsolete in the near future. Older schemes (like DES) are already being phased out, so there likely won’t be any software in 20-30 years that will be able to decrypt data encoded with it. If security and privacy are your concerns, I suggest physical security for archived discs, like a safe or safety deposit box at a bank.
5) Store it correctly. Put the burned disc into a jewel case and store the case in a vertical position in a climate-controlled environment. Remember, the burned disc’s two main enemies are heat and light, so store the discs in a cool, dark place – a bank’s safety deposit box is perfect for this. Also, consider making additional copies and storing them in multiple locations; keeping a backup of important household data in your house is pretty silly. Fires, flood and theft can make that backup disappear instantly.
6) Make copies. Every few years, take the copy out of storage and make a copy of it. Discard the “original” and keep the new copy. Be sure to use your CD burning software’s “verify” feature to make sure that the copies are identical. The anal retentive amongst you might consider manually running MD5s against the original and duplicate.
7) Keep abreast of things. You don’t have to read dozens of websites or computer magazines – just ask your favorite geek every so often if things are changing with your backup media of choice. It’s unlikely that optical media will go away any time soon, but improvements in lasers (and new formats like HD-DVD and Blu-Ray) could make the discs you burn today obsolete. There is almost always a “transition period” where both old and new technologies are supported. During this time you should have no problem upgrading your backups to the newer format.
And lastly, let me make one thing perfectly clear: I’m talking about home users here, not corporate ones. So please don’t deluge me with emails asking why I didn’t suggest tape – which, for most corporate types is the “holy grail” of backups. I *hate* tape drives, backup software and the tapes themselves. If I can use any other technology, I will. Besides, troubleshooting Backup Exec is hard enough for IT grunts like me – I sure as hell don’t want to inflict that on casual users.