Long term computer storage - Archival backups

[skp51443]

There have been a few discussions about long term storage of computer data here so I thought this might be of interest.

 

I was browsing around and saw an Mdisc drive on sale, it looked pretty interesting so I Googled up a few pages on them and they seem like a realistic long term solution to data storage. Media is a bit expensive for routine backups so for that DVD or BlueRay is still probably the way to go.




Wiki (snip)

M-DISC's design is intended to provide greater archival media longevity.[1] Millenniata claims that properly stored M-DISC recordings will last 1000 years.[2] While the exact M-DISC are a trade secret,[3] the patents protecting the M-DISC technology asserts that the data layer is a "glassy carbon" and that the material is substantially inert to oxidation and has a melting point between 200°-1000 °C. [4][5]

A stress test was performed by the Department of Defense of the media, proving at the very least that M-Disc DVDs and Blu-rays are more durable than conventional DVDs and Blu-rays. "The discs were subject to the following test conditions in the environmental chamber: 85°C, 85% relative humidity (conditions specified in ECMA-379) and full spectrum light"[6][7]

Conventional DVD-R and BD-R use recording layers of organic dye and separate reflective layers.[8] M-DISC uses a single inorganic recording layer, with a higher powered laser. Thus, the M-DISC physically alters the recording layer, burning a permanent hole in the material. Recorded discs are readable in conventional drives.

Mdisc Site

Amazon Page

NewEgg Page

[Mark and Dale Bruss]

The longevity of the media is not as much of a problem as the ability to read the media in the future with changes in operationg systems and hardware.

 

I remember the issues with bank archives on 7-track tape that was readable, if, a big if, a 7-track tape drive could be found.

 

Archive data should be change about every two tech turns of media.

[skp51443]

There have been a lot of folks that pulled up CD or DVD disks they thought were backing up their data on and found them unreadable so media life is actually a problem. Many more and worse stories about magnetic media, disk or tape too.

 

Buying high quality optical media and burning at a low speed followed by a scan for un-correctable and correctable errors on a different drive will help you avoid that but are no guarantee. The lifetimes of the CD, DVD and probably BlueRay dye layers vary a lot, even on quality media with many factors from the chemical makeup of the dye, disk construction, pre-use storage, burning drive, burning settings and post-burning storage all having an impact.

 

Burn a CD/DVD/BlueRay backup you'll toss in a year and verify the burn and you are probably 99% safe, need it in five or ten years you are a bit iffy and might be well served by having two copies and hoping they have errors in different spots.

 

The whole idea behind the Mdisc format is to eliminate the dye layer and the issues it has, still Mdisc is open to other problems but a whole host of them are eliminated along with the organic dye layer.

 

 

The worry about the media format changing is a valid one but for optical media I'd not worry that CD or DVD will go away any time soon. Other options like the BlueRay and Mdisc have their place but I just don't see them replacing these two given their capacity and price points in the near future. Even if they do loose ground many drive makers will be building backward compatible (readers at least) drives since it is a good selling point and only adds pennies to the hardware cost of the drive.

 

 

Mag tape is another issue as it is a small volume market and most folks in it are looking at high end/price users that are willing to pay a lot for fast small storage and many that do need access to old data have little heartburn in doing a migration to a new format when it hits their price/performance needs. I used to see a lot of old reel tape archives that hadn't been updated and that were actually worthless or nearly so since they hadn't been exercised, maintained or recopied when early failures were detected.

 

Non-reel formats are a bit more scary with formats coming and going far too rapidly for most folks serious about long term storage. I can't even keep track of the small form factor tape drive formats I've used that are long extinct, not to mention UltraDisk and such that came and went in just months.

 

 

 

Many years ago now I needed a huge amount of data from a library that was only available off IBM 1600 reel tape or from printed books. I bought a PC connected drive and a shiny new PC/AT to run it, found that almost every tape was in bad shape from shedding media, print-through and just plain unreadable blocks. Some I rescued by reading a few blocks of tape, cleaning the head and reading a few more feet, others I just had it write zeros for any bad blocks and hired a crew of typists to retype the missing data and proof readers to check everything. The librarian was horrified when I showed him what the issues were, he tried reading tapes I hadn't gotten to and saw the same issues. My contractor for the typing task got an extension to the contract to do the data I didn't need (and have to pay for) while I got told to be nice and give the library the tape data I had fixed. At least I got them to pay for the new tapes and the contractor gave me a coffee cup and a receipt to prove it was worth less than five bucks. Ever since then I have been wary as a cat around a rocking chair about backups and it has saved me much grief and the loss of irreplaceable information.

 

Still the champion for long term storage has to be 8 bit mylar tape, (punched holes not magnetic) that will be readable forever excluding fires, if nothing else you can make a reader in an afternoon for $20 worth of parts! :-)

[lappir]

Is Computer storage all that important to the average person? You NEED 7 years for the IRS and you may WANT a lifetime of photographs. I would hope that anyone doing critical work on a personal computer has a back up of such work "at work". I will have to send a copy of Stan's chart to a friend of mine who recently told me that going to a "Hard Drive" was the longest lasting back up available currently. He is an Avid photographer and spends hours on photoshop making sure he is happy with the image. At one time he backed up everything on CD discs, then flash drives and now an external hard drive. Me, I try to keep my tax information backed up using the Neat Scanner backup, but have never checked it for validity. Guess I should do that some day. Am also glad when I wanted to purchase a 1 TB external hard drive that they were out of stock. It appears a DVD back up or Flash drive (my current) will last much longer, and hopefully as long as the IRS can go back.

 

Rod

[vermilye]

I use a pair of external drives to mostly store photographs. After a bad experience with a RAID set up, I now simply load the images to each drive. My solution has been to purchase new larger capacity external hard drives when my current ones are around 80% full. I copy the old drive to the new one, and the old one goes on a shelf. I started with an 80MB SCSI drive (which I have to admit, I can no longer access since I don't have a SCSI port), & my last purchase was a pair of 3TB USB drives, which started out a little over half full. Maximum drive capacities have kept ahead of my total stored data so far.

 

I do check the older drives every once in a while, and, with the exception of the SCSI drives, the USB2 & Firewire drives are all still readable. I even have some 1996 recorded CDs that are still readable (Kodak Infoguard disks with a projected lifetime of 100 years). On the other hand, I have a few recordable audio CDs that the gold foil is pealing, and are unreadable. These were recorded on an early Yamaha audio recorder - each blank sold for $40.00!

 

While media does fail, it is far more probable that you will find that unless you pay attention and actively archive your data, finding the hardware to read tha date is more of a problem. Any archiving must be active, i.e. with every new change on hardware, you must insure the readability of your archived data. I try to make a point of this in photo workshops I teach by holding up an 8" floppy disk & asking how many could read one. So far, no takers!

[BooneDocks]

Thanks for this thread Stan! Has me thinking I should backup my picture libraries to M-DISCS.

[skp51443]

Hard drives for long term storage are tempting, you get a lot of storage cheap and they are pretty reliable. The connector types change from time to time but not fast enough to be a real issue. If you see an interface format problem coming up it is easy enough to migrate to the newer format and not really that expensive compared to other issues.

 

Hard drives are getting a bit iffy for my comfort though, while motors, bearings and electronics are improving they are reducing the magnetic domain size and going to overlapping tracks and such in search of more data per platter, there is no real good data on lifetimes of the newer recording formats and less information on recovery services should the drive hardware or electronics fail and you need it opened and the data salvaged. Older, lower tech drives might prove significantly safer long term.

 

Drives used regularly do benefit from the SMART drive monitoring utilities but for one fired up on a monthly basis the firmware just isn't collecting enough data to provide the level of safety I want for stuff that I do not want to go missing. It is not surprising to pick up a hard drive that has been on the back shelf sitting un-powered for a month or three and find that it will not spin up. You can usually get it to start with a bit of manipulation, maybe even get it to start again the next time but the risk of nothing happening when you flip the switch is there and growing each time. Same for drive electronics failures, sometimes a good connector cleaning can get you working but sooner or later that is going to come up short too.

 

Multiple hard drives and discarding any that exhibit any unreliability after recovering the data is probably the safest option but since the drives are re-writable there is always the chance of a glitch, bug or malware doing something nasty and it could happen to your whole backup drive collection if your luck is running low.

 

I really like write-once (WORM) type drives, an appendable CD, DVD or other optical disk format is a bit riskier as you are re-writing the index information with each append but with the right tools recovery is possible, safer is to not burn appendable format discs and just use more discs in write-once mode.

 

For now I'm sticking to high quality DVDs burned at low speed and carefully scanned for recoverable errors but I'm seriously looking at Mdisc as a better option. I'll watch both the reviews and price of drives and media and if or when it looks sweet I'l jump on board.