So I'm looking at archival storage options. I don't mean years. I mean decades. Many decades. CD media is good for 10 years, under good storage conditions. SSDs tend to start degrading on that same timescale. Data on regular hard drives may last longer, but then you've got to worry about the lifetime of motors and bearings. Lubrication. Ideas? M-disks are a possibility, though they've gotten pricey. Also, one can complain that optical media in general will be obsolete in a few decades, but there will ALWAYS be archival readers around somewhere. I can find someone to read a floppy disk right now. Suggestions?
archival storage options
- Thread starter Dlass
- Start date
Mercutio
Fatwah on Western Digital
The thing you are looking for is called LTO. You need to understand that you will need to buy multiple drives and that you will also want to preserve both a working PC and the relevant software to work the drive(s) you've chosen. LTO 5 is highly affordable for 1.5TB/tape and secondhand drives that can be had for under $200. Tape drives in active use tend to get worn to crap, but people who buy LTO usually aren't buying just one of them, so there are also many sitting around that just never get used.
Tape can sit in dark, climate and magnetic-controlled conditions and still be readable decades later. It's the best option. You just have to take the hit to buy in to the format and media. There is no reason to fool around with any optical format and while it's possible to find a mechanical drive that might spin up at 20 years old, that's nowhere near as likely as using a tape.
Tape can sit in dark, climate and magnetic-controlled conditions and still be readable decades later. It's the best option. You just have to take the hit to buy in to the format and media. There is no reason to fool around with any optical format and while it's possible to find a mechanical drive that might spin up at 20 years old, that's nowhere near as likely as using a tape.
Wow, that's kinda pricey, and I need to get multiples? Geez Louise. Also, to the extent it goes obsolete, there will be fewer of them around in the distant future. Also, LTO capacity is tens of TB. I need a few hundred GB or so, so it's kind of overkill. I'm thinking that BDXL M-disks might be the way to go, though they are also not cheap. They are advertised to be "forever" media. I mean, if I want to archive stuff in the long term, I really don't want to have to rewrite every decade or so.
sedrosken
Florida Man
Tape drives have pretty much always and forever been for the cost-no-object segment of the population, usually businesses who can use it for a tax write-off. That said, LTO5 is quite affordable as far as that technology goes. If you're serious about maintaining your backups, if your data is truly irreplaceable, then the cost of entry is just the cost of entry -- if you don't have multiples, you have no redundancy for the case of physical failure of either the drive or medium.
I personally view M-DISC as a marketing ploy. I don't rightly think they last as long as they say they do, and I don't know how they'd even go about testing their claims. I'm skeptical that anything basically made of oil won't just turn back into it within a few decades one way or the next, but that's a problem with most media. I don't personally think that optical media of any sort is a forever media, certainly -- we're already having issues in the retro space with CD-ROM drives and their old, tired laser mechs being unable to read anything without at minimum an intensity pot readjustment. I just don't think you'll be able to write an M-DISC now and expect to read it in anything in, say, fifty years. Backup plans go best with a plan to refresh them every so often and move them to newer media.
I personally view M-DISC as a marketing ploy. I don't rightly think they last as long as they say they do, and I don't know how they'd even go about testing their claims. I'm skeptical that anything basically made of oil won't just turn back into it within a few decades one way or the next, but that's a problem with most media. I don't personally think that optical media of any sort is a forever media, certainly -- we're already having issues in the retro space with CD-ROM drives and their old, tired laser mechs being unable to read anything without at minimum an intensity pot readjustment. I just don't think you'll be able to write an M-DISC now and expect to read it in anything in, say, fifty years. Backup plans go best with a plan to refresh them every so often and move them to newer media.
Mercutio
Fatwah on Western Digital
Early last year, someone asked me to recover data off a TR3 that was written in 1999. Travan was a consumer format that was available in roughly the same time period as CD-R and their CDs were unreadable. I had a HELL of a time finding a copy of BackupExec 7.2 to handle the restore, but it turned out I had a copy on a pressed disc in my binders full of old crap. It turned out that the old drive I have still worked once I re-spooled the tape I was given. The people I was working with only had the tape, not the software or a working drive. The drive I had was a parallel port model, but I was able to load up NT4 on an ancient notebook and complete the recovery.
The lessons to take from this: Travan was a consumer format meant for home users. The recovery software was necessary to do anything with it. I also had to have a PC with the right ports and OS to handle the data. The customer DID keep their old drive, but theirs didn't work.
Hilariously, I got this little side gig because the customer approached a much larger local MSP, who passed them on to me.
Anyway, if you're doing tapes, you buy at least a couple drives, make sure they work, meaning that you can read from and recover data. You keep a copy of the backup software somewhere you can find and know to be accessible. The backup drive needs to be cleaned regularly and you need to follow whatever standard you set for frequency of backups et al. LTO 5 is a good choice right now because the drives are relatively affordable and there are gobs of cheap tapes around to be had. Make sure you have some kind of PC set aside with the right data ports to handle the drive as well, since you can't be assured that you'll have SATA / SAS / USBwhatever / Firewire / SCSI on hand when you need it.
There's a story in the news right now about how people finally found the master copies of a cartoon called Reboot that was thought to be lost for something like 15 years. Same deal. They have the tapes but they don't have anything to read them. This is why we keep a spare drive or two handy.
The lessons to take from this: Travan was a consumer format meant for home users. The recovery software was necessary to do anything with it. I also had to have a PC with the right ports and OS to handle the data. The customer DID keep their old drive, but theirs didn't work.
Hilariously, I got this little side gig because the customer approached a much larger local MSP, who passed them on to me.
Anyway, if you're doing tapes, you buy at least a couple drives, make sure they work, meaning that you can read from and recover data. You keep a copy of the backup software somewhere you can find and know to be accessible. The backup drive needs to be cleaned regularly and you need to follow whatever standard you set for frequency of backups et al. LTO 5 is a good choice right now because the drives are relatively affordable and there are gobs of cheap tapes around to be had. Make sure you have some kind of PC set aside with the right data ports to handle the drive as well, since you can't be assured that you'll have SATA / SAS / USBwhatever / Firewire / SCSI on hand when you need it.
There's a story in the news right now about how people finally found the master copies of a cartoon called Reboot that was thought to be lost for something like 15 years. Same deal. They have the tapes but they don't have anything to read them. This is why we keep a spare drive or two handy.
jtr1962
Storage? I am Storage!
Regardless of whatever media you choose, it's worth noting that degradation slows down dramatically at lower temperatures. Put the media in a sealed bag to avoid condensation, then stick in the freezer. If it was good for, say, 20 years at room temperature, it might be good centuries later in the freezer. I would bet good money if kept at low temperatures even a humble USB drive might store data for many decades.
As an aside, with all this life extension research going in, safely storing data for many centuries might take on new importance if we can get people to live that long. Who wouldn't want to look at photos of themselves in college when they're celebrating their 1000th birthday? "Hey, you haven't changed a bit over the years!"
As an aside, with all this life extension research going in, safely storing data for many centuries might take on new importance if we can get people to live that long. Who wouldn't want to look at photos of themselves in college when they're celebrating their 1000th birthday? "Hey, you haven't changed a bit over the years!"
Assuming this is not for some type of Time Capsule, how will that archived data be managed over all those years and after you are deceased?
Who exactly wants to live 1000 years? Are you going to work 900 years and hope to still be alive after that to retire?
Last edited:
Mercutio
Fatwah on Western Digital
I wasn't paid enough to make that my problem. I did the job I was paid to do.
I copied the contents of one tape to a thumb drive after I recovered it and made sure that files opened. They didn't ask me to do anything else.
I was asking the OP.
I assumed you were just doing a one-time job of data recovery. In the late 20th century I had the OnStream 15/30 GB tape drive. Transfers were a painful 1-2 MB/sec. It saved my behind when I deleted a folder by mistake. I wonder if anyone could recover them now.
jtr1962
Storage? I am Storage!
Why would you need to work 900 years? People can usually save enough to retire on by their 60s. The very point of living longer is so you have a more years without working.
Well, the OP is not going to live 1000 years and the data does not need to last that long anyway.
What really lasts the longest is stone and then maybe sheets made from durable materials (not papers/wood pulps).
Peoples or institutions must have the will to keep the data safe over the decades and centuries.
What really lasts the longest is stone and then maybe sheets made from durable materials (not papers/wood pulps).
Peoples or institutions must have the will to keep the data safe over the decades and centuries.
This is getting off-topic. Nobody said anything about hundreds or thousands of years. I just want to do better than the decade-ish lifetime of CDs and thumb drives. For that matter,it is alleged that lifetime-wise SSDs are somewhat better than thumb drives, but I've never see evidence for that laid out. I'm going to be around for at least a few more decades, and I just don't want to have to worry about making copies. As to M-disks, there have been some convincing real studies, including by the DoD, that suggest near-permanence for them. Wikipedia gives references. If you have counterexamples, I'd be interested to get references. They are glassy-carbon, not hydrocarbon based. Freezing is an interesting option, but unless I see some real tests done on that, I don't consider it an entirely credible idea.
If only a few decades, then I'd make one copy on M-Discs and another on something else like tape. SSDs would be unreliable due to the memory cells, but also active and passive components on the PCBs.
Storage at refrigerated temperatures is not a bad idea since reaction rates approximately double with every 10°C increase in temperature. However, freezing can create damage if water is present so I'd just avoid that.
Storage at refrigerated temperatures is not a bad idea since reaction rates approximately double with every 10°C increase in temperature. However, freezing can create damage if water is present so I'd just avoid that.
Thanks, LunarMist, that sounds like the optimal smart near-term approach. I also appreciate the info about LTOs. I need to do some research on those. About those LTOs, I need to ask - what is the interface? The halfway economical ones I see advertised don't tell you that. Are these all USB?
Mercutio
Fatwah on Western Digital
They're gonna be SAS. Almost ever SAS adapter you'll run across will be an LSI-something-or-ther whether it says Dell or HP or Lenovo on it. SAS host bus adapters are usually 8-lane cards and obviously it's better if you can find a 12Gbps SAS card than a 3 or 6Gbps version. A lot of cheap SAS HBAs ship in RAID mode and may need to be flashed to Initiator Target mode before they'll recognize individual drives like normal disk controllers.
Drives? Ebay and Craigslist, or pester the guys at your local datacenter. You can buy single drives or tape changers. What you use for backup is up to you. Bacula runs on Windows and *nix and it's free, so there's a nice start.
Drives? Ebay and Craigslist, or pester the guys at your local datacenter. You can buy single drives or tape changers. What you use for backup is up to you. Bacula runs on Windows and *nix and it's free, so there's a nice start.
If your data has value then figure out how imporatnt it is to you. I assign a maintenance cost of ~$50/week to my data, which is obviously higher than average. On a low budget it may be better to keep transferring data to various media as one of the data sets.
A few other thoughts...
Make sure you have a CRC checksum or other means of verifying the data integrity. Backup software usually has various ways to test it.
I have spanned containers >20 years old that have been copied a dozen times, but can still easily be tested.
Be careful with passwords. It's easy enough for a normal to forget after 10-20-30+ years, then throw in the possibility of stroke, traumatic brain injury, dementia, etc. and it needs to be accessible somewhere outside the cranium.
A few other thoughts...
Make sure you have a CRC checksum or other means of verifying the data integrity. Backup software usually has various ways to test it.
I have spanned containers >20 years old that have been copied a dozen times, but can still easily be tested.
Be careful with passwords. It's easy enough for a normal to forget after 10-20-30+ years, then throw in the possibility of stroke, traumatic brain injury, dementia, etc. and it needs to be accessible somewhere outside the cranium.
Fair point about passwords. My passwords are in a protected file, and in a safety deposit box at the bank. No one will get them. My normal backup is with SuperDuper, which uses CRC32 checksums. $50/week is VASTLY more than I'm willing to spend on data protection. The issue is what economical data storage option has the longest lifetime.
Mercutio
Fatwah on Western Digital
Buy a straight-up SAS controller please. People can't even get a USB to fully implemented RS232 adapters that work every time. If you have to dedicate a $100 ebay special desktop to running the SAS controller, so be it, but you definitely don't want to lose precious hours of your life trying to figure out why the USB adapter didn't implement some or other important SAS feature that keeps your backup drive from running properly.
Most SAS controllers have either internal or external connectors but rarely both. Internal cabling is cheaper and less likely to be weird. What do I mean by weird? Some HBAs expect the external cabinet to be powered on and ready before the HBA is initialized. Some want the controller running before the cabinet (and its drives) start. Sometimes controllers/external cabinets don't like longer cables.
Find yourself some off-lease Optiplex or Thinkstation if you don't have a case with a 5.25" drive bay. Those things can be had for next to nothing. Make it your dedicated backup system.
computerdude92
What is this storage?
- Joined
- Jan 23, 2024
- Messages
- 1
Hi guys! I just joined the forum after noticing the link on the old Australian Red Hill Tech Guide website. Any affiliation?
So my question regarding storage is: How are PC tape drives more reliable than VHS tapes? Wouldn't the data rot away over time in storage too?
So my question regarding storage is: How are PC tape drives more reliable than VHS tapes? Wouldn't the data rot away over time in storage too?
Mercutio
Fatwah on Western Digital
Tony still pops in here on occasion.
We've mostly all known each other the best part of 25 years now.
PC tape drives are a lot more expensive than VCRs. They're still helical tape systems, so the mechanical aspects are very well understood, but the tape heads and how they operate separate the capabilities of one format from another. Tapes are meant to be kept long-term. Eventually, entropy and the Earth's innate magnetism will get the best magnetic media, but for a 30 - 50 year term and a mechanism that isn't engineered from science on a material like ceramics, it's the best we've got.
We've mostly all known each other the best part of 25 years now.
PC tape drives are a lot more expensive than VCRs. They're still helical tape systems, so the mechanical aspects are very well understood, but the tape heads and how they operate separate the capabilities of one format from another. Tapes are meant to be kept long-term. Eventually, entropy and the Earth's innate magnetism will get the best magnetic media, but for a 30 - 50 year term and a mechanism that isn't engineered from science on a material like ceramics, it's the best we've got.
LTO has much better tape base and aprticles, not to mention advanced error correction to reduce the potential for data loss and ensure longevity.
I don't recall that VHS ever had ECC other than maybe the versions used early for audio by PCM. But errors were dropouts or noise, not data integrity failures.
It's like comparing a economy car engine to a race car engine. They are only superficially the same.
I don't recall that VHS ever had ECC other than maybe the versions used early for audio by PCM. But errors were dropouts or noise, not data integrity failures.
It's like comparing a economy car engine to a race car engine. They are only superficially the same.