So I'm looking at archival storage options. I don't mean years. I mean decades. Many decades. CD media is good for 10 years, under good storage conditions. SSDs tend to start degrading on that same timescale. Data on regular hard drives may last longer, but then you've got to worry about the lifetime of motors and bearings. Lubrication. Ideas? M-disks are a possibility, though they've gotten pricey. Also, one can complain that optical media in general will be obsolete in a few decades, but there will ALWAYS be archival readers around somewhere. I can find someone to read a floppy disk right now. Suggestions?
archival storage options
- Thread starter Dlass
- Start date
Mercutio
Fatwah on Western Digital
The thing you are looking for is called LTO. You need to understand that you will need to buy multiple drives and that you will also want to preserve both a working PC and the relevant software to work the drive(s) you've chosen. LTO 5 is highly affordable for 1.5TB/tape and secondhand drives that can be had for under $200. Tape drives in active use tend to get worn to crap, but people who buy LTO usually aren't buying just one of them, so there are also many sitting around that just never get used.
Tape can sit in dark, climate and magnetic-controlled conditions and still be readable decades later. It's the best option. You just have to take the hit to buy in to the format and media. There is no reason to fool around with any optical format and while it's possible to find a mechanical drive that might spin up at 20 years old, that's nowhere near as likely as using a tape.
Tape can sit in dark, climate and magnetic-controlled conditions and still be readable decades later. It's the best option. You just have to take the hit to buy in to the format and media. There is no reason to fool around with any optical format and while it's possible to find a mechanical drive that might spin up at 20 years old, that's nowhere near as likely as using a tape.
Wow, that's kinda pricey, and I need to get multiples? Geez Louise. Also, to the extent it goes obsolete, there will be fewer of them around in the distant future. Also, LTO capacity is tens of TB. I need a few hundred GB or so, so it's kind of overkill. I'm thinking that BDXL M-disks might be the way to go, though they are also not cheap. They are advertised to be "forever" media. I mean, if I want to archive stuff in the long term, I really don't want to have to rewrite every decade or so.
sedrosken
Florida Man
Tape drives have pretty much always and forever been for the cost-no-object segment of the population, usually businesses who can use it for a tax write-off. That said, LTO5 is quite affordable as far as that technology goes. If you're serious about maintaining your backups, if your data is truly irreplaceable, then the cost of entry is just the cost of entry -- if you don't have multiples, you have no redundancy for the case of physical failure of either the drive or medium.
I personally view M-DISC as a marketing ploy. I don't rightly think they last as long as they say they do, and I don't know how they'd even go about testing their claims. I'm skeptical that anything basically made of oil won't just turn back into it within a few decades one way or the next, but that's a problem with most media. I don't personally think that optical media of any sort is a forever media, certainly -- we're already having issues in the retro space with CD-ROM drives and their old, tired laser mechs being unable to read anything without at minimum an intensity pot readjustment. I just don't think you'll be able to write an M-DISC now and expect to read it in anything in, say, fifty years. Backup plans go best with a plan to refresh them every so often and move them to newer media.
I personally view M-DISC as a marketing ploy. I don't rightly think they last as long as they say they do, and I don't know how they'd even go about testing their claims. I'm skeptical that anything basically made of oil won't just turn back into it within a few decades one way or the next, but that's a problem with most media. I don't personally think that optical media of any sort is a forever media, certainly -- we're already having issues in the retro space with CD-ROM drives and their old, tired laser mechs being unable to read anything without at minimum an intensity pot readjustment. I just don't think you'll be able to write an M-DISC now and expect to read it in anything in, say, fifty years. Backup plans go best with a plan to refresh them every so often and move them to newer media.
Mercutio
Fatwah on Western Digital
Early last year, someone asked me to recover data off a TR3 that was written in 1999. Travan was a consumer format that was available in roughly the same time period as CD-R and their CDs were unreadable. I had a HELL of a time finding a copy of BackupExec 7.2 to handle the restore, but it turned out I had a copy on a pressed disc in my binders full of old crap. It turned out that the old drive I have still worked once I re-spooled the tape I was given. The people I was working with only had the tape, not the software or a working drive. The drive I had was a parallel port model, but I was able to load up NT4 on an ancient notebook and complete the recovery.
The lessons to take from this: Travan was a consumer format meant for home users. The recovery software was necessary to do anything with it. I also had to have a PC with the right ports and OS to handle the data. The customer DID keep their old drive, but theirs didn't work.
Hilariously, I got this little side gig because the customer approached a much larger local MSP, who passed them on to me.
Anyway, if you're doing tapes, you buy at least a couple drives, make sure they work, meaning that you can read from and recover data. You keep a copy of the backup software somewhere you can find and know to be accessible. The backup drive needs to be cleaned regularly and you need to follow whatever standard you set for frequency of backups et al. LTO 5 is a good choice right now because the drives are relatively affordable and there are gobs of cheap tapes around to be had. Make sure you have some kind of PC set aside with the right data ports to handle the drive as well, since you can't be assured that you'll have SATA / SAS / USBwhatever / Firewire / SCSI on hand when you need it.
There's a story in the news right now about how people finally found the master copies of a cartoon called Reboot that was thought to be lost for something like 15 years. Same deal. They have the tapes but they don't have anything to read them. This is why we keep a spare drive or two handy.
The lessons to take from this: Travan was a consumer format meant for home users. The recovery software was necessary to do anything with it. I also had to have a PC with the right ports and OS to handle the data. The customer DID keep their old drive, but theirs didn't work.
Hilariously, I got this little side gig because the customer approached a much larger local MSP, who passed them on to me.
Anyway, if you're doing tapes, you buy at least a couple drives, make sure they work, meaning that you can read from and recover data. You keep a copy of the backup software somewhere you can find and know to be accessible. The backup drive needs to be cleaned regularly and you need to follow whatever standard you set for frequency of backups et al. LTO 5 is a good choice right now because the drives are relatively affordable and there are gobs of cheap tapes around to be had. Make sure you have some kind of PC set aside with the right data ports to handle the drive as well, since you can't be assured that you'll have SATA / SAS / USBwhatever / Firewire / SCSI on hand when you need it.
There's a story in the news right now about how people finally found the master copies of a cartoon called Reboot that was thought to be lost for something like 15 years. Same deal. They have the tapes but they don't have anything to read them. This is why we keep a spare drive or two handy.
jtr1962
Storage? I am Storage!
Regardless of whatever media you choose, it's worth noting that degradation slows down dramatically at lower temperatures. Put the media in a sealed bag to avoid condensation, then stick in the freezer. If it was good for, say, 20 years at room temperature, it might be good centuries later in the freezer. I would bet good money if kept at low temperatures even a humble USB drive might store data for many decades.
As an aside, with all this life extension research going in, safely storing data for many centuries might take on new importance if we can get people to live that long. Who wouldn't want to look at photos of themselves in college when they're celebrating their 1000th birthday? "Hey, you haven't changed a bit over the years!"
As an aside, with all this life extension research going in, safely storing data for many centuries might take on new importance if we can get people to live that long. Who wouldn't want to look at photos of themselves in college when they're celebrating their 1000th birthday? "Hey, you haven't changed a bit over the years!"
Assuming this is not for some type of Time Capsule, how will that archived data be managed over all those years and after you are deceased?
Who exactly wants to live 1000 years? Are you going to work 900 years and hope to still be alive after that to retire?
Last edited:
Mercutio
Fatwah on Western Digital
I wasn't paid enough to make that my problem. I did the job I was paid to do.
I copied the contents of one tape to a thumb drive after I recovered it and made sure that files opened. They didn't ask me to do anything else.
I was asking the OP.
I assumed you were just doing a one-time job of data recovery. In the late 20th century I had the OnStream 15/30 GB tape drive. Transfers were a painful 1-2 MB/sec. It saved my behind when I deleted a folder by mistake. I wonder if anyone could recover them now.
jtr1962
Storage? I am Storage!
Why would you need to work 900 years? People can usually save enough to retire on by their 60s. The very point of living longer is so you have a more years without working.
Well, the OP is not going to live 1000 years and the data does not need to last that long anyway.
What really lasts the longest is stone and then maybe sheets made from durable materials (not papers/wood pulps).
Peoples or institutions must have the will to keep the data safe over the decades and centuries.
What really lasts the longest is stone and then maybe sheets made from durable materials (not papers/wood pulps).
Peoples or institutions must have the will to keep the data safe over the decades and centuries.
This is getting off-topic. Nobody said anything about hundreds or thousands of years. I just want to do better than the decade-ish lifetime of CDs and thumb drives. For that matter,it is alleged that lifetime-wise SSDs are somewhat better than thumb drives, but I've never see evidence for that laid out. I'm going to be around for at least a few more decades, and I just don't want to have to worry about making copies. As to M-disks, there have been some convincing real studies, including by the DoD, that suggest near-permanence for them. Wikipedia gives references. If you have counterexamples, I'd be interested to get references. They are glassy-carbon, not hydrocarbon based. Freezing is an interesting option, but unless I see some real tests done on that, I don't consider it an entirely credible idea.
If only a few decades, then I'd make one copy on M-Discs and another on something else like tape. SSDs would be unreliable due to the memory cells, but also active and passive components on the PCBs.
Storage at refrigerated temperatures is not a bad idea since reaction rates approximately double with every 10°C increase in temperature. However, freezing can create damage if water is present so I'd just avoid that.
Storage at refrigerated temperatures is not a bad idea since reaction rates approximately double with every 10°C increase in temperature. However, freezing can create damage if water is present so I'd just avoid that.
Thanks, LunarMist, that sounds like the optimal smart near-term approach. I also appreciate the info about LTOs. I need to do some research on those. About those LTOs, I need to ask - what is the interface? The halfway economical ones I see advertised don't tell you that. Are these all USB?
Mercutio
Fatwah on Western Digital
They're gonna be SAS. Almost ever SAS adapter you'll run across will be an LSI-something-or-ther whether it says Dell or HP or Lenovo on it. SAS host bus adapters are usually 8-lane cards and obviously it's better if you can find a 12Gbps SAS card than a 3 or 6Gbps version. A lot of cheap SAS HBAs ship in RAID mode and may need to be flashed to Initiator Target mode before they'll recognize individual drives like normal disk controllers.
Drives? Ebay and Craigslist, or pester the guys at your local datacenter. You can buy single drives or tape changers. What you use for backup is up to you. Bacula runs on Windows and *nix and it's free, so there's a nice start.
Drives? Ebay and Craigslist, or pester the guys at your local datacenter. You can buy single drives or tape changers. What you use for backup is up to you. Bacula runs on Windows and *nix and it's free, so there's a nice start.
If your data has value then figure out how imporatnt it is to you. I assign a maintenance cost of ~$50/week to my data, which is obviously higher than average. On a low budget it may be better to keep transferring data to various media as one of the data sets.
A few other thoughts...
Make sure you have a CRC checksum or other means of verifying the data integrity. Backup software usually has various ways to test it.
I have spanned containers >20 years old that have been copied a dozen times, but can still easily be tested.
Be careful with passwords. It's easy enough for a normal to forget after 10-20-30+ years, then throw in the possibility of stroke, traumatic brain injury, dementia, etc. and it needs to be accessible somewhere outside the cranium.
A few other thoughts...
Make sure you have a CRC checksum or other means of verifying the data integrity. Backup software usually has various ways to test it.
I have spanned containers >20 years old that have been copied a dozen times, but can still easily be tested.
Be careful with passwords. It's easy enough for a normal to forget after 10-20-30+ years, then throw in the possibility of stroke, traumatic brain injury, dementia, etc. and it needs to be accessible somewhere outside the cranium.
Fair point about passwords. My passwords are in a protected file, and in a safety deposit box at the bank. No one will get them. My normal backup is with SuperDuper, which uses CRC32 checksums. $50/week is VASTLY more than I'm willing to spend on data protection. The issue is what economical data storage option has the longest lifetime.
Mercutio
Fatwah on Western Digital
Buy a straight-up SAS controller please. People can't even get a USB to fully implemented RS232 adapters that work every time. If you have to dedicate a $100 ebay special desktop to running the SAS controller, so be it, but you definitely don't want to lose precious hours of your life trying to figure out why the USB adapter didn't implement some or other important SAS feature that keeps your backup drive from running properly.
Most SAS controllers have either internal or external connectors but rarely both. Internal cabling is cheaper and less likely to be weird. What do I mean by weird? Some HBAs expect the external cabinet to be powered on and ready before the HBA is initialized. Some want the controller running before the cabinet (and its drives) start. Sometimes controllers/external cabinets don't like longer cables.
Find yourself some off-lease Optiplex or Thinkstation if you don't have a case with a 5.25" drive bay. Those things can be had for next to nothing. Make it your dedicated backup system.
computerdude92
What is this storage?
- Joined
- Jan 23, 2024
- Messages
- 1
Hi guys! I just joined the forum after noticing the link on the old Australian Red Hill Tech Guide website. Any affiliation?
So my question regarding storage is: How are PC tape drives more reliable than VHS tapes? Wouldn't the data rot away over time in storage too?
So my question regarding storage is: How are PC tape drives more reliable than VHS tapes? Wouldn't the data rot away over time in storage too?
Mercutio
Fatwah on Western Digital
Tony still pops in here on occasion.
We've mostly all known each other the best part of 25 years now.
PC tape drives are a lot more expensive than VCRs. They're still helical tape systems, so the mechanical aspects are very well understood, but the tape heads and how they operate separate the capabilities of one format from another. Tapes are meant to be kept long-term. Eventually, entropy and the Earth's innate magnetism will get the best magnetic media, but for a 30 - 50 year term and a mechanism that isn't engineered from science on a material like ceramics, it's the best we've got.
We've mostly all known each other the best part of 25 years now.
PC tape drives are a lot more expensive than VCRs. They're still helical tape systems, so the mechanical aspects are very well understood, but the tape heads and how they operate separate the capabilities of one format from another. Tapes are meant to be kept long-term. Eventually, entropy and the Earth's innate magnetism will get the best magnetic media, but for a 30 - 50 year term and a mechanism that isn't engineered from science on a material like ceramics, it's the best we've got.
LTO has much better tape base and aprticles, not to mention advanced error correction to reduce the potential for data loss and ensure longevity.
I don't recall that VHS ever had ECC other than maybe the versions used early for audio by PCM. But errors were dropouts or noise, not data integrity failures.
It's like comparing a economy car engine to a race car engine. They are only superficially the same.
I don't recall that VHS ever had ECC other than maybe the versions used early for audio by PCM. But errors were dropouts or noise, not data integrity failures.
It's like comparing a economy car engine to a race car engine. They are only superficially the same.
ArAfGo
How To Improve My Brain Storage?
The search engine here informs me that using the search term "USB" is too short, or some such information. Even when it is added to other words. So I have been going through various threads and found this:
I am needing to learn and this relates to archiving using USB sticks back more than ten years ago and needing to do more archiving within a few weeks to help an older Windows 7 run a little better. But I have a number of different units with all sorts of operating systems and am going to have to properly study about all this data storage stuff.
I haven't been careful about temperature control, but I have been very careful about wrapping all the sticks very carefully. In fact, one thing I have always used is those sheets and bags that have the little plastic bubbles built in and that was because this is an area of the planet that we are never 100% earthquake safe and so I figured even if an earthquake zapped me the USB sticks might be okay because they are so wrapped up that water probably couldn't get in the bags I use and those multiple layers of air pocket bubble bags might save them from getting smashed.
I always do two sticks of the same data every time I do an archiving job. Over the years it went from sticks able to store 16GB to 32GB to 64GB, but I have been not so sure about using any stick over 64GB. I probably have about 30 sticks already full, but I am wondering if I were to do the archiving all over again; meaning the oldest ones to new sticks --- would that be a worthwhile project?
I also have a piece of equipment from about 15 years ago that requires plugging in for power and a fancy connecting plug and did check it a few years ago and it seemed to be just fine and I'll dig it up and give y'all a model name and number and stuff and I'll ask for advice.
One thing that is important is some data that is absolutely of value for future students wanting to study the Net. I have stuff from when I was active with the ISOC. I have stuff from an early discussion platform that was one of the busiest in its time --- I was an employee admin and then almost got permission to buy the forum after the news people split their company section off from the discussion platform. During the discussion between my legal team and the new owner of the news section I was given permission to take control of the forum database and about 6 or 7 years ago a university fella that my chief doc graduated from indicated an interest in taking possession of that website data for historical research. Not a money thing for me. Just I hand it over, but we are also discussing some of my other work on the Net. There is also the VFW work that is an archive that goes way back, too.
So is it okay to be using this thread to be asking questions?
Anyone have negative thoughts about using USB sticks? Oh yes, so many are the 2.0 kind, I think that's the correct version. Had some trouble with the 3.0 when it first came out a few years or so ago. Maybe I should also buy a fancy storage machine or two as another storage backup? I've always been pretty good about making sure I buy quality USB sticks, but that is usually just determined by folks giving information about their experience or simply buying the most expensive. Lucky to be in Japan for all the years because it seems quality was reliable until maybe some much started getting made in Nam or China. Products out of Nam were really good quality when that first started. Mainland China has been a weird guessing game when it comes to quality control. Thing is, I don't mind paying extra for quality.
Last question, data stored about 20 or so years ago on the old floppy disks --- anyone reckon that could be recovered? I have two TRS80s and that huge manual and a whole bunch of original floppies that were sent out by Tandy Radio Shack for government use, so maybe a bit more from the company than to an average customer. Then a bunch of government data stored on the floppies purchased blank and filled up. Had at least one drive for the floppies that worked just fine about 6 or 7 years ago when we tested it. How long you reckon those old floppies can maintain data?
I am needing to learn and this relates to archiving using USB sticks back more than ten years ago and needing to do more archiving within a few weeks to help an older Windows 7 run a little better. But I have a number of different units with all sorts of operating systems and am going to have to properly study about all this data storage stuff.
I haven't been careful about temperature control, but I have been very careful about wrapping all the sticks very carefully. In fact, one thing I have always used is those sheets and bags that have the little plastic bubbles built in and that was because this is an area of the planet that we are never 100% earthquake safe and so I figured even if an earthquake zapped me the USB sticks might be okay because they are so wrapped up that water probably couldn't get in the bags I use and those multiple layers of air pocket bubble bags might save them from getting smashed.
I always do two sticks of the same data every time I do an archiving job. Over the years it went from sticks able to store 16GB to 32GB to 64GB, but I have been not so sure about using any stick over 64GB. I probably have about 30 sticks already full, but I am wondering if I were to do the archiving all over again; meaning the oldest ones to new sticks --- would that be a worthwhile project?
I also have a piece of equipment from about 15 years ago that requires plugging in for power and a fancy connecting plug and did check it a few years ago and it seemed to be just fine and I'll dig it up and give y'all a model name and number and stuff and I'll ask for advice.
One thing that is important is some data that is absolutely of value for future students wanting to study the Net. I have stuff from when I was active with the ISOC. I have stuff from an early discussion platform that was one of the busiest in its time --- I was an employee admin and then almost got permission to buy the forum after the news people split their company section off from the discussion platform. During the discussion between my legal team and the new owner of the news section I was given permission to take control of the forum database and about 6 or 7 years ago a university fella that my chief doc graduated from indicated an interest in taking possession of that website data for historical research. Not a money thing for me. Just I hand it over, but we are also discussing some of my other work on the Net. There is also the VFW work that is an archive that goes way back, too.
So is it okay to be using this thread to be asking questions?
Anyone have negative thoughts about using USB sticks? Oh yes, so many are the 2.0 kind, I think that's the correct version. Had some trouble with the 3.0 when it first came out a few years or so ago. Maybe I should also buy a fancy storage machine or two as another storage backup? I've always been pretty good about making sure I buy quality USB sticks, but that is usually just determined by folks giving information about their experience or simply buying the most expensive. Lucky to be in Japan for all the years because it seems quality was reliable until maybe some much started getting made in Nam or China. Products out of Nam were really good quality when that first started. Mainland China has been a weird guessing game when it comes to quality control. Thing is, I don't mind paying extra for quality.
Last question, data stored about 20 or so years ago on the old floppy disks --- anyone reckon that could be recovered? I have two TRS80s and that huge manual and a whole bunch of original floppies that were sent out by Tandy Radio Shack for government use, so maybe a bit more from the company than to an average customer. Then a bunch of government data stored on the floppies purchased blank and filled up. Had at least one drive for the floppies that worked just fine about 6 or 7 years ago when we tested it. How long you reckon those old floppies can maintain data?
Last edited:
Mercutio
Fatwah on Western Digital
USB sticks and for that matter memory cards just up and die sometimes. For USB drives, this might be because of damage to the PCB or the USB contacts. I keep a couple terabytes of 250 and 500GB SandDisk Fit drives in my pocket every day and some of them have damage to the plastic shell over the USB contacts but continue to work just fine. That being said, I treat my thumb drives as disposable.
SD cards are REALLY a your mileage my vary sort of matter. I've had better luck with some brands than others. Ironically, SanDisk (owned by notorious Bride of Satan Western Digital) has been very good for me, while I've had poor luck with otherwise well regarded Samsung. Lexar, which used to be the gold standard in SD cards, got eaten by a Chinese conglomerate and are now absolute garbage. But sometimes cards die. This is why camera people like to have two cards in their cameras. Things Happen.
I do keep camera cards in a wallet. They're numbered and tied to a spreadsheet with a listing of their contents. Since I only have a few high-capacity UHS II cards, THOSE get copied to new, slower cards so I can continue to use the fast ones, but overwhelmingly I do not erase data from my cards. They're functionally write-once media for me. It's fairly typical for me to fill a 250GB card every six weeks or so.
re: Floppies. I have retrieved data off a 3.5" 720k floppy this year. 2025. Yes. It is possible that the diskettes will still work. The biggest issue with floppies is that drive heads can get slightly out of alignment so the disks can only be read in the drive where they were made, but if that isn't a problem and the disks were stored properly, it's possible to read data off of them. I keep a USB 3.5" floppy around just in case I need it, which is most often because some secretary or bookkeeper who has been doing the same job since the late 80s or early 90s has files that might be in a box of floppies that have been in a drawer for the last 30 years.
SD cards are REALLY a your mileage my vary sort of matter. I've had better luck with some brands than others. Ironically, SanDisk (owned by notorious Bride of Satan Western Digital) has been very good for me, while I've had poor luck with otherwise well regarded Samsung. Lexar, which used to be the gold standard in SD cards, got eaten by a Chinese conglomerate and are now absolute garbage. But sometimes cards die. This is why camera people like to have two cards in their cameras. Things Happen.
I do keep camera cards in a wallet. They're numbered and tied to a spreadsheet with a listing of their contents. Since I only have a few high-capacity UHS II cards, THOSE get copied to new, slower cards so I can continue to use the fast ones, but overwhelmingly I do not erase data from my cards. They're functionally write-once media for me. It's fairly typical for me to fill a 250GB card every six weeks or so.
re: Floppies. I have retrieved data off a 3.5" 720k floppy this year. 2025. Yes. It is possible that the diskettes will still work. The biggest issue with floppies is that drive heads can get slightly out of alignment so the disks can only be read in the drive where they were made, but if that isn't a problem and the disks were stored properly, it's possible to read data off of them. I keep a USB 3.5" floppy around just in case I need it, which is most often because some secretary or bookkeeper who has been doing the same job since the late 80s or early 90s has files that might be in a box of floppies that have been in a drawer for the last 30 years.
sedrosken
Florida Man
I come into more regular contact with floppies than Merc just by virtue of my particular flavor of autistic hyperfixation, and these are my observations:
There's an inverse proportion to age of the disk and likelihood of recoverability. Older disks were simply made better -- the fractions of a cent they saved with worse materials and such as floppies transitioned from the main storage media of the day to an important everyday item for data transfer to finally being relegated to emergency bootdisk use or the odd FD Mavica simply means the older disks last better. Stored well, I would say I'm pretty likely to get legible data off of a floppy written in the mid to late 80s, whereas all of my disks from 1997-2002 or later all are nigh-inscrutable to any of my drives and usually unusable with a format, even.
I have had floppies that were babied but written in 2000 be completely unreadable while some from the 80s that I pulled out of a moldy shed worked fine.
Another aspect is that generic USB floppy drives are incredibly finicky. I've heard of some name-brand USB drives from Sony and I think Toshiba that are reportedly pretty decent, though. I have the best luck with my actual floppy controller-controlled drives in my 486 machine -- the 3.5" drives I put in my Pentium III, Geode, and Core2 boxes are descendingly less reliable. My guess is that as the floppy controller slowly grew into its vestigial state by the time the Core2 era had come, less and less attention was paid to making sure they worked well.
Now, as for flash drives -- I don't use them as an archival medium whatsoever. They're treated as disposable -- I use them for OS install media, or for sneakernetting data to something that I can't give access to my network storage for some reason, usually because they don't have NIC drivers -- and in that case usually the data I'm sneakernetting is a NIC driver. I have a couple dozen that I've accumulated over the years and I've had a handful of them fail.
SD cards I treat similarly. I use them for OS installs on my vintage boxes where I find they're faster and more convenient and usually more reliable than a period hard drive, but that comes with the territory of pulling regular backups off of them that I can restore to new cards when (not if) they fail. Failure of an SD card is pretty much inevitable with the write cycle load of an OS page file, but SD cards are cheap so I have no issue with replacing them as needed. No important data is kept on them, not for archival's sake, anyway.
Right now my method of backup is copying data to cold-stored hard drives. Not cold in that I put them in the freezer, cold as in left unpowered and stored properly. If I ever have the budget, my plan is to get an LTO6 tape drive and a couple sets of tapes that I can rotate and keep at various other places I have access to in the event of theft or destruction of my place of residence.
There's an inverse proportion to age of the disk and likelihood of recoverability. Older disks were simply made better -- the fractions of a cent they saved with worse materials and such as floppies transitioned from the main storage media of the day to an important everyday item for data transfer to finally being relegated to emergency bootdisk use or the odd FD Mavica simply means the older disks last better. Stored well, I would say I'm pretty likely to get legible data off of a floppy written in the mid to late 80s, whereas all of my disks from 1997-2002 or later all are nigh-inscrutable to any of my drives and usually unusable with a format, even.
I have had floppies that were babied but written in 2000 be completely unreadable while some from the 80s that I pulled out of a moldy shed worked fine.
Another aspect is that generic USB floppy drives are incredibly finicky. I've heard of some name-brand USB drives from Sony and I think Toshiba that are reportedly pretty decent, though. I have the best luck with my actual floppy controller-controlled drives in my 486 machine -- the 3.5" drives I put in my Pentium III, Geode, and Core2 boxes are descendingly less reliable. My guess is that as the floppy controller slowly grew into its vestigial state by the time the Core2 era had come, less and less attention was paid to making sure they worked well.
Now, as for flash drives -- I don't use them as an archival medium whatsoever. They're treated as disposable -- I use them for OS install media, or for sneakernetting data to something that I can't give access to my network storage for some reason, usually because they don't have NIC drivers -- and in that case usually the data I'm sneakernetting is a NIC driver. I have a couple dozen that I've accumulated over the years and I've had a handful of them fail.
SD cards I treat similarly. I use them for OS installs on my vintage boxes where I find they're faster and more convenient and usually more reliable than a period hard drive, but that comes with the territory of pulling regular backups off of them that I can restore to new cards when (not if) they fail. Failure of an SD card is pretty much inevitable with the write cycle load of an OS page file, but SD cards are cheap so I have no issue with replacing them as needed. No important data is kept on them, not for archival's sake, anyway.
Right now my method of backup is copying data to cold-stored hard drives. Not cold in that I put them in the freezer, cold as in left unpowered and stored properly. If I ever have the budget, my plan is to get an LTO6 tape drive and a couple sets of tapes that I can rotate and keep at various other places I have access to in the event of theft or destruction of my place of residence.
ArAfGo
How To Improve My Brain Storage?
I hope that "like" thingy works this time after I log out. I tried to "Like" Mercutio's post yesterday, maybe yesterday; but just now saw no Likey thing. Oh well, these various software platforms do tend to get all human like (my wife) and do little glitchy things and that makes it so fun for the admin. And now I just got a Super Dislike from the Admin Union of the Planet.
Okey-dokey, back to the proper subject --- storing digital data.
I get a sense that actually there is no sure way to save digital data in a digital form. Seems to me that printing it all out and wrapping it up properly is the only sure way to save the stuff for a century or more. Very odd, isn't it; we have all this super cool hi-tech stuff and we could lose all of it if the Sun decides to have a Galaxy produced Alka-Seltzer and burps a big one and fries all our fine hi-tech stuff on our poor little Earth. That paper-saved stuff might survive along with maybe half to two-thirds of the humans that survive.
Well, I really don't know if a super solar flare would kill many humans, but it would certainly be so bad for a whole bunch of folks on hospital machines that need electricity and stuff. Be a few folks way deep in some remote area with their electric vehicle of any sort that might be in a bind. So weird; we think we are all hi-tech and super cool with all the new stuff and one big burp from Ms./Mr. Sun and 'POOF' all gone.
Oh yes, and that description of a "cool" storage meaning no power to the unit caught my attention, as that was exactly what has happened with at least one device I have that needs power and a hooking up to one of the units to get data off it. That unit is a fair bit into a decade or more old and was used when I was doing the admin job on Japan Today and was sort of an emergency purchase and use, and then when the buyout plan didn't work I sort of just forgot about that as somebody else quickly made me an offer I couldn't refuse and I had my head back into those 12 to 14 hour days and all that goes with such a crazy work style. Didn't check that storage device again until about a year ago, I think. Was surprised it kicked out data in what seemed like just fine style.
But I see from the advice above that maybe I have been relying on those USB sticks way too much. Even though I have been careful about quality. Looks like I need to get my butt over to Bic Camera or Yodobashi Camera and pick up some fancy storage unit and then send a fax to Ms./Mr. Sun to please wait for about two centuries before trying that super Alka-Seltzer. Or any galactic baking soda. [Water would be hard to find for the galactic baking soda.]
By the way, I don't advise any electric flying machines. I strongly suggest you stay with them awful fossil fuels for now. A 'long' now. But cargo drones might be cool. Bet there is some company into research on that. But humans on an electric powered flying machine seems a tad too adventurous at present. Heck, it used to be that commercial entities wouldn't allow full auto rotations during training flights. We could do that in the military, though. Lots more money to spend on broken choppers and broken humans. Andbody ever seen a full auto rotation?
Note: Sorry about that last paragraph. I think I had in mind that other thread that started about keyboards and shifted to ugly fluid from our old, old dinosaur friends. Well, them and other stuff made that oil, yes?
Okey-dokey, back to the proper subject --- storing digital data.
I get a sense that actually there is no sure way to save digital data in a digital form. Seems to me that printing it all out and wrapping it up properly is the only sure way to save the stuff for a century or more. Very odd, isn't it; we have all this super cool hi-tech stuff and we could lose all of it if the Sun decides to have a Galaxy produced Alka-Seltzer and burps a big one and fries all our fine hi-tech stuff on our poor little Earth. That paper-saved stuff might survive along with maybe half to two-thirds of the humans that survive.
Well, I really don't know if a super solar flare would kill many humans, but it would certainly be so bad for a whole bunch of folks on hospital machines that need electricity and stuff. Be a few folks way deep in some remote area with their electric vehicle of any sort that might be in a bind. So weird; we think we are all hi-tech and super cool with all the new stuff and one big burp from Ms./Mr. Sun and 'POOF' all gone.
Oh yes, and that description of a "cool" storage meaning no power to the unit caught my attention, as that was exactly what has happened with at least one device I have that needs power and a hooking up to one of the units to get data off it. That unit is a fair bit into a decade or more old and was used when I was doing the admin job on Japan Today and was sort of an emergency purchase and use, and then when the buyout plan didn't work I sort of just forgot about that as somebody else quickly made me an offer I couldn't refuse and I had my head back into those 12 to 14 hour days and all that goes with such a crazy work style. Didn't check that storage device again until about a year ago, I think. Was surprised it kicked out data in what seemed like just fine style.
But I see from the advice above that maybe I have been relying on those USB sticks way too much. Even though I have been careful about quality. Looks like I need to get my butt over to Bic Camera or Yodobashi Camera and pick up some fancy storage unit and then send a fax to Ms./Mr. Sun to please wait for about two centuries before trying that super Alka-Seltzer. Or any galactic baking soda. [Water would be hard to find for the galactic baking soda.]
By the way, I don't advise any electric flying machines. I strongly suggest you stay with them awful fossil fuels for now. A 'long' now. But cargo drones might be cool. Bet there is some company into research on that. But humans on an electric powered flying machine seems a tad too adventurous at present. Heck, it used to be that commercial entities wouldn't allow full auto rotations during training flights. We could do that in the military, though. Lots more money to spend on broken choppers and broken humans. Andbody ever seen a full auto rotation?
Note: Sorry about that last paragraph. I think I had in mind that other thread that started about keyboards and shifted to ugly fluid from our old, old dinosaur friends. Well, them and other stuff made that oil, yes?
Mercutio
Fatwah on Western Digital
This is my experience with nearly all optical media as well. CD-Rs from the 90s are fine (as are pretty much all red book CDs; which DO have a staggering amount of error correcting data built in), and my oldest DVD-Rs. I was very picky about Writable DVD media for myself but even so almost no DVD discs but my very oldest are still usable.
I still run across people who think optical media represents a worthwhile backup format and oh boy I wish there were some way to communicate to everyone out there that this was barely even true 20 years ago.
sedrosken
Florida Man
I tend to think along the lines of Optical Media Bad and only use physical discs for OS reinstalls on stuff that can't boot a USB drive, but I've observed the same thing. I have CD-Rs from the earliest of the 2000s when they would still not have been very cheap and they're fine. Obviously my pressed discs from ~91-05 are more or less still fine, properly stored, but anything after 2005 is a crapshoot any more than 10 years after it was made. Which is interesting considering CDs were intended to be incredibly cheaply produced to begin with -- fractions of cents worth of polycarbonate and the thinnest microns of metallized layer to give the laser something to reflect off of. You'd think they'd have had problems from the very beginning with reliability.
CDs in general have quite a lot of error correction going on behind the scenes -- if you've noticed your "overburnt" CDs (in excess of 650MB on up to about 830MB) are less reliable, that'd be because it throws a lot of that error correction out with the bathwater. I also think mixed-mode CDs might also give up some of that error correction, but it matters less for the pure CD audio session as a small error in a sound sample will last literally for a small fraction of a second.
CDs in general have quite a lot of error correction going on behind the scenes -- if you've noticed your "overburnt" CDs (in excess of 650MB on up to about 830MB) are less reliable, that'd be because it throws a lot of that error correction out with the bathwater. I also think mixed-mode CDs might also give up some of that error correction, but it matters less for the pure CD audio session as a small error in a sound sample will last literally for a small fraction of a second.
jtr1962
Storage? I am Storage!
My thought about USB drives failing is if your carry them around you build up static charge while walking, especially on a carpet. Touch the drive before discharging yourself and you may well cause it to fail. Probably the USB drives with metal cases would be much better protected. I actually keep my USB drives in a small metal box for both ESD and EMI protection. I don't have carpets, and I don't walk around with them beyond going room to room. I haven't had one fail yet.
Regarding floppies, I strongly feel the earlier ones had a reliability which hasn't been surpassed by any other storage media. The downside is that reliability came at the cost of huge magnetic bits which limited the storage to an amount which is useless these days.
Regarding floppies, I strongly feel the earlier ones had a reliability which hasn't been surpassed by any other storage media. The downside is that reliability came at the cost of huge magnetic bits which limited the storage to an amount which is useless these days.
sedrosken
Florida Man
I'm not just talking about the density -- I'm talking about date of manufacture. 1.44MB disks I've had that are from 1988/1989 still work beautifully, meanwhile I just got an entire unopened box of NOS 1.44MB disks from 2001 a couple months ago and out of ten disks only one didn't report an insane amount of bad clusters. This is after nuking and repaving them with specialized format utilities.
jtr1962
Storage? I am Storage!
I'm aware the date of manufacture matters also. The 2.88 MB format didn't really catch on but I wonder if the early versions of those were also reliable, or if the bit density was just too high.
The larger problem with floppies was that their mechanism for finding a track was purely mechanical. The LS-240 drive could reformat regular floppies to 32 MB but I wonder how reliable this was long term.
The larger problem with floppies was that their mechanism for finding a track was purely mechanical. The LS-240 drive could reformat regular floppies to 32 MB but I wonder how reliable this was long term.
ArAfGo
How To Improve My Brain Storage?
Interesting point about static electricity being a possible problem. Thank you for bringing that up. Never thought of it and will study that.
As for that metal box idea, I started wrapping all my USB sticks in aluminum foil many years ago. I don't remember what got me started doing that as it was a long time ago.
Oh yes, and another post I did around here made me aware of my incorrect time statement around here in an earlier post when I think I wrote a decade or so ago. Actually it was two decades or so ago. In fact, that unit that requires plugging into electricity I made note of might have been used back a long time ago because there was no other storage device, like the USB stick. I can't remember when the USB stick hit the market, but . . . well, my memory in my personal USB is faulty - - - USB = Upper Stupid Brain.