TR Forums

I read an article this morning about scientists warning of a digital dark age with current digital media at risk of disappearing over time. I always thought optical media would last longer than magnetic if taken care of and not scratched up but they say otherwise. They mention synthetic DNA technology could be a solution for the longer term and that “A single gram of DNA could, in principle, store every bit of datum ever recorded by humans in a container about the size and weight of a couple of pickup trucks.”

People have been warning about this for a couple of decades already. Media degrades, and the equipment to read it becomes obsolete and difficult to repair/replace.

Longevity of optical media depends a lot on the quality of the media and how it is stored. Yes, there were some early audio CDs that were manufactured incorrectly, where the aluminum layer corroded over time; however, I have been buying music CDs since the 1980s, and have yet to encounter one that has degraded to the point where it does not play. So anecdotally, they last at least 30 years.

Burned CD/DVD/BD-ROMs are more of a question mark; however, I have an anecdotal data point for that as well. About 15 years ago I ripped my entire music collection (up to that point), encoded everything to MP3, and archived the raw WAV rips to DVD-ROM (disk space was a lot more expensive back then). About a year ago, I went back and reloaded all of the original rips, to create lossless FLAC versions of everything. Every single one of those DVD-ROMs (approximately 150 of them) was still readable. Now, I did take care to buy quality name-brand media, and the spindles of burned DVDs were stored in Ziploc bags. But aside from that I took no special precautions.

Flash media is definitely a big concern. With everything moving to TLC, I would not trust it to retain data for more than a few years.

I'm pretty sure long-term data survivability is going to suffer with the gradual shift from HDD to SSD.

BD-R I think should have better longevity than previous optical standards. The problem with optical media is capacity. CDs were good for their time, but since DVD it's just stagnation. I wouldn't mind at all to again have viable removable media. And if possible, with random rewritability, ok? Thanks.

meerkt wrote:

I wouldn't mind at all to again have viable removable media. And if possible, with random rewritability, ok? Thanks.

Get a hot swap bay or an external drive dock, and pretend that bare HDDs are removable media.

Every 10-15 years, copy your backups to a new external hdd. Now you've got a fresh copy that will last you another 10-15 years. Problem solved.

PS: I never actually learned how to do this and it seems like an opportune thread to mention how to do error checking on data that you're backing up. Can anyone provide links/info/instructions on how to do this? IIRC there's a way to do this in Windows without 3rd party software?

just brew it! wrote:

Burned CD/DVD/BD-ROMs are more of a question mark; however, I have an anecdotal data point for that as well. About 15 years ago I ripped my entire music collection (up to that point), encoded everything to MP3, and archived the raw WAV rips to DVD-ROM (disk space was a lot more expensive back then). About a year ago, I went back and reloaded all of the original rips, to create lossless FLAC versions of everything. Every single one of those DVD-ROMs (approximately 150 of them) was still readable. Now, I did take care to buy quality name-brand media, and the spindles of burned DVDs were stored in Ziploc bags. But aside from that I took no special precautions.

That's good news to me. I'm finally getting around to ripping and archiving my music CD collection from the 1990's into FLAC and storing on a NAS. After reading about the degradation of CD's I was concerned that I waited too long. However it is probably too late for my old '80's cassette tapes of Def Leppard, Dokken, etc.

JBI wrote:

Longevity of optical media depends a lot on the quality of the media and how it is stored. Yes, there were some early audio CDs that were manufactured incorrectly, where the aluminum layer corroded over time; however, I have been buying music CDs since the 1980s, and have yet to encounter one that has degraded to the point where it does not play. So anecdotally, they last at least 30 years.

I just digitalized a VHS from 1985, 32 years ago. The quality did not appear to be meaningfully degraded to my untrained eye, however, the lubrication on the tape itself clearly had some issues: I didn't have any overall obvious signal integrity issues (it even looked much better than I expected!), but I had semi-severe mechanical ones!

Well, you've got a multitude of things...

• Rapid obsolescence of interfaces and mediums. Regardless of durability, the tech might not be usable or meaningful past a handful of years
• Tech that's so new, its longevity is unproven or completely unknown (e.g., TLC flash)
• A race to the bottom for price, with the obvious result of lower quality. It's not that a storage medium can't be made to last, but that it increases costs, and longevity is of least concern

Like JBI, I've also yet to encounter any factory-pressed discs that degraded and became unplayable (without some sort of physical damage). A couple years ago, I bought a stack of archival-grade DVD-R's to store and distribute family photos (supposed to last 70 years). Optical is slowly on it's way out, but at the time it seemed a better option to last a couple decades than a hard drive or flash (especially after having just dealt with an 840EVO at the time). I imagine at the very least, some sort of cheap external reader should still be available.

The good news is, the prevalence of USB will make the INTERFACE part of accessing old media painless. USB has proven to be able to grow, while maintaining backward-compatibility. USB-C adapters will be around for at least the next 30 years, since nobody seems ready to kill the Type-A port yet.

You only need to keep a reader around (or buy media that has a USB port onboard).

And 12cm optical media isn't going anywhere. The world will never be 100% wired, so disc media will continue to be a thing.

There's now four distinct generations of optical using the same size media, and that will probably continue when we need another jump in capacity. Just because the PC world has moved to download-only, doesn't mean the rest of the world will anytime soon.

You will still be able to buy a Blu-Ray drive 50 years from now. They might be more expensive than they are now (due to slowdown in demand), but you will be able to buy them. Consoles aren't shifting to download-only anytime in the next decade.

You just need to do you due diligence and copy your library to something new every decade-or-two. All media wears out over time, so this is not a new issue.

Glorious wrote:

I just digitalized a VHS from 1985, 32 years ago. The quality did not appear to be meaningfully degraded to my untrained eye, however, the lubrication on the tape itself clearly had some issues: I didn't have any overall obvious signal integrity issues (it even looked much better than I expected!), but I had semi-severe mechanical ones!

I did the same a few years back (VHS and 8mm). While some tapes still looked pretty decent, many were beginning to show serious degradation, and a couple were quite poor. Like with optical media, I'm sure alot depended on the brand, not to mention whether we recorded and then re-recorded on the same tape multiple times (which I know was the case in some instances). Another 5 years, and the "poor" tapes probably would have been lost.

Since it seems to be pure chance and a big unknown on whether tapes hold-up, I'd recommend anyone who hasn't digitized their old analog home videos to do so as soon as possible.

just brew it! wrote:

Get a hot swap bay or an external drive dock, and pretend that bare HDDs are removable media.

I want the media to be passive, not something that can spontaneously cease to function.

defaultluser wrote:

There's now four distinct generations of optical

What's the 4th?

meerkt wrote:

defaultluser wrote:

There's now four distinct generations of optical

What's the 4th?

Ultra HD Blu-ray

That's not a real generation in my eyes, no more than DVD-R DL or 80min CD-R.

meerkt wrote:

That's not a real generation in my eyes, no more than DVD-R DL or 80min CD-R.

HD-DVD probably would count if anything used it - the physical format was different from DVD-9.

meerkt wrote:

That's not a real generation in my eyes, no more than DVD-R DL or 80min CD-R.

I almost agree, but it's not backwards compatible, so that qualifies enough for me.

Vhalidictes wrote:

meerkt wrote:

That's not a real generation in my eyes, no more than DVD-R DL or 80min CD-R.

HD-DVD probably would count if anything used it - the physical format was different from DVD-9.

That would be 3rd generation, the same as Blu-Ray. I still have ~20 movies on HD-DVD and the external drive, which still works in Windows 10 as well as the Xbox 360 I originally bought it for.

G8torbyte wrote:

I read an article this morning about scientists warning of a digital dark age with current digital media at risk of disappearing over time. I always thought optical media would last longer than magnetic if taken care of and not scratched up but they say otherwise. They mention synthetic DNA technology could be a solution for the longer term and that “A single gram of DNA could, in principle, store every bit of datum ever recorded by humans in a container about the size and weight of a couple of pickup trucks.”

This isn't anything new... just to give a real world example, when I was in college I used an Apple IIGS computer for just about everything... the OS and my utilities were all on 3.5" floppy disks. Over the time I owned that machine, I had several disks go bad... you'd pop one in and all of a sudden you'd get read errors or whatever. So I made a habit of duplicating my disks at the beginning of every semester, and always keeping spare copies of anything important.

Magnetic media can go bad... even optical can go bad. Flash can definitely go bad.

If you're really concerned about the data, there are relatively easy things you can do to protect it. The most obvious is to make copies. If you're worried about the data on an old HDD drive, put it on a new HDD drive. If you're worried that a flash will go bad, copy it to a new flash. If you're worried that a device is no longer going to be supported, copy it to a more recent device. While it's true that interfaces change, often in just a few years, they don't just suddenly go away; there is always some overlap. And the good news is that as technologies change, the trend is towards higher capacities and greater performance.

You can also let someone else worry about the details, as technologies change, by using cloud storage. Better yet, do both - keep your own copies and cloud copies.

In short, if you can't be bothered to take steps to protect your data, you risk losing it... but that's true no matter what medium you choose to keep your information. Paper copies can be destroyed; so can microfilm or the various other ways information is stored. With digital there is an additional twist in that the medium itself can degrade over time; but again, the solution is fairly straightforward; make copies.

meerkt wrote:

just brew it! wrote:

Get a hot swap bay or an external drive dock, and pretend that bare HDDs are removable media.

I want the media to be passive, not something that can spontaneously cease to function.

You're pretty much limited to LTO tape these days then, if you want reasonable cost/byte for the media. Unfortunately the drives are really expensive.

cphite: The potential problem here isn't a single person's backup strategy. It's the collective records of an era.

jbi: Tape is no fun.

meerkt wrote:

cphite: The potential problem here isn't a single person's backup strategy. It's the collective records of an era.

You want archival?

Written records - Ink on 100% rag paper/parchment/vellum or polyester-based microfilm.

Music - Vinyl.

meerkt wrote:

The potential problem here isn't a single person's backup strategy. It's the collective records of an era.

Even if this doomsday situation pans out, I think we're still doing MUCH better than previous eras regarding data retention.

DPete27 wrote:

meerkt wrote:

The potential problem here isn't a single person's backup strategy. It's the collective records of an era.

Even if this doomsday situation pans out, I think we're still doing MUCH better than previous eras regarding data retention.

Yeah, that's how it feels to me as well.

I've had a few burned discs I left in my car since all it has is a CD player and non HD radio. Those discs have been under the seat for probably a decade, and when we're talking about Texas triple-digit heat I used to find it surprising the things still play fine when I'm driving too far out to pick up radio stations.

This whole thread explains exactly why the tape market will live another few decades at least.

DPete27 wrote:

Even if this doomsday situation pans out, I think we're still doing MUCH better than previous eras regarding data retention.

All these articles assume paper is more durable than HDDs, and definitely more than flash. Paper doesn't need periodic copying to new media to stay readable.
Optical... I don't know. -ROM seems durable, but who knows what happens after 100 years. -R, although proven for 20+ years with good quality organic dye media, seems less certain.

meerkt wrote:

Paper doesn't need periodic copying to new media to stay readable.

Again, assuming acid-free, high-quality paper stored in low-light conditions with optimal humidity. In some respects vinyl records and magnetic tape might be equally durable with proper storage conditions, we just haven't been using either one for enough centuries to prove the point.