Ever wondered WHY we are in a Digital Dark Age?
It's much simpler than it seems.
As I always say, YOUR civil rights and the quality of YOUR life depend every year more on how software and digital data are used AROUND you. This begs the questions, which are today's topic, "Are those data really so terribly fragile and ephemeral as they seem, and if they are, what should we do about it?"
This is NOT a post about huge electromagnetic storms or EMP pulses by nuclear explosions that may destroy every digital devices and data. Never mind them. Should such events happen, either they wouldn't be so bad, or your data would be the last of your problems (1). This is about the digital dark age made of technical fragility, secrecy and decadence we already live in, every day, because we keep failing to see its real nature and causes.
The problem, as perceived
Yes, our lives depend on equipment, services and data that may too easily break or disappear any moment precisely because they are "digital". But what makes the core problem really bad is just the fact that, while it is relatively well delimited and easy to define, it takes so many different forms that most people struggle to see it... if and when they realize certain forms (still) exist.
Floppy drives, for example, are fifty-years old digital technology that for most people is as dead and distant as phone booths. But did you know that the German Navy has started to replace the floppy drives inside its anti-submarine frigates ONLY THIS YEAR? Or that Japan managed to free itself of more than one thousand regulations that mandated the usage of floppy disks ONLY THIS YEAR?
Did you know that floppy drives are still essential to load data and software programs in many other public and professional contexts, including:
older industrial machines and equipment, CT scanners, ultrasound machines and other medical devices
the San Francisco Muni Metro light railway, whose mission-critical Automatic Train Control System will boot up with a floppy, every morning until... 2033/34
older airplanes, which remain among the major users of floppy disks in the 21st century
Why is that so? I'll get back to that in a bit (of course), because a bigger digital fragility problem, which still goes largely unnoticed, is the low durability of consumer grade optical media. Many music CDs from the 1980's or 1990's, including yours, "aren't just aging, they're dying". Ditto for movies or video games on CD and DVDs, including FACTORY SEALED ones. What about YOUR CDs and DVDs? Do they show anything like "bronzing of discs, small pin-hole specs located on the discs, or “edge-rot"? Check them, and let me know in the comments!
"The cloud" isn't any better
The Web and Cloud storage were supposed to solve both the evident physical limits of distributing physical documents, and the invisible fragility and ephemerality of digital ones. Instead, as we let them grow unattended, Web and Cloud brought us the worst of both worlds, by marking the end of the end of files as individual objects that could a) live as long as they really could and b) controllable, or at least surely readable by their owners or viewers.
Link rot is a growing digital pandemic: in 2021, over 60% of older links in the New York Times articles still accessible online were already broken. Web-wise, 38% of webpages that existed in 2013 were no longer accessible in 2023, and 23% of news webpages contained at least one broken link.
Google makes things even worse. After years of deliberately forgetting perfectly working websites just because the weren't "new", one month ago Google stopped indexing altogether (as in "practically erase from the Web") millions of still working web pages with unique content, just because they were published before mobile phones, but their authors will never have the time or money to "optimize" them for such devices.
The situation is so bad to make E. Mollick argue that "only LLMs will remember the web, since our own knowledge is rotting", and then wonder "how many people would be willing to license unlimited LLM training on their content in order to ensure that their work is “remembered,” in some way, somewhere".
Hell, no, I say. Even without LLM hallucinations, if "the cloud is just somebody else's computer" LLMs would be just somebody else's memory. Somebody who could stop talking to you any moment.
Private files stored online are rotting as much as public web pages, albeit in a slower and very different way. This is because (source) “files are discrete objects that exist in a physical place [but] the internet is pretty much the opposite of that".
Services like Google Docs do not let us sort or organize files in any way that (see above...) doesn't just“prioritize [only] the new over the important”, and by "us" I don't mean septuagenarians mourning the good old days of handwritten letters posted with just a 6c stamp: we already are in the second or third round of university STEM students for whom it is possible to struggle with homework because they cannot find their own files in their own computers, and of younger students in no better conditions, as we all saw during the 2020 lockdowns. Unsurprisingly, these are the same generations that, in order to "own" again the stuff they paid for, are allegedly going... back to "more permanent" vinyl and DVDs. Oh, my...
The REAL nature of "Digital Dark Ages"
Civilizations and individual humans cannot really exist without permanent memories of their own lives. How did we came to this? Mainly, because there still is a dangerous level of ignorance, at all levels from lawmaking to parenting, around all things digital.
Let Big Tech Toddlers with way more money than their maturity can handle chase immortality. As I wrote years ago, if digital files really disappear as discrete, permanent objects, it will be only OUR fault. What we should all make immortal is OUR DATA, not our bodies.
This IS possible. Let's take books and e-books as an example. People who go back to paper books because "iPads are distracting and paper is unmatched" or "don't own what they (NEVER, actually) "bought", instead of demanding open formats and copyright reform really miss the point. The solution to the current stupid constraints on (NOT "of"!) e-books is to demand e-books that will never expire, and nothing less (for even more examples, see here).
The truth is that paper books, printed photographs, and pretty much everything "non-digital" document storage technology, including clay tablets, is much more fragile and durable than bits.
"Father of the Internet" Vint Cerf, who is rightly worried that future generation will have little or no record of the 21st Century also pointed out that “modern photographs may not last more than 150–200 years before they fade or disintegrate. Modern books, unless archival paper is used, may not last more than 100 years.” All it takes to destroy one unique, "eternal" papyrus or clay tablet is one fire, one spilled cup of coffee by overworked, underpaid archivists or similar minor accidents.
Bits are the ONLY way to PRESERVE in just one, extremely small and overall really cheap "bit container", literally tons of content of all the possible kinds that we would NEVER, EVER have the space or money to preserve in any pre-digital format.
Even better, bits and bit containers are the ONLY way, both space- and money-wise, to make and keep MULTIPLE COPIES of the same content in different places, that is to NEVER, EVER LOSE any document after it was digitized.
THIS is the reality. Durability of paper or vinyl my foot. If anything, there is too little available digitization of non-digital stuff that matters. There still are mountains of potentially valuable "dark data" "unknown to most of the world, some of it never even seen by human eyes". The United States have lost complete, original version of 75% of their silent films. More than one-quarter of scholarly articles are not being properly archived and preserved.
Hardware, electricity, software... those are not REAL problems
Yes, if civilization fell back to any state that makes electricity or computer technology unavailable, it would become impossible to read, write and produce the "bit containers". But that's a dumb concern to have.
The reality is that fully interchangeable "bit containers" are the only physically possible way to preserve everything that should be preserved and keep doing a million more good things we can't live without anymore, and that a regress to pre-electricity or pre-computer ages would be a catastrophe anyway. So, the right thing to do is to avoid that catastrophe and manage bits in the right ways, not to give them up.
This said, the "right way to manage bits" does include worrying about hardware in the right ways, places and moments. After all, the real reason why why we worry about the accessibility of bits that, unlike with pen and paper, we need two high-tech things to handle them (it's actually three, but I'll leave the most important for last), and one of them is objects, from monitors to microprocessors, that can be manufactured only with very advanced, very expensive machines.
That's not a big problem, however, except in very special cases. Some experimental and rare films and video works, for example, are "very difficult if not impossible to digitize without losses". But those are exceptions.
If the average life of a hard disk is, say, five years, having two different disks for backups, and replacing them every 3.5 or 4 years will keep all your document and memories safe as long as ANY kind of computer exists. Surely much longer than any vinyl, paper or tape could, IF you had space for them.
Important: I say "hard disk" to mean "whatever storage media is more common in any given period". I have files from the 1990s that I copied from floppy to CD, then to DVD, then to USB drive... and will continue to copy on whatever kind of "bit container" comes next. Bits don't care what they live on.
Even when the only way to access some valuable content is one very specific piece of hardware, if the designs and specs of that hardware are available, it is technically possible to make another copy of that hardware that behaves exactly in the same way. The extremely expensive machines that still work perfectly but need floppies, for example, will not be replaced until they break, which is good considering how scarce of raw metals we continuously forget we are. However, making those machines believe that any cheap USB key of 2024 is a real floppy disk from the 1970s is easy, with stuff like this, because the design of floppy drives is completely known. That's what matters, as far as hardware is concerned:
The other obstacle to managing bits without worries is software. This is often the biggest problem, partly because software is still black magic for 99.9% of human beings and will likely remain such for the foreseeable future, no matter how many coding classes are laid on children, or adults that should be "upskilled" to do "work" that shouldn't be done.
Many real problems with software, however, only happen when that software is secret or uselessly complex. Unless they were made with Flash, the archives of websites made 25 years ago are bunches of files that still “just work”, those of websites made 25 months ago are useless without getting mad with software dependencies. Static blogs like this or this are a wonderful way to self-publish online, not to mention a mandatory backup for every Substack author.
The root problem: ignorance, at all levels
We joke that executives politicians are clueless about everything digital, but even the creator of Star Trek was clueless about digital preservation, and even an intellectual giant like Umberto Eco got e-books wrong. This is the real problem: too many people, at all levels, still sincerely believe bits will surely last much less than paper, vinyl or film, and that it could't be any other way.
The (mandatory) road ahead
A complete strategy to avoid "Digital Dark Ages" would be much longer than one post (2), but it should surely include two or three basic guidelines.
One would be to acknowledge that not everything needs preserving, or even created in the first place. Check out all the pictures you took: how many of them will still make sense in one or five years?
The second thing to do would be to learn for good that, since the dawn of culture, all human beings have always managed information by combining three very different things:
Physical Supports, that is some the material objects "hosting" the information
Data Formats, that is the rules by which the information is encoded on the support, and
User Interfaces, that is the tools used to write and read the data according to the format
This is what everybody should learn, and politicians should enshrine in laws:
Starting yesterday, digital interface, supports and formats must return and remain as separate as they were in the Bronze Age, possibly more. To see what I mean, have a look at these screenshots of my evergreen seminar on file formats (2):
and then check the full thing, especially slides 8 to 13 and the "bit containers" part.
It doesn't matter if the "data format" is handwritten language, or a set of rules to decode a sequence of bits. What matters is only that the formats are completely open, to allow new software programs to keep handling them. Even the Vatican says so for their archives. This is hardly surprising for me, considering Free Software’s surprising sympathy with Catholic doctrine, but never mind me: when it comes to permanence and durability, who should you trust more than an institution that's lasted millennia and has every intention to last even more?
Two-sentence executive summary: The real problem of "Digital Dark Ages" is the same that Japan got partially rid of just this year: not technology, but laws and regulations that tolerate, when they don't impose, closed file formats, dumb practices like DRM, or ignorance about these issues. Get rid of those, and your bits will last forever.
Finally… the more direct support I get, the more I can investigate and share content like this with everybody who could and should know it. If you can’t or don’t want to do it with a paid subscription, you may fund me directly via via PayPal (mfioretti@nexaima.net), LiberaPay, or in any of the other ways listed here.
NOTES: 1. to put solar storms and EMPs in perspective, read here, here, here and here 2. Wanna hire me for tutoring, popularization or research on those topics? Just email