Author The State of Data Storage  (Read 1471 times)

0 Members and 1 Guest are viewing this topic.

The State of Data Storage
« on: August 27, 2014, 08:44:48 AM »
I recently saw an article on Tom's about Seagate beginning to ship 8TB drives to "key customers." Just thinking about a drive that holds 8 terabytes of data is staggering to me if I look back at some of the machines I've owned. The first five had (respectively) 16MB, 1.5GB, 40GB, 60GB, and 120GB of drive capacity. Now they're shipping something that holds 500,000 times what my first PC did.

Look at the growth in file sizes compared to that. I don't feel like file size has grown at quite the same rate as drive capacity. Basically, it seems we're becoming able to store more and more stuff.

Which brings the question: where does it end? Most data in some way, shape, or form is an electrical or digital signal that's stored somewhere. The cloud is still made up of physical servers at the most fundamental end. Something has to power all this.

My big question is where is data storage going? Yes, cloud this and cloud that. But what constructs the cloud? Beyond hard drives and flash memory, how are our descendants going to store their data?

A few starting points: coding into DNA strands and quantum computing. In some far-off future, will our data be coded into massive cell banks that exist in space? Will we take the cloud to yet another extreme? Or will our data become tucked into one of the other dimensions of matter?

The State of Data Storage
« Reply #1 on: August 27, 2014, 08:51:15 AM »
i remember the appleII hard drives. i think they were a whopping 5MB or something. in the era of 5.25 floppies (that were brand new at that time) that was a huge storage improvement.

The State of Data Storage
« Reply #2 on: August 27, 2014, 10:31:30 AM »
If any of you are interested, I have one of these I am willing to let go for a fraction of the original price.


The State of Data Storage
« Reply #3 on: August 27, 2014, 10:55:17 AM »
Remember using cassette tapes to upload files?
http://en.wikipedia.org/wiki/Commodore_PET

The State of Data Storage
« Reply #4 on: August 27, 2014, 11:54:13 AM »
You ain't old unless your first computer's drive had KB after the number.

The State of Data Storage
« Reply #5 on: August 27, 2014, 12:44:21 PM »
You ain't old unless your first computer's drive had KB after the number.

mine had a B after the number

The State of Data Storage
« Reply #6 on: August 27, 2014, 08:14:24 PM »
I had one of these:


The State of Data Storage
« Reply #7 on: August 28, 2014, 04:28:21 AM »
I had one of these:


Doing programming with a membrane keyboard is programmer hell.

The State of Data Storage
« Reply #8 on: August 30, 2014, 09:52:14 AM »
I don't know that I can compose a sufficient post about this subject, but I have a few thoughts to share.  I work in media, an industry that relies entirely on tech.  Storage media, storage capacity, processing power, etc. Even though the developments in these things seem fast in retrospect, on the front lines it almost feels like growing pains.  A slow trudge towards things being adequate for what you really need to do.  We are still fairly backwards in terms of our storage media.  The best case scenario in the immediate future is a widespread shift from magnetic drives to solid state (ie. no moving parts, instant access and loading, this is like flash memory on a thumb drive or video game cartridge).  Also hard to discount optical media since we are just experimenting with holographic reading and writing, which while carrying forward the problems traditionally associated with optical and magnetic drives, bring the benefit of massively increased storage capacity.  We're talking several TB on a single six inch disc.

The cloud is already here.  It's hard to say if it will ever make the jump to being a primary storage source not just for personal use but also professional and industrial use, but a few more leaps in internet speed and internet ubiquity and we are pretty much there.  Doesn't mean other things will cease to exist or cease to be used, even in a digital world we hold fast to paper files and information.  There will always be use for and value in physical media. 

It's my opinion, and I speak about it whenever I have the chance, that a move to only digital content, storage, and distribution would be a disaster and a mistake for us.  For me it brings to mind Ray Bradbury's Fahrenheit 451.  If we allow everything to be based on the cloud, we inbuilt a future for ourselves where the government or in actuality, the oligarchy, instead of burning physical books, media, information, history, could simply alter or erase them completely with a tap on a touch screen.  That is not a future I want to live in as a writer.

Luckily, I think like most technological advances, it's going to be longer than we expect for it to actually be upon us.  The near future is one of physical computing and storage, more accurately quantum computing and storage.  When we can store things by manipulating individual atoms rather than groups of them, there will be a boom in terms of storage capacity to hardware size (hopefully). 

Sorry if any of that seems rambley, just an early morning jumble of thoughts on the subject.

The State of Data Storage
« Reply #9 on: August 30, 2014, 10:03:33 AM »

It's my opinion, and I speak about it whenever I have the chance, that a move to only digital content, storage, and distribution would be a disaster and a mistake for us.  For me it brings to mind Ray Bradbury's Fahrenheit 451.  If we allow everything to be based on the cloud, we inbuilt a future for ourselves where the government or in actuality, the oligarchy, instead of burning physical books, media, information, history, could simply alter or erase them completely with a tap on a touch screen.  That is not a future I want to live in as a writer.
I agree, aside from the technological risks (recall the degrading of laser discs) the threat of large companies or governments with digital, particularly "cloud", storage and formats is amazing. Imagine the ability to know what everybody is reading, the ability to edit (or delete) any book or article, etc.

The State of Data Storage
« Reply #10 on: August 30, 2014, 12:01:58 PM »
If any of you are interested, I have one of these I am willing to let go for a fraction of the original price.



Love the physical key lock. Good luck if you lose that key haha!!!

The State of Data Storage
« Reply #11 on: August 30, 2014, 12:25:11 PM »
Love the physical key lock. Good luck if you lose that key haha!!!

One of the things I like about that ad is that the install kit alone costs about eight times what a 1.5TB drive (with a MILLION times more storage space) goes for these days.  And that's not even correcting for 1992 dollars compared to today.

To put it another way, looking at drive storage capacity alone, in 1982 you would have had to spend 2.5 billion dollars to have the same storage space you can get today for less than $100.

The State of Data Storage
« Reply #12 on: August 30, 2014, 12:43:34 PM »
I had one of these:


Our first "real" computer was an Atari 1040ST; bought for the kids, of course.  Very capable little machine...did most everything a Mac did for 1/3 the price. Had GIU and a mouse years before the first MS Windows machines.   I took it to work for some projects and the DOS guys were speechless at it's capabilities.  Saddled with the Atari name, distribution via dept. stores, and a proprietary OS it never had a change at being taken seriously.

The State of Data Storage
« Reply #13 on: August 30, 2014, 10:09:18 PM »
My first comp., back in the early 80's, was an IBM PC XT (That's for eXtended Technology, man!), with 10Mb hard drive.  I think the drive itself probably cost at least $500 at the time.  Now you can get a 16Gb flash drive off Amazon for like 10 bucks.  That's like roughly 1500 times increase in capacity for like 2% of the price, if I did the math right here after my first couple beers.  In any case it's a pretty staggering advance after only 30 yrs. 

[attachimg=1]

The State of Data Storage
« Reply #14 on: August 31, 2014, 07:28:49 AM »
It's my opinion, and I speak about it whenever I have the chance, that a move to only digital content, storage, and distribution would be a disaster and a mistake for us.  For me it brings to mind Ray Bradbury's Fahrenheit 451.  If we allow everything to be based on the cloud, we inbuilt a future for ourselves where the government or in actuality, the oligarchy, instead of burning physical books, media, information, history, could simply alter or erase them completely with a tap on a touch screen.  That is not a future I want to live in as a writer.
I agree, aside from the technological risks (recall the degrading of laser discs) the threat of large companies or governments with digital, particularly "cloud", storage and formats is amazing. Imagine the ability to know what everybody is reading, the ability to edit (or delete) any book or article, etc.
Excellent (and rather frightening) points.

Another thing to consider is the degradation of digital data over long periods of time.

Pressed or "replicated" CDs are starting to show their age. However, my oldest discs will still play back fine. Burned or "duplicated" discs last much shorter. I have many DVDs and CDs from ten years ago that will no longer read. (Yes, there's something to be said about media quality and burn speed...)

Over time, physical hard drives begin to experience issues mechanically. SSDs have shown to have corruption issues. So what from there? Must we continuously be backing up our digital data? What happens as more data is being created and stored? Will we reach a tipping point where it's just too overwhelming?

Now look at some older options. We have paper (or paper-like) documents spanning back ages. Stone tablets and wall etchings have held up thousands of years. Cave drawings, too. Maybe the golden ticket is avoiding the cloud and putting our money on something physical. I like the idea of quantum storage via atoms.

One other thing to think about. As we look into long-term data storage options, we must also consider technology that will be used to read/write to these media types and maintaining it's constant compatibility. I've had clients come to me with 3.5" floppy disks. Sure, their data on the floppy may still be very readable. But the disk is useless because we have no way to read it.

The State of Data Storage
« Reply #15 on: August 31, 2014, 07:39:23 AM »
My first comp., back in the early 80's, was an IBM PC XT (That's for eXtended Technology, man!), with 10Mb hard drive.  I think the drive itself probably cost at least $500 at the time.  Now you can get a 16Gb flash drive off Amazon for like 10 bucks.  That's like roughly 1500 times increase in capacity for like 2% of the price, if I did the math right here after my first couple beers.  In any case it's a pretty staggering advance after only 30 yrs. 

[attachimg=1]
We're all riding in my mini van
two drink holders and a captains chair
Electric windows electric doors
Four or five people sitting comfortably

The State of Data Storage
« Reply #16 on: August 31, 2014, 09:32:09 AM »
Excellent (and rather frightening) points.

Another thing to consider is the degradation of digital data over long periods of time.

Pressed or "replicated" CDs are starting to show their age. However, my oldest discs will still play back fine. Burned or "duplicated" discs last much shorter. I have many DVDs and CDs from ten years ago that will no longer read. (Yes, there's something to be said about media quality and burn speed...)

Over time, physical hard drives begin to experience issues mechanically. SSDs have shown to have corruption issues. So what from there? Must we continuously be backing up our digital data? What happens as more data is being created and stored? Will we reach a tipping point where it's just too overwhelming?

Now look at some older options. We have paper (or paper-like) documents spanning back ages. Stone tablets and wall etchings have held up thousands of years. Cave drawings, too. Maybe the golden ticket is avoiding the cloud and putting our money on something physical. I like the idea of quantum storage via atoms.

One other thing to think about. As we look into long-term data storage options, we must also consider technology that will be used to read/write to these media types and maintaining it's constant compatibility. I've had clients come to me with 3.5" floppy disks. Sure, their data on the floppy may still be very readable. But the disk is useless because we have no way to read it.

Longevity and standardization are huge issues for preservation.  If you want an even better example of this than floppy disks, think about video games.  In the video game industry, there are always multiple machines out by multiple companies, each with a proprietary media format, and this has been going on for decades already.  Can't pick up a Nintendo game from 1986 and bring it home to play on my Wii, can't play an Xbox disc in a Playstation, etc.  This is like the floppy disk, even if you have the content intact, we are rapidly losing machines that are capable of reading it.  That leads to a bigger problem where large amounts of people are cut off from accessing media and information, possibly their own data. 

Most industries are caught in this cycle of updating their content for new formats.  Films are constantly restored, transferred and re-transferred.  Albums have to be remastered and reissued.  Video games have to be ported to new hardware, entailing either a complete recoding to make it compatible with a new hardware infrastructure, or the development of complex emulation software to run it on that hardware.  Of course, only the popular stuff gets carried forth, meaning we are losing many thousands, probably millions, of games, movies, and albums (not just these, but all media forms really) to this forward march.  And that's in addition to what you mentioned about the comparison between digital devices and things like paper and stone, our hard drives aren't going to survive much less be readable for as long as those things have.  What we face should we fail to get a handle on that is the modern equivalent of the burning of the library of Alexandria.  Another scary thought to add to this.

But I agree with you, support physical as much as you can.  Digital and physical storage, cloud and localized storage, are flipsides, they support and drive one another.  We need both.

The State of Data Storage
« Reply #17 on: September 03, 2014, 12:58:38 PM »
To your main question: There is no end... it simply grows.

I started with a Sinclair ZX and yes it loaded off of a cassette tape. Then an Apple IIgs and Mac in high school. Then the x86 IBM / Gateway 2K and so on. My Gateway 2K in 1997 had a 3.2gig hard drive... for that time it was monstrous and 4gig was the only bigger size available. Today I have a home server with 10 terabytes of storage for A/V stuff. In another 5 years, I fully expect that to be 100tb.

Look at how much data is being created every day. There is no end to storage disk. There will eventually have to be a new storage technology to replace SSD and mechanical disk. But for the next 7 years, that is what you will be using. We are at least 10yrs from a new consumer tech for storage. In 7yrs, major data players like Google will have access to next-gen disk.

The State of Data Storage
« Reply #18 on: September 03, 2014, 03:46:52 PM »
TRS-80 Model 1.  I think it was 8K.  Maybe not that much. Before that, it was a Texas Instruments scientific calculator that you could program to run sequences of operations.